I have been practicing and teaching Scrum (and variants thereof) for nearly 10 years of a 23 year career in Information Technology, and I still find myself frequently learning new things about it.
In my role as an independent agile consultant, coach and trainer, I work with lots of different organisations who are trying to be successful using Scrum (or at least something resembling it). Many of the folks I encounter have received only a few days at most of training, little to no coaching, and are unsurprisingly struggling to get maximum benefit from the way they are working. In my conversations with them it becomes clear they lack even the most basic knowledge about what Scrum is or why they are using it.
When I sense a void of understanding about a topic in myself or those around me, those who have worked with me will know I’m a big fan of boiling complicated, down-in-the-weeds types of things into simple, helicopter views.
And this is what I am attempting to do with this article.
Please provide me with feedback on what you feel might improve the article, along with additional questions to pose and answer. I’d love to push it toward becoming a comprehensive, even definitive FAQ on Scrum.
And while you’re at it, I invite you to please check out my popular sister post: Agile FAQ.
Scrum is a framework for developing products (or, more generally, things of value) in complex domains and environments.
It is a proven approach for building high value products in an iterative and incremental way using teamwork and empirical process. Perhaps for this reason, it is probably the most widely used framework in software development across the world (at least in name, and at the time of writing).
The full definition of Scrum can be found in the official Scrum Guide.
You don’t “need” it, per se.
But to be successful at building products and services which generate value for your customers and your business in rapidly changing circumstances, you do need a way of managing risk, optimising value and navigating complexity in both the work and the surrounding environment. Scrum provides a framework for doing that.
I am framing “waterfall” here as referring to the traditional linear systems development lifecycle (SDLC) approach to software product development — characterised by Big Design Upfront (BDUF), linear / non-parallel phases, functional teams with hand-offs and documentation milestones.
While, from a pure delivery point of view, “waterfall” has its benefits (and could even be deemed “fit for purpose” for certain types of work and environments), any approach to development which does not explicitly include or encourage frequent feedback loops for the product and the process is at a high risk of not maximising value due to:
Note — You can substitute the word “market” for “customer” above, and the same applies.
Modern Scrum is heavily influenced by the Toyota Production System (TPS). This holistic delivery approach (as opposed to the phased model described earlier) started to gain traction in software and other non-manufacturing product development following a 1986 Harvard Business Review article called “The New New Product Development Game”.
Scrum as we now know it (created by Ken Schwaber and Jeff Sutherland) followed just a few years later.
You can read more about the history of Scrum on its Wikipedia page.
Putting aside the questionable name (given the desire for building high quality products at a sustainable pace) — “Sprints” combine two key concepts: time-boxing and short feedback loops.
Short (typically 1–2 week) time-boxes can help you:
You cannot turn back time, but you can always do more stuff if you have more time. By making each time cycle (aka Sprint or Iteration) very short, you have a chance of maximising the value of the overall time you have available. Or, put another way, you are minimising the risk of using more time — or doing more work — than you need to reach an outcome.
If, after a Sprint, you need to do more work, and that “more” is still the most valuable stuff to do, with the most valuable potential outcome, you can use the next Sprint. If you hadn’t run a Sprint, or made it short, you might not have exercised the option to pivot or persevere, or even known you had one. You might have persevered, blind to better information, or even despite having it.
Development Team — Build the highest value product possible by working together in the most effective way possible
Product Owner (PO) — Steer the Development Team toward delivery of the right things to maximise the value of the product
Scrum Master (SM) — Steer the whole Scrum Team toward becoming increasingly effective in theirs roles and responsibilities, thus increasing their chances of maximising the value of the product
For more detail about the activities and behaviours required by each role, read the Scrum Guide. Also check out my Scrum Master 1-pager.
Given Scrum is about maximising value within the constraints you have (budget, time, skills in team, etc.), the best way to do this is to follow the Scrum Guide’s suggestions closely and ensure the theory of empirical process is being applied (inspection, adaptation, transparency) in short cycles, underpinned by a team focused on goals/outcomes over output and living and breathing the Scrum values of Focus, Openness, Respect, Courage and Commitment (also found in the title graphic for this article).
Scrum doesn’t tell you how to estimate, so use what works best for the team. The key thing is to create transparency about the work remaining in both the current Sprint and for any larger outcome you are trying to achieve. Some teams estimate backlog items in ideal days, others use t-shirt sizing, story points or elapsed (cycle) time.
However, whichever the estimation approach used, be mindful that Scrum is an empirical approach, meaning Scrum teams use observation about what has happened (aka “yesterday’s weather”) to make decisions about what to do next. This is contrary to a predictive approach, where decisions are made based on what we think will happen.
In complex environments, making accurate predictions (at least with a useful level of precision) is very difficult, so it is far less risky to use an empirical approach than to ask teams to estimate years, months or even weeks into the future.
However, Scrum teams do forecast the work they will accomplish within a Sprint (as a byproduct of creating a plan called the Sprint Backlog), and then use historical data (in combination with the current state of the product and Product Backlog) to make forecasts (if needed) about e.g. what “stories” might be in a release, or when “x” might be complete.
Aside from all the other changing circumstances you most likely deal with in your working environment, the scope of work can (and most likely will) grow as your understanding of what needs to be done grows, hence why the term “forecast” is used in Scrum when talking about Sprint output. Scrum teams DO NOT “commit” or promise to deliver x number of features in y timeframe (or, at least, no one should be making them do so).
Uncertainty grows the further out you are trying to predict, and committing to the delivery of particular features in a schedule reduces agility, i.e. it makes it harder to “welcome changing requirements” or change tactical direction based on what you learn in the feedback loops. You are setting scope/date expectations with people rather than collaborating on the best path for building the highest value product.
There is an overwhelming amount of literature about software estimation in and out of Scrum, but you might like to start with this excellent article which I came across just the other day, or my own article about an alternative approach to using story points.
In order to maximise value, the PO must continuously order the backlog by value, and the Development Team must actually build the top items (in short Sprints) to derive said value.
Thus the sequence of development is governed by value. “Value” is the guiding direction for ordering the backlog, but you need to define what that means in your context. It might mean reducing risk, making short term revenue, enabling longer term revenue, entering a market, creating awareness, learning something important, doing social good, helping someone, etc.
Given the items at the top are the ones you (should) want to develop next (or soon), you need to be more granular, detailed and clear with those items (using techniques such as story slicing and acceptance criteria). There is only so much a team can accomplish in a sprint or number of sprints, so it makes sense to deliberately defer implementation of (and conversations about) items which are lower down the Product Backlog (or, more accurately, defer decisions about whether or not to implement those items, and how to do so if you choose to).
You have limited information now, and will have more later. You may learn about different ways to deliver higher value before you come to implement the items you currently think should be implemented. Thus it makes sense to minimise the amount of time you spend thinking about those lower items right now and make them broader in concept, with fuzzier detail.
Firstly note — while user stories are an extremely useful tool, they are not part of Scrum. The concept of “stories” as a way of capturing user requirements originated in Extreme Programming (XP), but “user stories” are an example of a complementary practice to either XP or Scrum (other examples are “3 amigos” and work-in-progress limits, to name only two of hundreds).
That said, user stories are a very useful way of describing our Product Backlog Items (PBI’s) because they are centred on a human being (a customer), and a capability they desire which will likely bring them value. Describing your work in this way makes it easier to slice (create narrower, independently valuable options) and prioritise (decide what to do now and what to defer).
Also note — stories (in this context) are not written, they are told. A user story is actually the promise of a conversation with a user about a desired capability. The magic is in the conversations, not the “writing”. We only write down stuff we might not remember, or want to pass on to future users or developers with whom we can’t have conversations (for this reason, using acceptance tests to document the “stories” in the product is the preferred approach of an agile team; these “tests”/”checks” not only describe the capabilities present in the product but also serve as a powerful way to ensure you are enabling and checking/testing the correct functionality).
Please Google and read about story slicing, story mapping, ATDD (acceptance test driven development), specification-by-example, “3 amigos” and BDD to learn more about stories, refinement and agile “testing”.
In both Agile and Scrum there is a focus on continuous improvement. This involves constant attention to removing barriers to success, both at the team and organisation level, including any wasteful steps or other impediments which slow you down or inhibit you from delivering value in a sustainable way.
If you are continually looking for ways to make your process and product better, then it follows that you are doing everything you can to become as efficient, productive and engaged as you can possibly be, and will likely make significant improvements.
The goal of every Sprint is to complete a potentially releasable product “Increment”. The team defines a shared standard of what “Done” looks like because this creates better transparency for everyone in terms of what progress and quality mean.
In order that the team can complete an Increment in every sprint, they need to identify a Definition of Done (DoD) which enables that. If the DoD is too aspirational, the team will be too focused on future quality needs over current, and not enough time will be spent on actually delivering a product Increment. If it is not aspirational enough, the Increment will not deliver as much value as it could, e.g. it is not on a customer-usable production environment, or it lacks sufficient automated regression checks or user documentation, etc.
So it makes sense to gradually evolve the DoD, Sprint by Sprint, to improve the quality and thus value of the product being built. Scrum formalises this activity in the Sprint Retrospective, but it does not need to be confined to only then.
This is the role of the Scrum Master (SM). The SM needs to have lots of experience, skill and techniques in a very wide range of things in order to do the role well. Aside from being an expert in Scrum (and Agile) itself, SM’s need to keep levelling up in areas such as servant leadership, coaching, mentoring, influencing, facilitating, building safe environments, leveraging collective wisdom, conflict resolution and much more.
My “Scrum Master 1 pager” gets folks started with learning about what this role entails, but the real learning is in all the further reading along with the doing, trying, failing, succeeding then failing repeatedly, over and over again.
Scrum teams do not look at any piece of work as owned by an individual. The team collectively owns all work they do. They learn to work well as a group of individuals with different specialities, needs, backgrounds, experiences, personalities and other attributes, but aligned around a common goal. This is called cross-functionality.
If something is blocking progress, the team calls it out, then figures out how to remove it (and, ideally, stop it from coming back).
Scrum can be applied to any project where the goal is to maximise the value of something (a “product”), and thus requires short feedback loops in order to govern what actually gets delivered and ends up being valuable. Research fits nicely in this category because, by definition, it is about learning something in order to achieve a broader purpose.
Define a vision about what you want to achieve, define a Product Backlog of valuable deliverables (including learning outcomes), define your first micro goal and start delivering in Sprints.
A time-box isn’t a true deadline (unless it happens to coincide with an actual deadline, a date or time after which there is a significant negative impact if an outcome has not been delivered), rather it is a self-imposed constraint. There are good reasons for doing this, as I explained earlier.
In a Sprint, the Development Team controls how much work they will try and accomplish, but they often still feel the pressure of the time constraint, causing stress, panic and compromises to be made (this even happens in a simulated setting such as my Scrum training!).
The trick is to identify the simplest thing to deliver, then deliver in very small pieces such that it is easier to reduce scope when you realise you will not get everything done you thought you would. Visualise the work accomplished and remaining to make it easier to understand progress. Adjust the plan (aka the Sprint Backlog) if the plan no longer makes sense.
Yes! Development Teams that do not communicate frequently with the PO (or customer) tend to struggle or be at risk of building the wrong thing.
The PO is part of the Scrum Team. Work closely with them. Ask them questions when things aren’t clear, or even when they are (assumptions are silent killers). Check that you are on the right path. Negotiate scope when it feels like things are off-track. Ask if there is a simpler way to meet the goal.
There is no right answer to this. You need to find the right balance in your own context between healthy communication and collaboration vs losing focus and context switching due to interruption.
A Product Owner disrupting a developer during a Sprint to “get an update” on a task is a great way of adding risk to them not accomplishing the Sprint Goal. An even better way is to add work into the Sprint. You’ve spent time prioritising the highest value work, the team has planned it and is building it, and now you have decided to confuse the issue by adding work which we didn’t prioritise in the first place! (Note — This also violates a core principle of Scrum; that only the Development Team decides how much work they can accomplish in a sprint. A healthy Scrum team collaborates and negotiates scope based on empirical process.)
Agree as a team how you would like to work, including how to balance individual and team needs when it comes to interruptions and other productivity killers. But don’t forget that great products are built by teams, not individuals, so you do need to prioritise collaborating with your teammates over being uninterruptible.
Hopefully this article has answered some of your questions about Scrum. If you have any more, please ask them in the comments and I will add the question and answer to the article.