Should We Estimate Software Projects… At All?

Tweet about this on TwitterShare on FacebookShare on LinkedInShare on Google+Email this to someone

estimationIntroduction

After a year or two of “having a hunch” about this, and after many years of either estimating work or working to someone else’s estimates, I’ve now finally come to the conclusion that the use of estimation of any kind in a project is not only a waste of time but is actually destructive.

I am fully aware this is an extremely controversial statement, so I am going to be as thorough as I can in explaining how I came to this conclusion via experience, data and validation. Indeed, when I read Vasco Duarte’s post about this several months ago, I saw his “point” (no pun intended) but also argued the merits of using story point estimation for the purposes of:

  • Up-front sizing of a project to determine its validity within a given budget or timeframe
  • Increasing shared understanding and knowledge within the team based on the discussions that arise from a Planning Poker session
  • Allowing the PO to make trade-off decisions between different sized stories (based on ROI)
  • Measuring team velocity
    • To continually validate the initial project sizing by predicting scope-fit within a given release date
    • To allow the team to measure and improve its performance

Why shouldn’t we estimate?

I have since come to the conclusion that some of these things do not need to be done at all, and the other things can be done without the need for estimating (guesswork) of any kind. I would now additionally argue that even if you acknowledge the shortcomings of estimation and use ranges, account for uncertainty, etc., the act of estimation in itself is destructive for the following reasons:

  • “Fixed” scope project delivery expectations are often (always?) based on an up-front estimate of scope (guess) and how long that scope will take to be delivered (another guess), leading to the obvious dysfunctions like death-marches, low quality, etc.

If the budget is fixed, there is no way of going “over budget” in order to deliver the fixed scope. Yet “over budget” is a common term used when describing failed projects. If your budget is truly a constraint then you will only deliver what can be delivered. Agile methods ensure that what you deliver is of the highest value to the business.

I chatted to a team member earlier and he complained of feeling pressure to increase velocity. I asked him where this pressure was coming from and he said that it stemmed from the concern that the project will fail if the team isn’t able to deliver more stories more quickly. No one is actually specifically asking the team to deliver more, but there is an implied pressure to do so because they are aware the budget is running out. This mindset comes from years of poorly funded, gated projects, death marches, focus on productivity rather than quality and canned or failed projects.

  • Asking teams to estimate how long their work will take (or how many points they will deliver in a Sprint or a Release, same thing) has connotations that their output is being measured by an external party (manager), creating an environment of fear and massaging figures to reflect what is desired rather than what is predicted

To increase velocity the team simply needs to over-estimate stories to give the illusion of delivering more. They may not consciously do this but it may happen sub-consciously. The project manager pats them on the back, but all that has happened is the same amount of “done” working software has been delivered.

It’s time to get real and use real data to reflect real progress, whether it’s good news or bad.

  • We shouldn’t be defining all our scope up front, meaning we shouldn’t estimate all our scope up front, meaning we shouldn’t be defining our delivery date based on our scope

We should be fixing our Release 1 delivery date and aiming to build the best possible product by that date (variable scope).

As soon as we introduce the word “estimation”, the default mindset is to consider “how long will this project take?” (if this isn’t asked explicitly). This causes us to consider the complete scope and duration of the project (this is anti-Agile and I won’t go into why it’s a bad idea because enough has been written about that already elsewhere)

How do we size a project?

Short answer – you shouldn’t. If you don’t have a firm deadline for your project (e.g. day 1 of the Grand Prix for a Grand Prix app), you will have a budget for your project (set by the PMO or the external customer), from which you can derive a deadline. The smart thing to do is to then plan an interim release (say at the halfway point) where you can gauge how the project is going based on the working software measure.

For example, if your budget gives you enough cash for ten 2-week Sprints (given a fixed, 100% allocated team), clearly you need to assume that your go-live date is in 20 weeks time. But the aim should be to get working software in a production environment in 2 weeks time (after Sprint 1). You should then iterate over the product, allowing requirements (scope) to emerge and shape the direction the product takes, and take time to reassess after Sprint 5.

These things are not predictable up front – estimation will set you up with a load of scope (expectations) that will not get delivered and will only create unnecessary analysis time (money) and pressure.

How does the team get shared understanding of a story?

Simple. When a new item is added to the top of the product backlog, the team will discuss it in Sprint Planning and break it down if necessary. If it doesn’t need breaking down then it is likely already well understood. If it does then the act of breaking it down will necessitate conversations around the implementation detail that will facilitate shared understanding.

In short, the team does not need to be in an estimation session to discuss and break down a story.

How can the PO make trade-off decisions?

The PO probably needs to know the ROI of a story when introducing it to the team to be delivered. In order to calculate the ROI she needs to know how much it will cost to be delivered (how long).

Here a team would estimate the item using story points and then the PO, armed with the team’s velocity, can estimate the item’s ROI. But without story points how can this be done?

This is where the concept of “implicit estimation” comes into play. In order to create predictability in the flow of work, the team will break down stories just-in-time (in Sprint Planning) so that they are all roughly the same size. This is something that happens naturally throughout the course of the project. Over time the size of stories normalises because the team naturally wants bite-size chunks to work on in the short time period of the Sprint. They get used to delivering a certain number of stories, give or take, in a Sprint.

So for the PO to cost the item, she just needs to ask the team if it is understood or needs breaking down. If the PO considers it high enough priority she will want to introduce it in Sprint Planning so that it gets built right away, if it makes sense to do so. Sprint Planning is the place for the team to break down the story if required and decide if it can be delivered in the Sprint. If it can, the cost of the item is essentially 2 weeks of team wages (assuming production deployment is done at the end of the Sprint – a continuous delivery model can improve speed to market and ROI, but that’s a discussion for another day).

If the item can’t be delivered in the Sprint, the PO can simply look at how many stories have been spawned from the epic item and determine the likelihood of it being delivered in the next Sprint or the Sprint after, based on how many stories the team usually gets through. This leads me nicely on to the topic of how we measure velocity in the absence of story points.

How do we measure velocity?

Now I’m moving firmly into Duarte territory. The answer is we count stories rather than accumulate story points, hence negating the need to estimate. As I mentioned before, teams break stories down into roughly the same size, so counting how many stories are delivered in each Sprint makes for a satisfactory measure of velocity. If the team usually delivers 5 stories with zero defects and then one Sprint delivers 6 or 7 stories with zero defects, an improvement has been made (disregarding variance, which exists whatever unit you use to measure velocity).

Due to the hunch I mentioned earlier, I have been tracking velocity as both story count and points for my current team and making projections using both methods. As I suspected (and as Duarte points out with much supporting data), story count provides just as good, if not better a measure of progress and predictability as story points do. Therefore why spend all the time, cost and effort on estimation sessions and velocity calculations?

While story count works great for velocity, I would still warn against using this or any other velocity measure as a way of predicting when you can deliver. You should know when you are delivering and only be predicting what you can deliver at that date. Don’t leave your delivery date to chance, even if you are using historical data rather than guesswork to predict how many stories can be done.

What you can do, however, is use velocity to help the PO understand scoping trade-offs in the backlog (“the data tells me the team can deliver 20 more stories before the release date, so I’ll make sure the most important 20 are at the top of the backlog“).

Conclusion

It’s taken me several years to come to this conclusion. But, if you think about it, people laugh and joke about estimates all the time. Everyone knows they’re a guess. Everyone knows they’re wrong. Yet we continue to do them. I believe it is time for us to acknowledge that it makes far more sense to eliminate the risk and cost of estimation completely and use only empirical data (as Agile and Scrum promotes) to make predicitions.

In a world without estimation overhead the team is likely to be more happy and productive, the inefficiency of spending time on estimating rather than delivering working software is eliminated and the PO will have real data with which to make decisions rather than guesses made under pressure.

To summarise:

  • Don’t estimate your delivery date – base it on your budget or a firm deadline
  • Don’t estimate your scope – allow it to emerge in order to reap the benefits of building products with agility
  • Don’t explicitly estimate product backlog items (stories)
  • Use historical data (story count) to predict scope delivery on a given date
  • Use just-in-time implicit estimation (story breakdown in Sprint Planning) and past data to estimate cost (ROI) of story delivery

I don’t like to guess, but I predict that not estimating your projects will make success far more probable :)

Tweet about this on TwitterShare on FacebookShare on LinkedInShare on Google+Email this to someone

9 thoughts on “Should We Estimate Software Projects… At All?

  1. Hi,

    Good idea, I don’t know how much are you familiar with the Kanban method, but this is pretty much simpler version what is described there :).

    Cheers
    Maciek

  2. Thanks for the post. That is an interesting proposition.

    Measuring the velocity in similar sized stories is very attractive and naturally inclines teams towards story decomposition to the smallest viable units which can help a great deal with story clarity, rapid feedback and predictability of flow. There are also savings in estimation effort as you point out.

    Where I think there may be some issues with some aspects of your suggestions is in assisting Product Owners with prioritisation and release preparation.

    A PO role, depending on the commercial context, may involve scenarios that benefit from sizing more in the backlog than just what is coming up in the next few sprints and benefit from sizing larger grained items in the backlog relative to sprint-ready sizes.

    Imagine a PO is preparing for a trade show to announce a new product/release or preparing for the actual market release of a new product. Such things require some preparation of marketing materials, an advertising campaign and potential hardware with longer lead times than software (for example custom hardware for an embedded system). In each of these cases, some idea of what is in the product is needed before the announcements, ad campaigns or commissioning custom hardware that is feature dependent can be executed.

    In an ideal world, for full flexibility, since a PO will know that feature delivery can be unpredictable particularly over a longer time-frame, the PO could attempt to keep prior announcements as vague as possible and, if hardware is involved, make the hardware as general as possible. However, how excited would you be by an ad campaign or trade show stand that proclaimed “Coming next June, version 4 of super product x: there’ll be new stuff in it!” or a new games console that uses lowest common denominator generic HW to support unpredictable product feature sets?

    A PO could elect only to announce a release when it is ready but what if they are competing with other companies that are pre-anouncing latest forthcoming features and creating a market buzz around their products? This can be a difficult trade-off.

    For another interesting scenario, consider your example about the release date being known and the product owner knowing that 10 small sprint-ready stories can be developed in that time. In order to determine the highest value stories to place in that window, the PO would need either to have everything in the backlog broken down into small sprint-ready stories in order to compare ROIs or to have larger grained items in the backlog sized by some means in order to determine if they are worth further decomposition.

    In other words, the most valuable small items may be “hiding” in larger items. Some sizing of the larger items can help focus a PO’s attention on those that are worth the effort of decomposing and sizing further.

    Do you think there are means other than relative sizing throughout the backlog that could help with some of the above scenarios or that the scenarios themselves need to be challenged?

    1. Hi Flex. Wow, there is so much to cover in your comments I don’t think I’m going to be able to do it right here. Some of it is covered in stuff I’ve written or spoken about elsewhere. I encourage you to read my other posts and Slideshare presentations, and watch my videos and talks.

      Short answer though: I think the idea of big marketing launches for products and their features is a very dated idea. I can’t think of too many of the great software suppliers of our time such as Facebook, Google, Yahoo or Microsoft doing this. Excitement is generated in clever ways that keep options open for features and release dates. Furthermore, companies know they can upgrade software, and even hardware (via firmware updates) after release.

      Nowadays it is completely the norm for customers to buy products with only the key features that serve their needs available, and then update to the latest version if they want the latest features. This is happening everywhere with software and tech gadgets.

      1. Thanks for taking the time to reply Neil. I will certainly take a look at the other posts and blogs as I am a relative newcomer to this site and I don’t want to unnecessarily repeat points that have been well covered elsewhere.

        I agree entirely about the companies that you mention and the age of detailed feature announcement prior to release being replaced, particularly in pure SW companies, with other techniques for generating and maintaining excitement in releases but that does not mean that, in the initial development of new product, choosing which key and subsequent features to offer and release in which order for best ROI does not still require an ability to size larger grained backlog items.

        For post development feature upgrades, in the type of environments that you mentioned, I think I can agree with you that the case for sizing is far less clear-cut.

        I also wonder whether marketeers are driving the best possible marketing approach or instead limiting themselves to only approaches that fit with software estimation uncertainty unless we assume that such uncertainty is an unsolvable problem (which you, along with many others, may well be right about).

        Taking aside the Google, Microsoft, Facebook and similar, the environments that I work with (mainly embedded systems with dedicated hardware) can be a common exception to this and are a sizable minority within the larger world of software development so I think still worthy of some consideration.

        To take one example, for medical embedded systems, prior announcement of features is not only wanted by the market (because clinical evidence is needed before adopting new features and there are considerable costs and planning in re-training clinicians and long lead times in purchasing in this area), it is also required for regulatory purposes. There are lead times involved in proposing a set of features and gaining regulatory approval for their release. Because of this, trade shows announcing forthcoming features are alive and well in this domain.

        1. Where there are “hard” deadlines, this should be called out so everyone is aware of it. What I would now do is iterate over the solution in short timeboxes. This helps us measure progress toward the desired outcomes and manage our risk. What I *wouldn’t* do is define all the features that we think we need and simply try and deliver them all.

          We need to have a clear idea of the problem to be solved, and then (properly) iterate our way to solving it. An iterative approach is well recognised as an effective way of managing risk and increasing the chances of success when we have constrained cost or timeframes. If you don’t believe me, just ask NASA :) – they take this approach with all their software-intensive endeavours.

          1. I couldn’t agree with you more about iterating away uncertainty rather than attempting to plan it away and am familiar with the NASA approach.

            I just don’t quite see that iterating and estimating (or rather predicting continuously based on what is known as one iterates) are an either/or choice.

            There is still some value I think (at a cost) in estimation at some level in order to inform decisions about budget, dates, feature expectations (if important – especially minimal operational requirements for new products) and aspects of projects which are less able to react quickly to change (i.e., less able to efficiently adapt) such as investments in high cost long-lead-time premises, plant and equipment to test industrial control systems).

            Perhaps it is simply a case of doing a light-weight ROI analysis on the estimation activity itself in different environments to determine what level of estimation (if any) is valuable for particular purposes rather than assume it is or isn’t useful as a general rule?

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current ye@r *