#NoEstimates puzzle experiment in Melbourne

Tweet about this on TwitterShare on FacebookShare on LinkedInShare on Google+Email this to someone

Craig Brown and I ran a variant of Chris Chapman’s now famous jigsaw puzzle experiment at a meetup of the Melbourne Agile and Scrum User Group on Wednesday evening.

Everyone had fun and was intensely engaged throughout. There were loads of interesting dynamics emerging from the teams, perhaps surprising given the contrived nature of the experiment.

600_301226032Set up

  • We set up three same-sized (10-12 people) teams, each with:
    • an identical jigsaw puzzle (way too big to be completed)
    • a Product Owner (to provide the vision and direction) and
    • a Scrum Master (to help the team achieve the PO’s vision)
  • We opted for 3 * 15-minute iterations, with 3 minutes for a Retro in between
  • Each team was told to use a different method –¬†one was a Scrum team, one was a “mob team” and one was a “no rules” team. Here’s what that meant:
Scrum team
  • Must have Planning (including estimation), Review and Retro in each iteration
    • We provided Planning Poker cards for the estimation but the team was free to choose whatever estimation method they liked
  • Must only work on “stories” agreed in Planning – new stories can’t be introduced mid-iteration
  • Stories are only “done” when PO accepts them (in Review or before)
“Mob” team
  • No formal ceremonies required
  • Team all works on one story at a time until “done” (single-piece flow approach)
  • No estimation
  • Retro encouraged but not “enforced”
“No Rules” team
  • Can work like the Scrum team, the Mob team, any combination of the two, or any other way they like

Outcome

  • 600_301537052600_301537062Scrum team delivered most stories (3; the other teams delivered 2 each)
  • Whole group was asked to vote on which they thought was the best outcome
    • “No rules” team won (emphatically)
    • Scrum team lost

Interesting Observations

Here are some empirical observations of the evening’s events and outcomes, along with my interpretation of what they indicate in an Agile/#NoEstimates context (==> in bold italics underneath the observation).

Scrum team
  • Delivered most in terms of stories but least in terms of value, both for their Product Owner and as voted for by the wider group
    ==> Output <> Value
    ==> Comparing teams in a useful way would require consistent measures of both effort and value velocity across teams
     
  • Spent far too large a proportion of time (particularly the first iteration) in planning, and needed to be alerted to this fact
    ==> Consistent timeboxing is important to ensure there is time to do all that is required, and for less variability of outcomes 
  • A member of the team openly admitted that he inflated an estimate because he did not agree with the value of the story that the PO wanted to do next
    ==> Estimates are often gamed, and for various reasons
“No rules” team
  • Implicitly chose not to estimate, but instead to maximise the time they had for building
  • Eventually delighted their Product Owner (and wider group), but during the game the PO felt like:
    • The approach to delivery was too ad-hoc, even chaotic, especially at the beginning
      ==> Teams must collaborate in order to be co-ordinated, improve and deliver the right outcomes 
    • Stories were too large (epic) so delivery all happened near the end rather than incrementally
      ==> Smaller stories have lower variability and can help with early and frequent delivery, creating better predictability for PO/customer and lessening the need for estimates
      ==> Larger, higher variability stories rely on estimates of time, or at least relative size, to provide the illusion of predictability
  • Started with no process at all but this was deemed unproductive (with such a big team), so they split into smaller teams with focused goals
    ==> Smaller teams are more effective because it is easier to collaborate, change direction, gain consensus, etc.
General
  • Scrum and Mob team both delivered purely incrementally (concentrating on edges) rather than iteratively (identifying a recognisable area of interest and building upon it), although stories were clearly too big
    ==> An iterative approach is crucial for risk management, predictability and delivering the right thing (value), i.e. without such an approach you have no choice but to estimate
  • Product Owners all felt like they weren’t being listened to – this had particularly bad consequences for the Scrum and Mob teams, perhaps due to their purely incremental approach
    ==> Important for all team voices to be heard, especially given the PO is driving what should be built in order to deliver on the vision
Tweet about this on TwitterShare on FacebookShare on LinkedInShare on Google+Email this to someone

2 thoughts on “#NoEstimates puzzle experiment in Melbourne

  1. An interesting approach to discussing estimates.

    From my understanding of #NoEstimates, we should be focusing on what has most value to the business, and starting from there.

    Have you tried telling all teams that a certain area of the jigsaw has the highest value? It would be interesting to see how the various team types worked/fared in that context (e.g. “completing the central 2 houses in jigsaw delivers the most value to the business”).

    1. Hi David. Each team has a Product Owner, who is responsible for conveying to the team what is most “valuable” and the best sequence of development from their own point of view. Of course each PO has a different idea of what this means.

      This is inevitable given that the POs in this experiment are representing different “companies”. However, your point does highlight the importance of Product Owners within the same business having a shared understanding of what constitutes the most valuable outcomes for the business as a whole, not just their own product. Local optima can destroy businesses.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current ye@r *