How Story Points in Scrum can reveal more than hours tracking

My team recently received a couple of very interesting burndown charts from our previous sprint, and we’ve had a very good discussion on how this happened. We’re feeling this case could be very convincing evidence to support that using Story Points to estimate is better than using actual hours.

Before we look at the charts (real charts grabbed directly from our task tracking systems), let me introduce a little bit about our current estimation/tracking process.

• We estimate Story size using Planning Poker: we give each Story a number of Story Points by comparing the size with our base story.
• We break the User Stories down into detailed tasks and estimate how many hours we need to complete each task because our client requires us to estimate and track our velocity and capacity in hours.

We use the client Jira site (Atlassian Jira + GreenHopper) to track task completion using an hour based burndown chart. We have the policy that each time when the individual logs the real effort into Jira they need to re-estimate how many hours are remaining before the task can be completed.

At the same time we’re still using our own whiteboard-based tracking system which uses a Story Point burndown chart produced by hand to publish the day-to-day User Story completion status.

You may have already realized what is happening in our project – we use two different ways to estimate and track:

• Hours-based estimation for the detailed tasks, and an hours-based burn down chart tracking the task completion.
• Story Point based estimation for the User Stories, and a whiteboard Story Point burndown chart tracking the Story completion.

Choosing a Global Software Development Partner to Accelerate Your Digital Strategy

To be successful and outpace the competition, you need a software development partner that excels in exactly the type of digital projects you are now faced with accelerating, and in the most cost effective and optimized way possible.

Get the Guide

This provided a great opportunity to compare which works better for us using a single set of project data.

Now let’s take a look at the real interesting things, the two different burn down charts for our last Sprint.

Hour-based burndown chart:

Story Point-based burndown chart:

If we just look at the hours burndown, you’l see that everything was going perfectly. The burndown trend was as good as any Scrum examples: we only had 9 hours among the planned 240 left at the last day. But if you look at the story point burn down, things are totally different. By estimation, the planned Stories are 50 Story Points in total, but by the last day we just delivered 16 points – that definitely was a significant failure.

My team analyzed how this situation took place. When using the hours burndown chart generated by Jira, the system doesn’t care about whether or not the user stories are really completed, it just adds all the hours we logged to the tasks together and uses that aggregated number as the “completed” work. Jira is calculating using this formula:

Time spent = Work done

But that formula is NOT telling us the whole truth, the relationship between time and delivery are indirect:

• The tasks which one team/individual can finish in 10 hours may cost 100 hours for another team/individual.
• It’s possible that the team spends 100 hours while delivering nothing.
• As a User Story, it’s either “Done” or “Not Done”, we cannot use a ratio number (as we easily go with when estimating how much effort is remaining to complete) to say it’s  “Partially Done” – we cannot burn 33.3% story points down for one user story if it’s not done yet.

The conclusion my team had are:

• Hours estimation is never accurate, different people working on one task takes different hours.
• Hours spent doesn’t equal to task completion, “partially done” is a dangerous status which hides the problems.
• Story Point estimation is simpler and more Agile although it’s not easy.

When you perform a Google search, you will find a lot of discussion around how to do Story Point estimation. We were also questioning this and some of our team members were still hesitating to use Story Points because it’s not easy to understand and use; but after this Sprint we really realized that we should give up the traditional hours estimation because it’s giving us the wrong feeling and leading us in the wrong direction. In the retrospective meeting my team decided that we’ll do more practice in the future and incrementally make our Story Point estimation accurate. I’ll be glad to share our experience if in the future anything interesting happens during this process.