A Burndown chart is the most important tool we use to represent work left over time in our Scrum toolbox. We use this diagram to measure the current progress and assess how healthy the project status is by looking at the trend. It also provides a good way for the team to know the deviation of the current team velocity vs. the historical velocity if we use the historical data to estimate.
Let’s take a look at some sample burndown charts I created using sample data so that we can easily see how the burndown charts illustrate the project status. Notice that all these projects are running in parallel and are having the same start date.
Project A, each Sprint lasts 3 weeks, the historical team velocity is around 165 Story Points per Sprint.
Project B, each Sprint lasts 2 weeks, the historical team velocity is around 30 Story Points per Sprint.
Project C, each Sprint lasts 1 week, the historical team velocity is around 50 Story Points per Sprint.
If you are the ScrumMaster taking care of one of these projects, these burndown charts are already good enough. But what if you are a PMO manager or a Project Portfolio Manager who is taking care of all these three projects at the same time? Are you comfortable that at one single moment you can identify which project has the least schedule deviation? You might realize the following problems if you have to put those diagrams together trying to come up with a quantitative comparison:
- The Sprint duration varies. If I want to compare Project C Sprint 1 (1 week) with Project A Sprint 1 (3 weeks) I have to wait until we complete Project A Sprint 1. But by that time Project A is in Sprint 3 already.
- The team velocity varies. Team B completes around 30 Story Points in 2 weeks while Team C completes around 50 in 1 week. Which team is delivering more?
- The comparison base standard varies. All the three projects are using different technologies to build different products, 1 Story Point in Project A definitely doesn’t equal to 1 Story Point in Project C: how can we say which project is having less schedule problem if both A and C has 20 points left unfinished?
In order to resolve these problems, very naturally , we would come to the solution of equalizing the both horizontal and vertical dimensions in the burndown charts using ratio numbers (percentage) to replace the real values (Story Points and days) so that all the projects with different characteristics will generate similar burndown charts and the value on these diagrams will become comparable – we can easily say a project having 5% schedule delay is performing better than a project having 20% schedule delay if we compare only from the schedule perspective.
When thinking about how to have a quantitative calculate method I recalled the Earned Value Analysis technology in PMI. It uses a very simple formula to calculate the schedule deviation:
SPI (Schedule Performance Indicator) = EV (Earned Value) /PV (Planned Value)
As the numerator, EV (earned value) stands for the total number of the original estimation (planned Value) for those as of completed stories.
Here actually the “PV (Completed)” could simply be translated to the “Original Estimation of those completed work”, and since when using Story Points to estimate we don’t usually do re-estimate so this value could be easily calculated by using a MS Excel formula – to add up the story points belong to those User Stories marked as “Done”.
But the denominator, PV is kind of tricky because in Scrum we don’t plan for when to complete how much work. However, we can have our own definition to get the rough “Planned Value”, I would like to just simply use the below formula since ideally every day the Scrum team should be delivering some pieces of working software.
According to the above definition, SPI is just a simply ratio number (percentage) representing as of the current moment the work that has been done over the work should be complete.
Let’s use Project A as the sample instance. Below are the metrics data we collected daily, and the SPI value calculated from the below formula.
Now we have a mathematics basis to calculate the SPI value for the different projects. If we put all 3 project data together and calculate the SPI on a weekly basis we get the table below:
If you’re a PMO manager or a Project Portfolio Manager I bet you’ll be very happy because based on these data you can very easily compare quantitatively the schedule deviation in different projects with different characteristics as follows:
When practicing this quantitative measurement technology, in order to calculate the SPI value we need to have our own definition for PV which best fits project reality. Moreover, we need to collect the metrics on a regular basis. Our best practice is to do it weekly to generate a quantitative report for the different projects. If you’re interested we invite you to apply this in your projects and contribute more great ideas. We look forward to hearing about other’s experience and best practices.
