This article first appered in Devops.com, written by Perficient Latin America Scrum Master, John A. Jaramillo.
Ah, metrics. A company’s best friend. Without them, a business can’t be certain it’s on track with deadlines. It doesn’t know if it’s operating efficiently, or even if the last organizational change had any impact at all. According to serial Entrepreneur David Skok, “Great companies are almost always run by great management teams. And great management teams know that the only way to improve a process is to start by measuring it.”
However, choosing great metrics isn’t always easy. When simultaneously managing multiple agile projects and incorporating new agile initiatives, it can become difficult to measure the impact your changes are making, or if your team is actually improving.
So what’s a company to do? To gain an accurate picture of what is actually happening, it’s imperative businesses use a combination of both business and agile metrics to keep track of projects. Business metrics should help measure the success of your agile initiatives, and agile metrics will allow you to continuously improve upon processes and deliver exceptional results to clients—which should be the entire mission of an agile approach.
But which metrics are most relevant for agile teams?
Choosing the Right Agile Metrics
Deciding which metrics to incorporate into your business takes plenty of thought. Most importantly, it’s key to select metrics you will actually use. If you don’t imagine you’ll have the resources to measure niche data sets, for example, it’s a waste of time to carve up ambitious metric tracking schemes.
Our experience has lead us to focus on three key areas: quality, performance and budget. We choose these categories because they offer a robust picture of the team, the company and the client side. Additionally, these will help you to determine whether projects are running on time and/or if you have enough resources to fulfill them.
It can be daunting to start a project with a whole list of in-depth metrics. Generally what we do is begin each project with a light set of metrics that allow us to monitor the project efficiently without taking away time or energy from the actual project itself. Then, we adjust or add metrics based on the particulars of that project and the needs of the client. Usually, this means defining the project road map with the number of sprints to be completed distributed in blocks or releases. This clearly outlines the time to market of the MVP and the testing needs of the client. From this point, we derive other necessary metrics such as initial budget definition based on high-level comparative estimates and the known speed of the teams working on the project.
After the first sprint has been completed, engineering metrics such as defect injection rate, testing automation coverage and technical debt analysis can be added based on needs. It goes without saying that all metrics are oriented to exceed client expectations and maintain product quality.
After establishing general categories, you can begin to use more specific metrics to measure and improve agile projects or products. It’s a great idea to turn to the 12 agile principles for inspiration—these speak to the importance of continuous delivery of impactful software, simplicity and changing behaviors to increase efficiency, among other values. Based on this list, companies may want to consider the core metrics of velocity, sprint burndown and release burndown chart. Obviously there are others that might work for your organization; we recommend selecting at least two metrics to accurately gauge agile projects.
Tracking velocity can help measure a particular team’s output throughout each sprint, and whether management decisions are stimulating or hindering this output. There are no baselines with velocity, because velocity is particular to each team. If you have a team of five people working on a Java project, the velocity for that team is completely different than three people working with .Net.
After the first sprint is completed by your team, you can understand the amount of features that can be completed by the team for that project. However, don’t get too caught up with only increasing your velocity. There are other important metrics that must also be used to measure the success and continuous improvement of agile initiatives.
A sprint burndown is another great way to measure output. With this metric, leaders can see the rate at which his/her team finishes the sprint and create a chart to clearly illustrate where they are and if they are on track.
A release burndown chart is a great measure of the overall project. This entails creating a chart to highlight which features have been completed and which are yet to be completed. The chart is updated at the end of each sprint by the Scrum Master.
To further breakdown these metrics—data permitting—productivity can be measured member by member. This entails measuring the velocity of each member to give a better picture of the overall working speed, and provide estimates for the project.
Choosing the Right Business Metrics
Whilst agile metrics help measure aspects of the development process and the objectives of your partnership with your client, they do not shed much immediate light on whether you are meeting your overall business goals and objectives. This is where business metrics come in. Keep in mind that correct business metrics should depend on company goals, objectives and the input of your team.
Software veteran Joel Trammell recommends that all metrics have five key characteristics: They are easily measurable, directly correlated to business performance, predictive of future business performance, isolated to factors controlled by the group it is measuring, and comparable to metrics of competitors.
Some businesses find it important to focus on performance and project management metrics with clients. These can help reveal efficiency and accuracy of planning, showing how many times a business hit the specified target or improved client relations. We have a number of key performance indicators (KPIs) that are maintained by the company overall.
When managing budgets, project management metrics are useful, too. To calculate the remaining amount of budget, look at the backlog and what you have already created. The first goal is to complete the initial components or objectives with the client. After you have completed your initial minimum viable product, you can start adding more features to the product based on what’s left in your budget and what you know about the efficiency of your team.
Other Success Indicators to Take into Account
Although metrics are important, they should not be the only indicators used to measure performance and success. Alongside quantifiable data are intangible elements such as employee satisfaction, company culture and team motivation. Really, these elements can make or break a project; if your staff is not being challenged or feels burnt out, the project is more likely to fail.
Measuring these factors is another process altogether; no set of numbers can tell you if your company has a healthy culture or if it is rotten to the core. This requires both instinct and communication.
Based on our own practices, we recommend conducting a retrospective assessment after every project, including the whole team if possible. This allows time to gather suggestions from every team member, as well as concerns, best practices and areas for improvement. Again, this information is not quantifiable, but is extremely valuable in improving and developing business processes, working relationships and company culture.
From the information obtained in the assessment, an action plan can be drafted up, potentially implementing a new helpful technology or working to solve an existing problem.
A great way to carry out this assessment is on a “360” basis, whereby all the team members evaluate a person trying to discover his or her improvement areas. As with any agile initiative, this should be a core component of your strategy.
Also, regular meetings—every two to three months—between the Scrum Master and each team member can facilitate this process. They can review a “Professional Development Plan”—a format particular to the company where the employee records his expectations and needs. Together, the team member and the Scrum Master can then come up with an action plan for growth.
Conclusion
Simply put, metrics are essential to keep track of businesses and teams. Make sure to only choose metrics that are highly applicable, and to resist the temptation of measuring too many—after all, biting off more metrics than you can chew will take away the focus from your agile strengths.
About John A. Jaramillo:
John A. Jaramillo is a Scrum Master, coach and teacher at Perficient Latin America, a software development company located in Medellín, Colombia. John leads coaching programs for Agile strategies and coordinates the corporate social responsibility program in which Perficient Latin America partners with local universities to develop project-based engineering classes for students. He’s an avid vocalist and rocks out with two punk rock bands.
