How often have you been part of a software development project where Quality Assurance efforts seemed to produce relatively meaningless figures and statistics? What does 69% execution rate, with a 53% pass rate really mean? How do you make sense of 400 defects, even when they are set to appropriate priority and criticality? Does this tell you anything about the health of a development initiative?
Let’s say, for example, that you are in the PMO group of an organization launching a brand new wireless service offering. This offering includes all facets of a customer facing wireless service solution (from the network, to billing system, to customer self care and CSR systems).
Each system should go through its own testing process, but at some point, there will be an integrated end to end testing effort. The results from this effort often come back in reports that talk about:
• Thousands of test cases executed
• Test pass rate
• Test coverage
• Defect counts
• Defect priority
• Etc.
Looking at stats like this can produce very little tangible information for non-technical key business stakeholders. It shows very little about the real problems that may hinder a project from going live on time, with acceptable performance, functionality, and customer satisfaction.
So, is this the best way to display results from testing efforts to key business stakeholders? Is there a different way to present this information, so that these stakeholders can make informed decisions about the direction of the program? Is there a way to do this better? Is there a way to present defect reports in a fashion that better represents the launch readiness of a solution?
In the next part of this blog, we will explore ideas on how to present test results in a more meaningful fashion so that business stakeholders get the information they need without diving deep into the weeds.
Making Sense of QA Test Results – Part 2
Making Sense of QA Test Results – Part 3
Tags: Defects, Quality Assurance, Testing