Imagine as a Quality Analyst, you begin your day of testing. You log into Sitecore Content Editor, create a page, name it, and open it in Experience Editor but receive a white screen “Our Services are unavailable.” You investigate and see a deployment happening to your environment, which is DEV. All this has taken 10 minutes but happens throughout the day every time a developer checks in code. A page that should have taken 15 minutes to create now takes 2 hours because of all the disruptions and recreation of content. Sounds awful; you might be thinking, why is there not a separate environment for testing? Of course, even today, there are cases where projects may not need a QA environment for many reasons, but I am here to try to persuade anyone who might be on the fence or think you might not need a QA environment why you should.
My Top 3 Reasons for a QA Environment
Happiness
Every team needs essential tools to complete their job; when they don’t, it can be frustrating and lead to unhappiness. An environment is a tool that is necessary for software development. Developers need to be able to check in and deploy code to DEV without worrying that it can delay the other team members. Integration of code is critical to keeping a healthy main branch, so it should be happening often. If a merge breaks the environment, no big deal; that is what this environment is for, right? Unless you have 2 QA folks waiting to use the environment for testing. The sharing of environments now puts a strain on both parties. Every day it can feel like moving roadblocks all day long to complete simple tasks, which can lead to unhappiness. Having a separate QA environment solves all the listed issues. Developers can freely check in and deploy code to DEV throughout the day. If DEV breaks, developers have time to investigate, and it doesn’t affect QA folks since not in a shared environment. QA folks can work and test in a stable gated QA environment, and simple tasks become simple again.
Time
The amount of time lost adds up. Most people think, who cares? It is only a few minutes when a build is deploying, and the environment only blips. All those blips add up. If code was being checked in every hour for 8 hours a day, on average, that is about 10 minutes of un-usability per deployment, which adds up to 80 minutes per day. One hour and 20 minutes daily is spent re-logging into Sitecore, refreshing screens, reauthoring lost content, and testing new changes. QA folks must now watch builds and deploys to see if they can create pages between deploys. The simplest tasks in Sitecore now take double the time. Developers must coordinate to see if their check-in will disrupt QA testing. Any delay in code reaching a testable environment puts sprints at risk and can cause slippage in timelines. Sprints become challenging to plan and finish when sharing environments could take up undetermined amounts of time. Teams can plan for these outages, but they will never be 100% accurate. Having a separate QA environment solves the lost time during code deploys.
Money
Cost Benefit- Let’s take a small project’s cost of lost time vs. the cost of the QA environment for ONE YEAR. In this scenario, a 7-person team loses 10 minutes an hour during deploys which adds up to 80 minutes per day, per person. In the example below, we will use easy math to say we only lose 60 minutes daily or 5 hours per business week. (This is a low estimate). Therefore, a team of 7 people costs the project $94,420 in lost time wages (Please see below calculations). Now let’s take the cost of a QA environment vs. the cost of the lost time labor. First, we need to determine the cost of the QA Environment. Sitecore environments have many variables that determine price, but we will use an average of $4000 a month for this example, bringing its yearly cost to $48000. Now that we have the cost of the QA Environment below, you can see the cost of the time lost, $94,420, is more than the cost of the QA environment of $48,000 for one year. Just by having a QA environment, this team would have saved $46,420 ($94,420-$48,000) in one year. As the team grows or if another group is added, the cost of Time Lost will continue to go up, while the cost of the environment will stay the same. Having a separate QA Environment prevents time loss, saving money for the project in employee costs.
| Role | Annual Salary | Hourly Rate | Lost Time | QA Environment |
|---|---|---|---|---|
| Scrum Master | $144,092 | $69.28 x (1 person x$69.28) | 260 hours lost x $69.28= $18,013 | |
| Business Analyst | $87,660 | $42.15 x (1 person x $42.15) | 260 hours lost x $42.15=$10,959 | |
| Developers | $124,978 | $180.27 x (3 people x $60.09) | 260 hours lost x $180.27= $46,870 | |
| Quality Analyst | $74,662 | $71.80 x (2 people x $35.90) | 260 hours lost x $71.80= $18,668 | |
| Calculations | Average USA salaries | Annual/2080 (40 hours x 52 weeks) | 52 weeks x 5 hours per week= 260 hours per Year | $4000 x 12 months |
| Totals | $431,392 | $363.50 (7 people total) | Cost of Lost Time $94,420 | Cost of QA Environment $48,000 |
Conclusion
As shown above, not having a QA environment thinking it could save money will likely do the opposite by costing the team more in happiness, time and money. Set your project up for success by having a QA Environment from the beginning.
