In Devops, change is imminent but when poorly managed it can lead to these five common mistakes and further performance issues.
Because DevOps centers around change, and consistent change at that, it’s easy to encounter instability during a project. No one wants that, but avoiding it entirely is not possible.
You see, in Ops we are constantly evolving, changing, and adapting to meet not just the market trends and client expectations but internal requirements as well. For the most part this can be beneficial. However, there are two sides to “change” or flexibility. The good or positive side leads to growth, innovation, and ultimately success. The opposite leads to downtime, performance hiccups, and poor results at the worst.
So, even though change is both good and necessary, it can be a hindrance when not properly managed. Ask any software engineer what they think is the most common reason for system downtime, most will agree it’s due to software, network, or configuration changes.
The best — and only way really — to deal with growing instability is to catch and solve mistakes as early as possible. It’s all about preparation and preventative maintenance.
In light of that, we’re going to explore some of the most common ops mistakes, and how you can correctly deal with them. If you learn to identify the issues now, you’ll be better off when you encounter them later.
1. Ineffective test environments
Want to experience some real setbacks? Mix up your test and production environments. Or, you can make the poor decision of running all your tests on a local machine. The latter will cause some serious issues when you realize that applications run differently on different machines.
You’re not the only one in the field to experience difficulties choosing the appropriate test environments. According to the World Quality Report 2016-17 from Capgemini, here’s the average breakdown of the most common environments used.
What makes an environment is not the application or database, it’s the configuration. It’s the use of a controlled setting to conduct activities and monitor accuracy. So choosing the appropriate configuration should always be a priority, be it cloud based, virtualized or something else entirely.
Right from the start, keep your test environments separate. Furthermore, establish a proper testing protocol by using virtual machines. You’ll find that not only is it easier, but also it will save you lots of time. You can also better simulate platforms that your clients might have access to but you don’t.
Notice in the figure above, temporary and virtual testing collectively makes up the most usage? That’s because it’s so effective and much safer than deploying via live platforms.
2. Poor deployments
Each piece of code — during its entire lifecycle of development — must be deployed consistently. Otherwise, you risk experiencing configuration drift in which changes are made ad hoc or not recorded and the infrastructure becomes more and more different, or drifts. This is often exacerbated by rapid release schedules. This also means that time and resources are wasted when moving environments because you’ll likely be trying to identify why things aren’t working the way they should.
To ensure a more reliable process, stick with the same deployment steps from the beginning of the project to the end of it. This especially helps when you are moving from lower environments with more frequent deployments to those with fewer deployments.
3. Risk or incident management faults
You must develop and comprehensively document your incident management process. Failure to do so will result in severe inefficiencies.
This means building an incident response plan, defining roles and responsibilities within your team, and keeping your clients in the loop. The latter is only possible with proper documentation, which further highlights the need to have a good system in place.
Don’t neglect the generated incident reports either. Review them regularly to ensure that the operation is running smoothly and that issues are being handled in a timely manner.
4. No real-time monitoring or alerts
The tool itself, of which there are many, doesn’t matter. But monitoring in real-time is absolutely vital to a successful DevOps strategy.
You can select from open-source and premium tools, the choice is up to you. Just make sure you have something prepped and ready to go, and that it’s accurately sending the alerts and information you need.
5. Not maintaining backups
The question of whether or not you should make regular data backups is non-negotiable.
In fact, if you use S3 or rely on similar platforms, conducting regular backups should be familiar to you. It’s an industry practice that’s really become something of a standard, and for good reason.
Pro Tip: If you really want to be safe, you can even open your production datasets and backups in a virtual test environment to make sure everything is working correctly. That may save you some time later, especially if something fishy is up with your backup process or tools.
Bonus: Common security traps
Just to touch on a few more common mistakes, you may also want to avoid doing the following:
- Not using or assigning individual user accounts
- Failing to select or enable encryption as part of the development cycle
- Relying solely on SSH instead of gateway boxes for your database servers
- Ignoring internal IT requests and demands
- Deploying tools without performing extensive research
- Neglecting physical and local security within your office
?
Provided you avoid the basic mistakes here and continue to develop and manage your risk management strategy, you should be well-prepared for anything encountered during your next deployment. Catch those bottlenecks and failures early, and you can curb growing instability before it gets out of hand.
Sources:
https://www.sogeti.com/globalassets/global/downloads/testing/wqr-2017-2018/wqr_2017_v9_secure.pdf
https://www.guru99.com/devops-tools.html
https://devops.com/modern-database-backup-recovery-checklist/
