Blue/Green Deployment Pattern
The goal of any software development program is to release system changes into production. There are many ways to safely and securely deploy software into a production environment. In most cases these patterns follow a similar strategy of limiting exposure of the released software changes to the overall user audience. This is done to reduce the impact of a failed deployment, to offer an opportunity for feedback from a select user group, or to provide a “zero-outage” deployment with no downtime to the users. One of the most common is the ‘blue/green’ deployment pattern.
A blue/green deployment is where the production system is segmented into two environments – one (active) with the current production system deployed and the other (staging) for the new release. As shown in Figure 1 below these environments switch roles from one release to the next, acting in turn as the staging or active target. The remainder of this blog post will present the blue/green deployment strategy using the standard software pattern format.
- Pattern Name: Blue/Green Deployment
- Intent: Reduce new release risk by deploying to a parallel non-live environment
- Also Known As: Zero Downtime Deployment
- Motivation (Forces): Deployment to production always entails some risk of a failed deployment leading to system outage and user impact. By deploying to a separate non-live environment the release can be completed and verified prior to being made the production site. Additionally, the original production environment can be maintained as a fall-back during the “burn-in” period for the new release.
- Applicability: This pattern is well suited to rapid production release schedules based on Continuous Deployment (CD) via an automated DevOps pipeline. In particular, this pattern supports agile development practices with short release cycles (e.g. 2-3 weeks) by allowing controlled deployments to a ‘staging’ environment prior to full production release.
- Structure:
Figure 1. Blue-Green Deployment Pattern
- Participants:
- Collaboration: The ‘blue’ and ‘green’ environments rotate responsibility for hosting the production application/system. The Load Balancer is key to the pattern implementation to facilitate transition of user traffic from one environment to the other with minimal disruption. The addition of automated deployment via a DevOps pipeline ensures a secure and consistent mechanism of deployment configuration.
- Consequences: This pattern requires the ability to have all production traffic directed toward one environment (i.e. ‘blue’ environment) while a release deployment is in progress to the other target. Post-deployment the user traffic is re-directed to the newly deployed system (e.g. ‘green’) while the original production system is maintained as a fall back (e.g. ‘blue’). For subsequent deployments the pattern of environment use is alternated.
- Implementation:
On-Premise (Ground)
For a standard data-center hosted solution, the typical implementation is to have long-standing environments that are configured as a primary (“hot’) and secondary (“cold”) set of servers. These are fronted by the Load Balancer acting as a gate-keeper for traffic routing to the two environments. In this configuration, all of the network elements are pre-configured with the addresses for the target servers (virtual or non-virtual) which do not change during or after the release is complete. Both the ‘blue’ and ‘green’ environments are required to be identical, but may be hosted in different geographic locations.
Off-Premise (Cloud)
For a cloud-based solution to this pattern, the typical implementation will be via hosted virtual servers contained with in a virtual network. This option is more flexible in that resources can be created or destroyed as needed (e.g. see Variant – Infrastructure as Code below), and network addressing is automatically updated as necessary for the virtual routers. Moreover, servers can be automatically configured to expand capacity (‘auto-scale’) as required by user traffic. Similar to the ‘ground’ based pattern implementation, the resources used in the ‘blue’ and ‘green’ environments can be geographically distributed.
- Variant – Infrastructure as Code (IaC)
Figure 2. Blue-Green Deployment Variant – Infrastructure as Code (IaC)
in this variant of the pattern the release deployment target environment does not exist until it is created by the DevOps pipeline. This requires that all of the necessary environment configuration (including security hardening) is coded and tested prior to the release deployment. Using Infrastructure as Code (IaC) the target servers and other network components can be pre-defined and stored with the application code-base. In this manner the application and the target environment are always evolving together. In the blue-green deployment the ‘green’ environment is first created and then the deployable unit is deployed to the new environment. Post deployment the original ‘blue’ environment is scheduled for destruction once the ‘green’ environment is considered stable in production.
Note that this variant may be used in either ‘ground’ or ‘cloud’ implementations, but it is more common to find IaC approaches used in cloud-based agile development.
- Variant – Container-based Deployment
Figure 3. Blue-Green Deployment Variant – Containers
In this variant of the pattern the release deployment target is represented as a collection of one or more containers (e.g. Docker). The target environment is managed by the ‘container manager’ that is responsible for running and managing all container hosts (e.g. Kubernetes). The deployment is managed by the DevOps pipeline automation, but in this instance the deployment is of a container rather than a ‘deployable unit’ maintained in a container registry. Post release, once the ‘green’ environment is considered stable in production, the containers represented by the ‘blue’ container group are scheduled for destruction.
- Trade Offs: Depending on the variant of the pattern used (ground-based, cloud-based, IaC, or container) there are benefits and drawbacks. In the case of ‘ground’-based implementations there is a requirement to maintain a separate “staging” environment at all times. This represents a significant expense, but is off-set to some degree by the support for fail-over business continuity. In the IaC-based approach it is necessary to fully-define all of the target environment as declaration (Ansible/Chef-Infra/Azure Resource Template/etc.) files. This includes all necessary secret management, network configuration, and operating system security hardening. For container based deployments, a registry and container manager must be added to the solution pattern, as well as proper creation of the initial container images used for deployment.
- Known Uses: This pattern is well-known and utilized by a variety of software development teams. It is most often found with agile development teams utilizing cloud-based resources, but is also frequently found implemented in on-premise data centers using non-virtualized target environments.
Conclusion
A consistent pattern for automated deployment, such as presented here, provides your teams with a predictable mechanism for production releases. There are many related patterns to a blue/green, such as progressive disclosure, canary deployment, feature toggle, and A/B testing. Each of these patterns offers a different focus for an automated production deployment, and will be explored in more detail in future blog posts.
