The Canary Deployment Pattern, or canary release, is a DevSecOps deployment strategy that minimizes risk by targeting a limited audience. As with all deployment patterns, the goal is to introduce the newly deployed system to the users with as least risk and in as secure a manner as possible. As noted below, the motivation of this particular approach is to identify a small segment of the user community that can act as an initial response group. Typically, this means that the selected user segment is “friendly” to the idea of trying out a new and possibly unfamiliar set of system features. By limiting the general release in the this manner, feed-back can be gathered on the impact/acceptance of the new features. Once the “canary” group has had sufficient time to validate the deployment, the full user-base can be updated.
Canary Deployment
-
Pattern Name: Canary Release
- Intent: A canary release is a way to identify potential system problems early without exposing all users. The intent is to deploy the application to a limited user audience and gain feed-back on any issues that may arise.
- Also Known As: Limited Rollout, Feature Trial, Beta Release, Soak Deployment
- Motivation (Forces): Reduce the impact of a new and potentially disruptive change to the user community. For example, a significant change to a user experience, such as release of a new user interface.
- Applicability: Any system where users can be impacted by a significant functional change; often applied to system releases that have readily identified user sub-populations (i.e. ‘friendly’ or ‘pre-release’)
- Structure:
Figure 1. Structure of rollout in the canary release pattern
- Participants:
| Participant | Role | Description |
|---|---|---|
| Production Release | Current production release | This is the current production release system. It will not be affected by the experimental release. |
| Primary User Group | Represents the core user group for the system | This group is comprised of the core system users who will remain on the current release as the “control” group. |
| Load Balancer | Channel traffic from specific user groups or regions to target server environment | Depending on how the user population is segregated (e.g. organization internal users vs. external users) the load balancer is used to direct user traffic to a well-defined release endpoint |
| ‘Canary’ User Group | Represents a ‘friendly’ user group for experimental potentially disruptive releases | This is a select group of system users who have agreed to try out the new functionality/capabilities and provide feedback as needed. |
| ‘Canary’ Release | Proposed experimental release | This is the experimental or potentially disruptive release. It will be separated from the current production release. |
- Collaboration: This pattern depends on the ability of the load balancer to properly shift user traffic so that the “canary” group is the only set of users who see the experimental release. Typically, this group will include a set of pre-selected ‘friendlies’ who understand that they will be using a new, possibly unstable system. It is expected that the ‘canary’ users will be contacted to review the results of the system use.
- Consequences: Given that the ‘canary’ release is potentially disruptive or unstable it should be closely monitored during the ‘canary’ testing period. Follow up with the targeted user group is also highly recommended to gain feedback on usability and stability. If necessary, the ‘canary’ release can be rolled-back to the previous release version while discovered issues are remediated.
- Implementation: This pattern requires that the production environment be capable of segregation into two groups. It is necessary that the production release servers be capable of handling the full user load in the case where the ‘canary’ release must be rolled back. Moreover, the user base must have some differentiating factor that can be used at the load balancer level to route traffic to the appropriate end-point (e.g. IP address range, internal/external user group, VPN/IPSEC tunnel, etc.).
Figure 2. Canary deployment traffic routing
- Trade Offs: This approach requires that a portion of the production environment be deployed with a different release version. There is additional monitoring overhead and potentially data migration that will be required once the ‘canary’ period expires. While the ‘canary’ version is in production there may also be difficulty with any additional release development and support as the development team may be required to review issues and observations.
- Known Uses: This pattern is often found during ‘beta-releases’ of new systems where stability and/or capabilities are rapidly changing. It is also found where development teams are rolling-out significant modifications to core system functionality and are concerned with user acceptance or system stability under production loads.
Conclusion
The canary release pattern offers product development teams an opportunity to validate system changes on a small sub-set of the user population, thereby avoiding wide-spread disruption if a set of new features fails to meet expectations. Moreover, for very large user groups (e.g. national or international in scope), this approach provides a mechanism to target specific user communities in isolation. The ‘canary’ group can therefore provide valuable feedback that can be incorporated into the overall system prior to a full user-base exposure.
