Terraform
Terraform is a HashiCorp-owned Infrastructure as Code (IaC) technology that allows you to develop, deploy, alter, and manage infrastructure using code. It maintains your infrastructure’s lifespan, enables you to define resources and infrastructure in human-readable, declarative configuration files, and manages your infrastructure’s lifecycle.
Code is simply instructions written in the HCL (Hashi Corp Configuration Language) language in a human-readable format with the extension (.tf) or (.tf.json) which is written in HCL (Hashi Corp Configuration Language) Language.
What is IaC?
Infrastructure as code (IaC) refers to using configuration files to control your IT infrastructure.
What is the Purpose of IaC?
Managing IT infrastructure has traditionally been a laborious task. People would physically install and configure servers, which is time-consuming and costly.
Nowadays, businesses are growing rapidly, so manual-managed infrastructure can no longer meet the demands of today’s businesses.
To meet the customer’s demands and save costs, IT organizations quickly adopt the Public Cloud, which is mostly API-driven, and they architecting their application in such a way that to support a much higher level of elasticity and deploy their application on supporting technologies like Docker container and Public Cloud. To build, manage, and deploy the code on those technologies, a tool like Terraform is invaluable for delivering the product quickly.
Terraform Workflow
Terraform Init
- The Terraform Init command initializes a working directory containing Terraform configuration files.
Terraform Plan
- The Terraform Plan command is used to create an execution plan.
Terraform Apply
- The Terraform Apply command is used to apply the changes required to reach the desired state of the configuration.
Terraform Refresh
- The Terraform Refresh command reconciles the state Terraform knows about (via its state file) with the real-world infrastructure. This does not modify infrastructure but does modify the state file.
Terraform Destroy
- The Terraform Destroy command is used to destroy the Terraform-managed infrastructure.
Jenkins Pipeline
A Jenkins Pipeline is a suite of plugins that supports building, deploying, and automating continuous integration and delivery (CI/CD) workflows. It provides a way to define the entire build process in a scripted or declarative format called a Jenkinsfile. This allows developers to manage and version their CI/CD processes alongside their application code.
Why Jenkins Pipeline?
Infrastructure as Code (IaC)
- The build process is defined in a Jenkinsfile written in Groovy-based DSL (Domain-Specific Language).
- The Jenkinsfile can be stored and versioned in the same repository as the application source code, ensuring synchronization between code and build processes.
Reusability and Maintainability
- A single Jenkins pipeline can be reused across multiple environments (development, testing, production).
- Update the Jenkinsfile to change the build process, reducing the need to manually modify multiple jobs in Jenkins.
Improved Version Control
- Both the application code and build process are versioned together.
- Older releases can be built using the corresponding Jenkinsfile, ensuring compatibility.
Automation and Scalability
- The pipeline automates the entire CI/CD workflow, including code fetching, building, testing, and deployment.
- It supports parallel stages, enabling multiple tasks (e.g., unit and integration tests) to run concurrently.
Simplified Configuration Management
- Job configurations are no longer stored as XML files in Jenkins. Instead, they are defined as code in the Jenkinsfile, making backup and restoration easier.
Types of Jenkins Pipelines
Jenkins provides two types of pipelines:
Declarative Pipeline
- Easier to use, structured, and designed for most users.
- Uses a defined syntax and provides built-in error handling.
Scripted Pipeline
- More flexible but requires advanced Groovy scripting knowledge.
AWS ECS
AWS ECS (Elastic Container Service) is an AWS Container managed Service that allows you to run and manage Docker containers on a cluster of Virtual Servers.
Container Deployment Era
The container OS maintains application isolation. Container service is trending nowadays.
- Lightweight: Containers have less overhead than virtual machines. They can be used with the host OS without installing it; they contain only the libraries and modules required to run the application.
- Portable: Containers can be moved from one host to another and run across the OS distribution and clouds.
- Efficient: Better resource utilization than the virtual machines, which cannot occupy the entire hardware and gradually increase based on requirement.
- Fast Deployment: Containers can build quickly from container images, and it is easy to roll back.
- Microservices: It is based on loosely coupled architecture and supports distributed best for microservices.
Architecture
In this architecture diagram, we will launch an EC2 Instance using Terraform in AWS and the user data for Jenkins Server configuration. By Jenkins CICD Pipeline, we fetch the Source Code from GitHub, create a Docker Image, and upload it to the ECR Docker Registry. We will deploy the application on the ECS Cluster with this docker image.
Step 1: Create an IAM user and an Access Key/Secret Key for the IAM user, and provide the appropriate permissions, such as ECR and Docker Container Policy.
Step 2: Create an ECR Repository to store the Docker Images.
Step 3: Create an ECS Cluster
Step 3.1: Create a task Definition. The Task Definition contains all the information to run the Container, such as the container Image URL and Compute Power.
Step 3.2- Execution Role: It is attached to the Task Definition with permission from CloudWatch Logs to collect the real-time logs of the Container and ECS Task Execution Role Policy.
Step 3.3: Create a Service in Cluster: If Task Definition cannot handle the deployment, we have to create a service that is an intermediary between the application and Container Instances.
Step 4: Jenkins Server Configuration
- Repository to Launch EC2 Instance with user data to configure Jenkins Server by Terraform: https://github.com/prafulitankar/GitOps/tree/main/terraform
- Install Docker on the Jenkins Server to create and upload a Docker Image to the Docker Registry (ECR). Link to Install Docker: https://github.com/prafulitankar/GitOps/blob/docker/docker-setup.md
- Install AWS-CLI on Jenkins Server: https://github.com/prafulitankar/GitOps/blob/main/aws-cli
- Install Required Plugins. Follow the links for the list of Jenkins Plugins https://github.com/prafulitankar/GitOps/blob/main/Jenkins-Plugin
- Restart Jenkins Server.
- Setup IAM User Credentials in Jenkins Server, which will connect to ECR Repository and ECS Cluster.
Let’s deploy the code using Jenkins on ECS Cluster: Jenkinsfile for CICD Pipeline: https://github.com/prafulitankar/GitOps/blob/main/Jenkinsfile
Create a Jenkins Pipeline, which should be a Declarative Pipeline.
We have done with the Infra setup and Jenkins Pipeline. Let’s Run the Jenkins Pipeline:
Once Jenkins Pipeline is successfully executed, the ECS service will try to deploy a new revision of Docker Image.
Output: Once the Pipeline was executed successfully, our application deployed successfully on the ECS Cluster output of the application.
We launched the Jenkins Server on an EC2 Instance with Terraform. Then, we created an ECR repository to store the Docker image, ECS Cluster, task definition, and Service to deploy the application. Using the Jenkins pipeline, we pulled the source code from GitHub, built the code, created a Docker image, and uploaded it to the ECR repository. This is our CI part, and then we deployed our application on ECS, which is CD.
