Creating a CI/CD Pipeline with GitHub Actions and Docker to Deploy on AWS Fargate

Introduction

In this tutorial, we’ll set up a continuous integration and continuous deployment (CI/CD) pipeline using GitHub Actions to automate the deployment of a Spring Boot application packaged in a Docker container to AWS Fargate. This process will streamline our deployment workflow and enable efficient updates to our application.

Prerequisites

Before diving into the setup, ensure you have the following:

1. Spring Boot Application: A functional Spring Boot application that we want to deploy.
2. AWS Account: An active AWS account to manage services like ECR and ECS.
3. Docker: Installed locally to build our Docker image.
4. AWS CLI: Installed and configured to interact with AWS services.
5. GitHub Account: To host source code and manage CI/CD workflows.
6. IAM User: An IAM user with permissions for Amazon ECS, ECR, and CloudWatch.

Setting Up AWS Environment

Create ECR Repository

Amazon Elastic Container Registry (ECR) is a managed container image registry that makes it easy for developers to store, manage, and deploy Docker container images. To start:

1. Navigate to the ECR Dashboard: Sign in to the AWS Management Console and open the ECR service.
2. Create a Repository: Click on Create repository.
3. Repository Name: Enter a unique name for your repository (e.g., my-spring-boot-app).
4. Settings: Leave the default settings and click Create repository.

This repository will host the Docker images of your Spring Boot application, allowing ECS to pull the images for deployment.

Create ECS Cluster

Amazon Elastic Container Service (ECS) allows you to run Docker containers at scale. We will use Fargate, a serverless compute engine for containers:

1. Go to ECS Dashboard: In the AWS Management Console, navigate to ECS.
2. Create Cluster: Click on Clusters and then Create Cluster.
3. Select Cluster Type: Choose Fargate.
4. Cluster Name: Provide a name for your cluster (e.g., production-cluster).
5. Create Cluster: Click Create

Our application will run in the ECS cluster. Using Fargate allows us to focus on building and running our application without managing the underlying server infrastructure.

Create ECS Task Definition

A task definition is a blueprint for your application that specifies what containers to run and how to run them.

1. Navigate to Task Definitions: In the ECS Dashboard, click on Task Definitions and then Create new Task Definition.
2. Select Launch Type: Choose Fargate as the launch type.
3. Task Definition Name: Fill in the task definition name and assign an execution role (you can create a new role with the necessary permissions).
4. Container Definitions: Under the Container definitions section, click Add container:
   • Container Name: Enter my-spring-boot-app.
   • Image: Leave it blank for now; we will update this once we have pushed the Docker image to ECR.
   • Memory Limits: Set a memory limit (e.g., 512 MiB).
   • Port Mappings: Enter the port your Spring Boot application uses (commonly 8080).
5. Create Task Definition: Click Create.

The task definition serves as the configuration for our service, detailing which Docker image to use, resource allocations, and port mappings.

IAM Permissions

Ensure that our GitHub Actions have the correct permissions. Attach the following IAM policy to the user, allowing access to ECS and ECR:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ecs:UpdateService",
        "ecs:DescribeServices",
        "ecs:DescribeTaskDefinition",
        "ecr:GetDownloadUrlForLayer",
        "ecr:BatchCheckLayerAvailability",
        "ecr:BatchGetImage",
        "ecr:GetAuthorizationToken",
        "ecr:PutImage",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    }
  ]
}

Please refer to the How to Deploy Spring Boot Microservices on AWS ECS with Fargate for detailed guidance.

Dockerize Spring Boot Application

Dockerizing your application allows you to package your application with all its dependencies into a single container image that can be deployed anywhere.

Create a Dockerfile: In the root of your Spring Boot project, create a file named Dockerfile:

# Use a base image with JDK
FROM openjdk:11-jre-slim

# Set the working directory
WORKDIR /app

# Copy the jar file from the target folder to the working directory
COPY target/my-spring-boot-app.jar app.jar

# Expose the port your app runs on
EXPOSE 8080

# Command to run the application
ENTRYPOINT ["java", "-jar", "app.jar"]

Build the Docker Image Locally:

Before pushing to ECR, let’s build and test the Docker image locally to ensure everything is working:

./mvnw clean package  # Package your Spring Boot application
docker build -t my-spring-boot-app .

Run the Docker Image Locally:

After building, run the image to verify it behaves as expected:

docker run -p 8080:8080 my-spring-boot-app

This Dockerfile uses the OpenJDK slim image, copies the JAR file into the container, exposes the application port, and specifies how to run the application. Testing locally ensures that your application is functional before deployment.

Push Docker Image to ECR

After confirming that the Docker image works, the next step is to push this image to the ECR repository:

Authenticate Docker to ECR:

Use the AWS CLI to authenticate the Docker client to our ECR repository:

aws ecr get-login-password --region your-region | docker login --username AWS --password-stdin your-account-id.dkr.ecr.your-region.amazonaws.com

Tag the Docker Image:

Tag the Docker image with the ECR repository URI:

docker tag my-spring-boot-app:latest your-account-id.dkr.ecr.your-region.amazonaws.com/my-spring-boot-app:latest

Push the Docker Image to ECR:

Finally, push the image to ECR:

docker push your-account-id.dkr.ecr.your-region.amazonaws.com/my-spring-boot-app:latest

Pushing the Docker image to ECR makes it accessible for ECS to deploy our application. The tagging process ensures the image is correctly identified in the repository.

Configure GitHub Repository

Now that our application is containerized and pushed to ECR, we need to set up GitHub Actions to automate the deployment process.

1. Create a GitHub Repository: If you haven’t already, create a new repository in GitHub and push your Spring Boot application code along with the Dockerfile.
2. Add GitHub Secrets: For the GitHub Actions workflow to authenticate with AWS, we need to store sensitive information in GitHub Secrets:
   • Go to your GitHub repository > Settings > Secrets and variables > Actions.
   • Click on New Repository Secret and add the following secrets:
     • AWS_ACCESS_KEY_ID: Your AWS IAM user’s access key.
     • AWS_SECRET_ACCESS_KEY: Your AWS IAM user’s secret key.
     • AWS_REGION: Your AWS region (e.g., ap-southeast-1).
     • AWS_ACCOUNT_ID: Your AWS account ID.
   • Using GitHub Secrets is crucial for maintaining the security of sensitive information. This prevents exposing credentials in your repository.

Create GitHub Actions Workflow

GitHub Actions enables us to automate workflows for our projects directly from the GitHub repository. In this step, we will create a workflow that builds the Docker image, pushes it to Amazon ECR, and updates the ECS service depending on the branch of the code being pushed.

Create Workflow File

1. Navigate to Your GitHub Repository: Open your GitHub repository where your Spring Boot project is hosted.
2. Create a New Directory for Workflows: In your repository, create a new directory named .github/workflows. This is where GitHub looks for workflow definitions.
3. Create a New YAML File: Inside the workflows directory, create a new file named ci-cd-pipeline.yml. This file will define the steps for your CI/CD process.

Define the Workflow

Now, let’s define the workflow in the ci-cd-pipeline.yml file. Below is an example of a workflow configuration that builds and deploys your Docker image based on the branch that is pushed:

name: Deploy to Amazon ECS

on:
  push:
    branches:
      - main    # Deploy to production when code is pushed to main
      - '*'     # Deploy to UAT for any other branch

env:
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
  AWS_REGION: ${{ secrets.AWS_REGION }}
  AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
  IMAGE_REPO_NAME_PROD: production-repo
  IMAGE_REPO_NAME_UAT: uat-repo
  ECS_CLUSTER_PROD: production-cluster
  ECS_CLUSTER_UAT: uat-cluster
  ECS_SERVICE_PROD: production-service
  ECS_SERVICE_UAT: uat-service

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Log in to Amazon ECR
        run: |
          aws ecr get-login-password --region ${{ env.AWS_REGION }} | docker login --username AWS --password-stdin ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com

      - name: Build the Docker image
        run: |
          docker build -t spring-boot-app:latest .

      - name: Tag the Docker image
        run: |
          if [ "${{ github.ref }}" == "refs/heads/main" ]; then
            docker tag spring-boot-app:latest ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com/${{ env.IMAGE_REPO_NAME_PROD }}:latest
          else
            docker tag spring-boot-app:latest ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com/${{ env.IMAGE_REPO_NAME_UAT }}:latest
          fi

      - name: Push Docker image to Amazon ECR
        run: |
          if [ "${{ github.ref }}" == "refs/heads/main" ]; then
            docker push ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com/${{ env.IMAGE_REPO_NAME_PROD }}:latest
          else
            docker push ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com/${{ env.IMAGE_REPO_NAME_UAT }}:latest
          fi

      - name: Update ECS Service in Amazon ECS
        run: |
          if [ "${{ github.ref }}" == "refs/heads/main" ]; then
            aws ecs update-service --cluster ${{ env.ECS_CLUSTER_PROD }} --service ${{ env.ECS_SERVICE_PROD }} --force-new-deployment
          else
            aws ecs update-service --cluster ${{ env.ECS_CLUSTER_UAT }} --service ${{ env.ECS_SERVICE_UAT }} --force-new-deployment
          fi

Workflow Explanation

Update ECS Service: The ECS service is updated with the new image using the aws ecs update-service command, triggering a new deployment.

Triggers:

• When code is pushed to the main branch, it deploys to production.
• For any other branch ('*' wildcard), it deploys to UAT.

Steps:

1. Checkout code: The actions/checkout action checks out the latest version of your code from GitHub.
2. Setup Docker Buildx: This action sets up Docker to build and push multi-platform images.
3. Log in to Amazon ECR: The pipeline authenticates Docker to the AWS ECR registry using your credentials stored in GitHub Secrets.
4. Build the Docker image: The Spring Boot application is built into a Docker image using the docker build command.
5. Tag the Docker image: Depending on the branch (either main for production or any other branch for UAT), the image is tagged with the appropriate ECR repository URL.
6. Push Docker image to Amazon ECR: The tagged Docker image is pushed to the corresponding ECR repository (either production or UAT).

Commit and Push Your Changes

Now that your workflow is set up, commit the changes to the .github/workflows/ci-cd-pipeline.yml file:

git add .github/workflows/ci-cd-pipeline.yml
git commit -m "Add CI/CD pipeline with GitHub Actions"
git push

When you push changes to the main or other branches, the workflow will automatically run. You can monitor the progress of the workflow by navigating to the Actions tab in your GitHub repository.

Conclusion

In this tutorial, we’ve walked through the process of creating a CI/CD pipeline using GitHub Actions to deploy a Spring Boot application on AWS Fargate. By following this guide, you’ve set up GitHub Actions to automate the deployment of your Spring Boot application to Amazon ECS.

This workflow ensures that whenever you push to the main branch, your application is automatically deployed to the production environment, while other branches trigger a deployment to UAT. This CI/CD approach helps streamline your deployments and improve the consistency of your release process.