Agata Skorupka
7 March, 2023

Running a SaaS business means that your customers expect minimum or zero downtime. That has a scary implication: there is no room for any kind of mistake. There arises a question: how to ensure seamless deployment, given that “to err is human”? Thankfully, there are automations and processes to help us minimise human errors.

Seamless deployment with CodeDeploy

The first component which can assist you in that task is CI/CD pipeline (you can read more on them in my previous article). It will automate testing, build and deployment stages based on a single source of truth - your code. Thanks to CI/CD tools built in version control system hosting services, such as GitHub Actions, it can be conveniently automated further and trigger deployment on a commit or merge to a specific branch (for example, either release branch or a branch specific for testing or development purposes).

If it sounds like a solution to all problems, why do we need anything else on the top of it - namely, CodeDeploy? First of all, CI/CD tools themselves will not guarantee minimum downtime. That is something you need to manage on your side. Another reason is ease of rollback in case something goes not according to the plan, possibility of running customised tests before migrating the live traffic and last, but not least, customisation of the deployment process. With CodeDeploy you can steer how traffic will be shifted from the old version of the app to the new one (e.g. all at once or only a percentage at a time).

So to sum up, CodeDeploy is a scalable automation tool allowing you to customise and boost the deployment process. Of course, you could write a similar tool and manage it on your own, but CodeDeploy helps you reduce that overhead, while also being easy to use and platform-agnostic. With that tool you can deploy to EC2, Lambda functions, ECS, or even on-premises.

Blue/green deployments

CodeDeploy gives you a choice between two deployment options: in-place and blue/green deployment. The first one is a classic rolling update, where you can define the number of instances subject to update at a given time. On the other hand, the latter allows you to launch a set of new application version (blue) instances next to the old one (green) and control the switch of the live traffic in two dimensions: time and mode (in case of Lambda and ECS), therefore ensuring seamless and almost risk-free deployment. You can switch the traffic either after some period of time (e.g. when automatic tests will be completed - or even immediately, if you don’t mind) or when you will be done with manual testing. When deploying to Lambda or ECS, you can also specify how you want the live traffic to be migrated from an old application version: all-at-once, linear (given percentage of traffic in the given amount of time, e.g. 25% each 5 minutes) or canary (in two parts: a defined percentage of traffic in the first one, then after a defined amount of time, then all the rest).

Implementation

Let’s get hands on. In the following part of the article, I will show how to implement CodeDeploy with ECS using Terraform and integrate it with GitHub Actions (tip: you can also manage GitHub Actions with Terraform, although I won’t cover that in this article). To be clear, our goal is that pushing to the repository triggers a build of a new image and its deployment to ECS using CodeDeploy, having our infrastructure configured with IaC (Terraform).

GitHub Actions

First of all, let’s focus on the relevant GitHub Actions job steps. Please note that I am using hardcoded values for visibility purposes, but you may want to parametrize them with inputs or extract them as outputs from previous steps.

- name: Task definition download
  id: download-task-def
  env:
    ecs_task_def_name: example_codedeploy_ecs_task_def
  run: |
    aws ecs describe-task-definition --task-definition "$ecs_task_def_name" --query taskDefinition > task-definition.json    

- name: Fill in the new image ID in the Amazon ECS task definition
  id: update-task-def-image
  uses: aws-actions/amazon-ecs-render-task-definition@v1
  with:
    task-definition: task-definition.json
    container-name: container_name
    image: ${{ steps.build.outputs.image }}

- name: Set appspec.json values
  id: set-appspec-values
  env:
    container_port: 5000
    container_name: example_container
    cluster: example_codedeploy_ecs_cluster
    service: example_codedeploy_ecs_svc
    codedeploy-application: example_codedeploy_ecs_app
    codedeploy-deployment-group: example_codedeploy_ecs_dg
  run: |
    export TASK_DEF_ARN=$(aws ecs describe-task-definition --task-definition="$container_name" | jq '.taskDefinition.taskDefinitionArn')
      cat ./.github/workflows/appspec.json | jq --arg key "$TASK_DEF_ARN" '.Resources[].TargetService.Properties.TaskDefinition=$key' \
      | jq --arg key "$container_port" '.Resources[].TargetService.Properties.LoadBalancerInfo.ContainerPort=$key' \
      | jq --arg key "$container_name" '.Resources[].TargetService.Properties.LoadBalancerInfo.ContainerName=$key' > .aws/appspec.json
      sed -i 's#\\"##g' appspec.json    

  
- name: ECS task deployment using CodeDeploy
  uses: aws-actions/amazon-ecs-deploy-task-definition@v1
  env:
    ACTIONS_STEP_DEBUG: true #enable step debug logging
  with:
    task-definition: task-definition.json
    service: example_codedeploy_ecs_svc
    cluster: example_codedeploy_ecs_cluster
    wait-for-service-stability: true
    codedeploy-appspec: appspec.json
    codedeploy-application: example_codedeploy_ecs_app
    codedeploy-deployment-group: example_codedeploy_ecs_dg

As you can see, the workflow consists of four steps (most of them using predefined AWS actions). That is because the last step, i.e. ECS task deployment using CodeDeploy, requires the presence of two json files: task-definition.json and appspec.json to work properly, and we are populating them in advance with a new image version by downloading task definition via AWS CLI (download-task-def step) and updating it with a new image ID coming from the build step (update-task-def-image step). We also update appspec.json values accordingly (appspec.json step), having appspec.json template already defined in our directory:

{
 "version": 0.0,
 "Resources": [
   {
     "TargetService": {
       "Type": "AWS::ECS::Service",
       "Properties": {
         "TaskDefinition": "arn:aws:ecs:aws-region-id:aws-account-id:task-definition/ecs-demo-task-definition:revision-number",
         "LoadBalancerInfo": {
           "ContainerName": "your-container-name",
           "ContainerPort": "your-container-port"
         }
       }
     }
   }
 ]
}

Setting the environment variable ACTIONS_STEP_DEBUG to true allows us to debug logging in case of errors (especially if you are not passing parameters as hardcoded values). Please check the repository of specific AWS actions to make sure you set the correct permissions for CodeDeploy. In particular, make sure that you have passed task definition task and task execution roles to CodeDeploy.

Terraform

In the following part I will show how to configure CodeDeploy resources in Terraform. Please note that for brevity purposes I am assuming that you have your ECS and Load Balancer already configured - including two target groups (blue and green) and an additional test traffic listener (optional). Please remember to set the following parameters in aws_ecs_service resource:

 lifecycle {
   #ignore changes in image ID outside Terraform, i.e. in GitHub actions
   ignore_changes = [task_definition]
 }

 deployment_controller {
   type = "CODE_DEPLOY"
 }

First of all, we need to define aws_codedeploy_app resource, being a collection of CodeDeploy resources and binding it to the desired platform. In the configuration I am setting compute_platform to ECS. Please note that while I am hardcoding variables for the visibility purposes (e.g. how it corresponds to GitHub Actions workflow), it is advised to use variables in your code. I am assuming you know how to do that.

resource "aws_codedeploy_app" "example" {
  compute_platform = "ECS"
  name             = "example_codedeploy_ecs_app"
}

Now it is time to define the aws_codedeploy_deployment_group resource and configure deployment options we would like to implement. Regarding the deployment_config_name parameter, please see available options in the docs.

resource "aws_codedeploy_deployment_group" "example" {
  # basic configuration
  app_name               = "example_codedeploy_ecs_app"
  deployment_group_name  = "example_codedeploy_ecs_dg"
  service_role_arn       = aws_iam_role.code_deploy_role.arn
  deployment_config_name = "CodeDeployDefault.ECSAllAtOnce"

  # automatic rollback configuration
  auto_rollback_configuration {
    enabled = true
    # type of the event trigerring rollback (DEPLOYMENT_STOP_ON_ALARM - if CloudWatch alarm is associated or DEPLOYMENT_FAILURE)
    events = ["DEPLOYMENT_FAILURE"]
  }

  # configuration of blue/green deployment
  blue_green_deployment_config {

    # information about the action to take when new instances are ready
    deployment_ready_option {
      # CONTINUE_DEPLOYMENT: register new instances with load balancer immediatelly, STOP_DEPLOYMENT: wait for the traffic to be rerouted manually before the end of specifiec wait_time_in_minutes
      action_on_timeout = "CONTINUE_DEPLOYMENT"
    }

    terminate_blue_instances_on_deployment_success {
      action = "TERMINATE" # or KEEP_ALIVE
      # time to wait after a succesful deployment before terminating old instances - time for rollback
      termination_wait_time_in_minutes = 5
    }
  }

  # blue/green deployment configuration
  deployment_style {
    deployment_option = "WITH_TRAFFIC_CONTROL"
    deployment_type   = "BLUE_GREEN"
  }

  # ECS service configuration
  ecs_service {
    cluster_name = "example_codedeploy_ecs_cluster"
    service_name = "example_codedeploy_ecs_svc"
  }

  # load balancer information: target groups and listeners
  load_balancer_info {

    target_group_pair_info {

      prod_traffic_route {
        listener_arns = [aws_lb_listener.example.arn]
      }

      target_group {
        name = aws_lb_target_group.example_blue.name
      }

      target_group {
        name = aws_lb_target_group.example_green.name
      }

      # optional: path used by load balancer to serve test traffic
      test_traffic_route {
        listener_arns = [aws_lb_listener.example_test.arn]
      }
    }
  }
}

Summary

At this point I have successfully gone through all the components needed to implement CodeDeploy blue/green deployment with ECS using GitHub Actions (in a quite detailed way). Now every time you will push to your branch in a way defined to trigger workflow, a new image will be built, task definition and appspec files updated and deployed via CodeDeploy. You can observe the process in AWS Console or define a test traffic route (load balancer listener) to test it manually before a given period.

We offer hands-on AWS training as part of our SaaS Growth subscription, to help your team make the best use of the AWS cloud. Book a free chat to find out more.

For some topics, you can also get the same training just on the topic you need - see our Terraform training and Kubernetes training pages.

This blog is written exclusively by The Scale Factory team. We do not accept external contributions.