AWS Best Practice of DevOps Agile Delivery for the Financial Services Industry

The previous chapter AWS DevOps+Q Agile Delivery of 16 Leadership Principles for the Financial Services Industry shared how AWS DevOps pipelines can solve pain points in the financial services industry, and utilize the Amazon 16 Leadership Principles.

In this chapter, you will learn how to build an AWS DevOps pipeline:

AWS Services	Description
IAM	Identity and Access Management
EC2	Cloud-computing platform
Elastic IP address	Static IPv4 address designed for dynamic cloud computing
Route53	Cloud domain name system (DNS) service
CodeDeploy	Automate application deployments to Amazon EC2 instances
GitHub Actions	Easy to automate all your software workflows
Pricing Calculator	Create an estimate for the cost of your use

2.0 AWS DevOps Pipeline

2.1 Pre-requisites

2.1.1 Knowledge Pre-requirements

• Create an EC2 server
• Have a GitHub account and know basic Github Actions.
• Know how to setup NGINX
• Know basic AWS services, including EC2, CodeDepoly, IAM.

2.1.2 Project Requirements

First upload a simple static web project codedeploy.nginx.001 on Github, which includes:

Object	Location
index.html	./	Static Web Page
ic_alana_002_20241022_a.jpg	./icons	images on a static web page
appspec.yml	./	CodeDeploy code
application-stop.sh before-install.sh after-install.sh application-start.sh validate-service.sh	./scripts	CodeDeploy code
appspec.yml	./github/workflows	CodeDeploy code

Also, GitHub access tokens are needed to configure codeDeploy permissions.

Github -> Setting -> Developer Setting -> Tokens. Add a GitHub access token.

2.2 Creating IAM Roles

A good naming style is important because as the number of IAM roles grows, it can be confusing for developers.

AmazonSageMaker-ExecutionRole-20240805T101031
AmazonSagemakerCanvasBedrockRole-20240801T140683

{service}-{role}-{datetime}-{version}. AWS Bedrock and SageMaker auto-generated IAM naming style.

AWSCodeDeployService-EC2AccessCodeDeployRole-20241024T000000
AWSCodeDeployService-DepolyEC2Role-20241024T000000
AWSCodeDeployService-GitAssumeRoleWithAction-20241024T000000

This is the clear IAM naming style, so we will create three IAM roles for EC2, CodeDeploy, and GitHub Actions, respectively, following this official IAM naming style.

2.2.1 AWSCodeDeployService-EC2AccessCodeDeployRole-20241024T000000

Select EC2 on Use Case Tab。

AmazonEC2FullAccess
AmazonEC2RoleforAWSCodeDeploy
AmazonS3FullAccess
AmazonSSMManagedInstanceCore
AWSCodeDeployFullAccess
AWSCodeDeployRole

Add AmazonEC2, AmazonS3, and AWSCodeDeploy permissions.

2.2.2 AWSCodeDeployService-DepolyEC2Role-20241024T00000

Select CodeDeploy on Use Case Tab.

AWSCodeDeployFullAccess
AWSCodeDeployRole

Add AWSCodeDeploy permissions.

2.2.3 AWSCodeDeployService-GitAssumeRoleWithAction-20241024T000000

Select Access management -> Identity providers -> Add provider.

Used to listen to GitHub Actions.
Provider URL: token.actions.githubusercontent.com
Audience: sts.amazonaws.com
The GitHub Identity Provider then adds the AWSCodeDeployService-GitAssumeRoleWithAction-20241024T000000 role.

Select Assign Role -> Web identity -> GitHub organization.

AmazonS3FullAccess
AWSCodeDeployFullAccess

Add S3, AWSCodeDeploy permissions.

2.3 Create Amazon EC2

1. Fill in the name ec2.cheaper.001
2. Click Amazon Linux 2023 AMI
3. Click t3a.nano

Finally, click Launch instance to create EC2.

2.3.1 Associate Elastic IP address

1. Click on Elastic IPs
2. Click the Allocate Elastic IP Address button

1. Select the name ec2.paper.001 where EC2 has just been created
2. Select the default Private IP address
3. Click the Associate button

2.3.2 Amazon Route 53

1. Fill in the sub-domain name
2. Fill in the EC2’s Private IP address
3. Click the save button

Successfully set up the static sub-domain name and IP address.

2.3.3 Add AWS IAM roles

1. Select Actions
2. Select Security
3. Select Modify IAM role

Add AWSCodeDeployService-EC2AccessCodeDeployRole-20241024T000000.

2.3.4 Install CodeDeploy Agent on Amazon EC2

Enter the Amazon EC2 terminal.

1. Select Connect button
2. Select EC2 Instance Connect tab
3. Select Connect button

Successfully log into the Amazon EC2 terminal.

sudo apt update
sudo yum install ruby
sudo apt install wget
cd /home/ec2-user
wget https://aws-codedeploy-us-east-2.s3.us-east-2.amazonaws.com/latest/instal
chmod +x ./install
sudo ./install auto

Install CodeDeploy Agent

Success, CodeDeploy Agent is running.

2.3.5 (Optional) Install Git on Amazon EC2

sudo yum install git-all
git clone https://{YOUR_GITHUB_SECRET_ID}@github.com/{YOUR_GITHUB_ORGANIZATION_NAME}/{YOUR_GITHUB_PROJECT_NAME}.git
git checkout .
git pull origin main
sudo chmod 777 -R PATH

Install git and pull the project to Amazon EC2.

2.3.6 (Optional) Install NGINX

sudo yum update
sudo yum install nginx -y
sudo service nginx start
sudo service nginx status

Install NGINX

sudo netstat -tunpl

Show Amazon EC2 listening ports. At this moment NGINX is on port :80.
The default home page of NGINX is in /var/www/html/index.html.

Ensure that Source and Destination are publicly accessible, set to 0.0.0.0/0.

2.3.7 Appspec.yml

Reference Articles:

• (AWS Dcos) CodeDeploy AppSpec file reference
• (AWS DevOps Blog) Build and Deploy Docker Images to AWS using EC2 Image Builder
• (AWS GitHub Example) Build and Deploy Docker Images to AWS using EC2 Image Builder

Appspec.yml is used to indicate the codeDeploy procedure code.
Deployment is divided into 5 steps: (1) BeforeInstall -> (2) BeforeInstall -> (3) AfterInstall -> (4) ApplicationStart -> (5) ValidateService.

In the root directory, add ./appspec.yml.

version: 0.0
os: linux
files:
    - source: /
      destination: /usr/share/nginx/html
hooks:
    ApplicationStop:
    - location: scripts/application-stop.sh
      timeout: 300
      runas: root
    BeforeInstall:
    - location: scripts/before-install.sh
      timeout: 300
      runas: root
    AfterInstall:
    - location: scripts/after-install.sh
      timeout: 300
      runas: root
    ApplicationStart:
    - location: scripts/application-start.sh
      timeout: 300
      runas: root
    ValidateService:
    - location: scripts/validate-service.sh
      timeout: 300
      runas: root

• Source is the root directory of the GitHub project.
• Destination is the project pulled into Amazon EC2.

In addition, a new ./scripts folder, in which there are 5 xxxxxxxx.sh respectively.

application-stop.sh
before-install.sh
after-install.sh
application-start.sh
validate-service.sh

There are 5 xxxxxxxx.sh in there, which are the 5 steps of codeDeploy.

(1) application-stop.sh

#!/bin/bash

Empty. There is no need to stop the application in this tutorial.

(2) before-install.sh

#!/bin/bash

Empty. There is no need to stop the application in this tutorial.

(3) after-install.sh

#!/bin/bash
 
sudo yum update
sudo yum install nginx -y

Install NGINX

(4) application-start.sh

#!/bin/bash
 
sudo service nginx start

restart NGINX

(5) validate-service.sh

#!/bin/bash

Empty. There is no need to stop the application in this tutorial.

2.3.8 Static Website Pages

Added ./icons folder, which shows the site image ic_alana_002_20241022_a.jpg.

Also, added index.html home page.

<html lang="en" data-bs-theme="dark">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no">
<script src="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.3.3/js/bootstrap.min.js" integrity="sha512-ykZ1QQr0Jy/4ZkvKuqWn4iF3lqPZyij9iRv6sGqLRdTPkY69YX6+7wvVGmsdBbiIfN/8OdsI7HABjvEok6ZopQ==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.3.3/css/bootstrap.min.css" integrity="sha512-jnSuA4Ss2PkkikSOLtYs8BlYIeeIK1h99ty4YfvRPAlzr377vr3CXDb7sb7eEEBYjDtcYj+AjBH3FLv5uSJuXg==" crossorigin="anonymous" referrerpolicy="no-referrer" />
<title>Alana Lam</title>
</head>
<body>
<div class="container">
  <div class="row">
  <div class="col-12 mt-4 text-center">
  <h1>CodeDeploy + Github Actions + EC2</h1>
  <img src="./icons/ic_alana_002_20241022_a.jpg" class="mt-4 rounded-circle" alt="Alana Lam" width="200" height="200">
  <h5 class="mt-4">Alana Lam (AWS Builder Community Manager, Hong Kong)</h5>
  </div>
  </div>
</div>
</body>
<html>

A simple static site with text and images.

If you have completed “2.3.5 Install GIT” and “2.3.6 Install NGINX”, you can type EC2 EIP or the domain name in your browser, to see the Static Website Pages.

2.4 Create AWS CodeDeploy

2.4.1 Create the AWS CodeDeploy application

1. Fill in the application name test.codeDeploy.001
2. Select EC2/On-premises
3. Select Create application button

2.4.2 Create AWS CodeDeploy Deployment Group

1. Select Create deployment group button

1. Fill in the Deployment group name test.deploymentGroup.001
2. Select the IAM role, AWSCodeDeployService-DepolyEC2Role-20241024T000000
3. Remove Enable load balancing, because this is the simplest DevOps pipeline case, so there is no need for additional AWS services

2.4.3 Create AWS CodeDeploy Deployment

Go to test.deploymentGroup.001
Select Create deployment button
First, Select My application is stored in GitHub

1. Fill GitHub token name
2. Fill in the Repository name, codedeploy.nginx.001
3. Fill in Commit ID
4. Select Create deployment button

2.4.4 Successful run of AWS CodeDeploy

Successfully run AWS codeDeploy

2.5 Create GitHub Actions

Reference Articles:

• Why I switched from AWS CodePipeline to GitHub Actions

2.5.1 Create GitHub Actions workflow

1. Click New workflow button
2. Select set up a workflow yourself link
3. After writing the GitHub Actions command, click the Commit changes button

2.5.2 Configurate GitHub Actions secrets and variables

1. Select Settings Tab
2. Select Secrets and variables -> Actions Tab
3. Select Secrets Tab

2.5.3 Add GitHub Actions secrets variables

1. Add a new secrets variable with name IAMROLE_GITHUB_ARN
2. The value is the ARN of the IAM role arn:aws:iam::{xxxxxxxxx}:role/AWSCodeDeployService-GitAssumeRoleWithAction-20241024T000000
3. Click the Add secret button

2.5.4 Add GitHub Actions variables

1. Select Variables Tab
2. Add four of Actions Variables
3. Select New repository variable button

Variables Name	Value	Description
AWS_REGION	us-east-1	The default region is US East (N. Virginia)
CODEDEPLOY_APPLICATION_NAME	test.codeDeploy.001	2.4.1 Create the AWS CodeDeploy application
CODEDEPLOY_DEPLOYMENT_GROUP_NAME	test.deploymentGroup.001	2.4.2 Create AWS CodeDeploy Deployment Group
IAMROLE_GITHUB_SESSION_NAME	AWSGitAssumeRoleWithAction	2.2.3 AWSCodeDeployService-GitAssumeRoleWithAction-20241024T000000

2.5.5 Write GitHub Actions Code

.github/workflows/main.yml

name: Deploy
 
on:
  workflow_dispatch: {}
 
jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: Prod
    permissions:
      id-token: write
      contents: read
    steps:
    - name: Git clone the repository
      uses: actions/checkout@v2
 
    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v4
      with:
        role-to-assume: ${{ secrets.IAMROLE_GITHUB_ARN }}
        role-session-name: ${{ vars.IAMROLE_GITHUB_SESSION_NAME }}
        aws-region: ${{ vars.AWS_REGION }}
    - run: |
        commit_hash=`git rev-parse HEAD`
        aws deploy create-deployment --application-name ${{ vars.CODEDEPLOY_APPLICATION_NAME }} --deployment-group-name ${{ vars.CODEDEPLOY_DEPLOYMENT_GROUP_NAME }} --github-location repository=${{ github.repository }},commitId=${{ github.sha }} --ignore-application-stop-failures

A basic version of the GitHub Actions Code.

2.5.6 Run GitHub Actions Code

1. Select Actions Tab
2. Select Deploy Tab
3. Select Run workflow button

2.5.7 Successfully running GitHub Actions

Successfully ran main.yml

4.0 Cost

Plan	USD
Monthly cost	$11.83
Total 12 months cost	$141.96

Overall, AWS’s prices are quite competitive. The most important thing is that CodeDeploy is cheap, and the cost of using Amazon EC2 t4g.nano is very low, so AWS is a low-cost + efficient cloud service provider.

4.1 Detailed Estimate

Service	Monthly	First 12 months total (USD)
AWS CodeDeploy	$8.8	$105.6
Amazon EC2	$1.533	$18.4
Amazon Route 53	$0.4	$4.8
VPN Connection	$1.1	$13.2

5.0 Summary

GitHub Actions + CodeDepoly are powerful DevOps tools that fulfill the principle of “think big, take small steps” in a business environment.

To conclude, let’s summarize the key points of this chapter:

5.1 Principles

• The new “Macro Portfolio” system is to comply with the “Least Effort Principle”, which includes (1) agile development, and (2) agile deployment
• The real issues were (1) the project took too long to deploy, and (2) automated deployment was not achieved
• Success is due to the following: (1) Other departments want small features in small increments. (2) More simplicity means more understanding of the problem’s root cause.

5.2 Action

• Give the “Updated API Manual” to other departments to try before every Thursday
• Simplicity is a good result of the Highest Standards because we performed (1) a “DIVE DEEP investigation” and (2) understanding the root cause of the problem

5.3 AWS DevOps

• The development engineer commits the code via GitHub Push
• GitHub Actions trigger workflows
• IAMROLE_GITHUB_ARN authorizes access to AWS resources
• GitHub Actions triggers AWS CodeDeploy
• AWS CodeDeploy triggers deployment to Amazon EC2 instances
• AWS CodeDeploy pulls Github resources and deploys to Amazon EC2 instances

5.4 AWS IAM （CodeDeploy, EC2, Github）

• AWSCodeDeployService-EC2AccessCodeDeployRole-20241024T000000
• AWSCodeDeployService-DepolyEC2Role-20241024T000000
• AWSCodeDeployService-GitAssumeRoleWithAction-20241024T000000

5.5 AWS CodeDeploy （Appspec.yml）

• BeforeInstall
• BeforeInstall
• AfterInstall
• ApplicationStart
• ValidateService

5.6 Cost

• Monthly cost: $11.83 (USD)
• Total 12 months cost: $141.96 (USD)

---

Postscript

On 14 December 2024, I attended the annual Amazon Greater China Community Gathering. I am very thankful to AWS for bringing me an unforgettable experience.

📷Shoot and 🎬Edit by Kenny Chan
Also, thanks to Smile (Lingxi) Lv - Developer Experience Advocacy Program Manager for supporting AWS Community Builder.