AWS Codepipeline Docker Vulnerability Scan

CloudFormation template for deploying CoreOS Clair and setting up AWS CodePipeline for automated vulnerability scanning of Docker Image pushed to Amazon Elastic Container Registry (ECR).

Reference Architecture

Prerequisites

• Docker
• Git
• AWS CLI installed.
• AWS CLI is configured with IAM Access Key, Secret Access Key and Default region as US-EAST-1.

Clone the GitHub repository

To help you quickly deploy Clair on AWS and set up CodePipeline with automatic vulnerability detection, we will use AWS CloudFormation. The repository also includes a simple, containerized NGINX website for testing your pipeline.

# Clone the GitHub repository
git clone https://github.com/aws-samples/aws-codepipeline-docker-vulnerability-scan.git

cd aws-codepipeline-docker-vulnerability-scan

VPC Requirement

A VPC in AWS-Region us-east-1 with:

• 2 Public Subnets
• 2 Private Subnets
• NAT Gateways to allow internet access for services in Private Subnets.

You can create such a VPC using the AWS CloudFormation template networking-template.yaml that is included in the sample code you cloned from GitHub.

# Create the VPC
aws cloudformation create-stack \
--stack-name coreos-clair-vpc-stack \
--template-body file://networking-template.yaml

# Verify that stack creation is complete
aws cloudformation wait stack-create-complete \
–stack-name coreos-clair-vpc-stack

# Get stack outputs
aws cloudformation describe-stacks \
--stack-name coreos-clair-vpc-stack \
--query 'Stacks[].Outputs[]'

Click here to open the Cloud9 console and open the Codepipeline-Docker-Vulnerability-Scan IDE created by the above CloudFormation stack.

We will continue to execute the remainder of the workshop from the Cloud9 IDE terminal.

Clair Deployment

First, create an Amazon Elastic Container Registry (Amazon ECR) repository to host your Clair Docker image. Then, build the Clair Docker image on your workstation and push it to the ECR repository that you created.

cd aws-codepipeline-docker-vulnerability-scan

# Create the ECR repository
# Note the URI and ARN of the ECR Repository
aws ecr create-repository --repository-name coreos-clair

# Build the Docker image
docker build -t <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/coreos-clair:latest ./coreos-clair

# Push the Docker image to ECR
aws ecr get-login --no-include-email | bash
docker push <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/coreos-clair:latest

Deploy Clair using AWS CloudFormation

Now that the Clair Docker image has been built and pushed to ECR, deploy Clair as an ECS service with the Fargate launch type. The following AWS CloudFormation stack creates an ECS cluster named clair-demo-cluster and deploys the Clair service.

# Create the AWS CloudFormation stack
# <ECRRepositoryUri> - CoreOS Clair ECR repository URI without an image tag
# Example - <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/coreos-clair

aws cloudformation create-stack \
--stack-name coreos-clair-stack \
--template-body file://coreos-clair/clair-template.yaml \
--capabilities CAPABILITY_IAM \
--parameters \
ParameterKey="VpcId",ParameterValue="<VpcId>" \
ParameterKey="PublicSubnets",ParameterValue=\"<PublicSubnet01-ID>,<PublicSubnet02-ID>\" \
ParameterKey="PrivateSubnets",ParameterValue=\"<PrivateSubnet01-ID>,<PrivateSubnet02-ID>\" \
ParameterKey="ECRRepositoryUri",ParameterValue="<ECRRepositoryUri>"

# Verify that stack creation is complete
aws cloudformation wait stack-create-complete \
–stack-name coreos-clair-stack

# Get stack outputs
# Note the ClairAlbDnsName
aws cloudformation describe-stacks \
--stack-name coreos-clair-stack \
--query 'Stacks[].Outputs[]'

Deploying a sample website container

Deploy a simple static website running on NGINX as a container. An AWS CloudFormation template is included in the sample code that you cloned from GitHub.

Create a CodeCommit Repository for Nginx Website

You create an AWS CodeCommit repository to host the sample NGINX website code. This repository is the source of the pipeline that you create later. For more information on accessing CodeCommit from Cloud9 refer to the documentation.

# Create the CodeCommit repository
# Note the cloneUrlHttp and Arn value
aws codecommit create-repository --repository-name my-nginx-website

# Configure CodeCommit access
git config --global credential.helper '!aws codecommit credential-helper $@'
git config --global credential.UseHttpPath true
git config --global user.email "<YOUR EMAIL>"
git config --global user.name "<YOUR NAME>"
 
# Clone the empty CodeCommit repository
cd ../
git clone <cloneUrlHttp>

# Copy the contents of nginx-website to my-nginx-website
cp -R aws-codepipeline-docker-vulnerability-scan/nginx-website/. my-nginx-website/

# Commit the changes
cd my-nginx-website/
git add *
git commit -m "Initial commit"
git push

Build Nginx Docker Image

Create an ECR repository to host your NGINX website Docker image. Build the image on your workstation using the file Dockerfile-amznlinux, where Amazon Linux is the parent image. After the image is built, push it to the ECR repository that you created.

# Create an ECR repository
# Note the URI and ARN of the ECR repository
aws ecr create-repository --repository-name nginx-website

# Build the Docker image
docker build -f Dockerfile-amznlinux -t <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/nginx-website:latest .

# Push the Docker image to ECR
docker push <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/nginx-website:latest

Deploy the NGINX website using AWS CloudFormation

Now deploy the NGINX website. The following stack deploys the NGINX website onto the same ECS cluster (clair-demo-cluster) as Clair.

# Create the AWS CloudFormation stack
# <ECRRepositoryUri> - Nginx-Website ECR Repository URI without Image tag
# Example: <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/nginx-website

cd ../aws-codepipeline-docker-vulnerability-scan/

aws cloudformation create-stack \
--stack-name nginx-website-stack \
--template-body file://nginx-website/nginx-website-template.yaml \
--capabilities CAPABILITY_IAM \
--parameters \
ParameterKey="VpcId",ParameterValue="<VpcId>" \
ParameterKey="PublicSubnets",ParameterValue=\"<PublicSubnet01-ID>,<PublicSubnet02-ID>\" \
ParameterKey="PrivateSubnets",ParameterValue=\"<PrivateSubnet01-ID>,<PrivateSubnet02-ID>\" \
ParameterKey="ECRRepositoryUri",ParameterValue="<ECRRepositoryUri>"

# Verify that stack creation is complete
aws cloudformation wait stack-create-complete \
–stack-name nginx-website-stack

# Get stack outputs
aws cloudformation describe-stacks \
--stack-name nginx-website-stack \
--query 'Stacks[].Outputs[]'

Note the AWS CloudFormation stack outputs. The stack output contains the Application Load Balancer URL for the NGINX website and the ECS service name of the NGINX website. You need the ECS service name for the pipeline.

AWS CodePipeline

In this section we will build a CodePipeline to automate the vulnerability scanning of future Nginx-Website docker image builds. The nginx-website folder includes a buildspec.yml file that provides build instructions to CodeBuild.

We will be using Klar, a simple tool to analyse images stored in a private or public Docker registry for security vulnerabilities using CoreOS Clair. Klar serves as a client which coordinates the image checks between the ECR and Clair.

The buildspec.yml we have set the CLAIR_OUTPUT=High. CLAIR_OUTPUT variable defines the severity level threshold, vulnerabilities with severity level higher than or equal to this threshold will be outputted. Supported levels are Unknown, Negligible, Low, Medium, High, Critical, Defcon1. You can configure Klar to your requirements by setting the variables as defined in the Klar documentation.

Deploy the pipeline

Deploy the pipeline using the AWS CloudFormation template provided with the sample code. The following template creates the CodeBuild project, CodePipeline pipeline, Amazon CloudWatch Events rule, and necessary IAM permissions.

# Deploy the pipeline
 
# Replace the following variables 
# WebsiteECRRepositoryARN – NGINX website ECR repository ARN
# WebsiteECRRepositoryURI – NGINX website ECR repository URI
# ClairAlbDnsName - Output variable from coreos-clair-stack
# EcsServiceName – Output variable from nginx-website-stack
# NginxWebsiteCodeCommitRepoArn - Nginx CodeCommit Repository ARN

aws cloudformation create-stack \
--stack-name nginx-website-codepipeline-stack \
--template-body file://clair-codepipeline-template.yaml \
--capabilities CAPABILITY_IAM \
--disable-rollback \
--parameters \
ParameterKey="EcrRepositoryArn",ParameterValue="<WebsiteECRRepositoryARN>" \
ParameterKey="EcrRepositoryUri",ParameterValue="<WebsiteECRRepositoryURI>" \
ParameterKey="ClairAlbDnsName",ParameterValue="<ClairAlbDnsName>" \
ParameterKey="EcsServiceName",ParameterValue="<WebsiteECSServiceName>" \
ParameterKey="NginxWebsiteCodeCommitRepoArn",ParameterValue="<NginxWebsiteCodeCommitRepoArn>"

# Verify that stack creation is complete
aws cloudformation wait stack-create-complete \
–stack-name nginx-website-codepipeline-stack

The pipeline is triggered after the AWS CloudFormation stack creation is complete. You can log in to the AWS Management Console to monitor the status of the pipeline. The vulnerability scan information is available in CodeBuild Logs.

Changing the Base Image

The previous build failed due to vulnerabilities in the base image. Let us now use amazonlinux" base image that is secure.

cd ../my-nginx-website/
PipleineArtifactBucketName
rm -rf Dockerfile

mv Dockerfile-amznlinux Dockerfile

git add *
git commit -m "amazon linux image"
git push

The CodePipeline will be automatically triggered. Monitor the CodeBuild Logs to monitor the vulnerability scan results. The Build process now will be successful as the "amazonlinux" base image meets our security threshold.

Cleanup

1. Delete nginx-website-codepipeline-stack CloudFormation Stack

# Empty the Artifact S3 Bucket
aws s3 rm --recursive s3://<PipleineArtifactBucketName>

# Delete the CloudFormation Stack
aws cloudformation delete-stack --stack-name nginx-website-codepipeline-stack

2. Delete nginx-website-stack CloudFormation Stack

aws cloudformation delete-stack --stack-name nginx-website-stack

3. Delete Nginx Website ECR Repository

aws ecr delete-repository --force --repository-name nginx-website

4. Delete Nginx Website CodeCommit Repository

aws codecommit delete-repository --repository-name my-nginx-website

5. Delete coreos-clair-stack CloudFormation stack.

aws cloudformation delete-stack --stack-name coreos-clair-stack

6. Delete coreos-clair ECR repository.

aws ecr delete-repository --force --repository-name coreos-clair

7. Delete coreos-clair-vpc-stack CloudFormation Stack.

aws cloudformation delete-stack --stack-name coreos-clair-vpc-stack

Summary

I’ve described how to deploy Clair on AWS and set up a release pipeline for the automated vulnerability scanning of container images. The Clair instance can be used as a centralized Docker image vulnerability scanner and used by other CodeBuild projects. To meet your organization’s security requirements, define your vulnerability threshold in Klar by setting the variables, as defined in https://github.com/optiopay/klar.

• CoreOS Clair GitHub repository – https://github.com/coreos/clair
• Klar GitHub repository – https://github.com/optiopay/klar

For more details refer to the Scanning Docker Images for Vulnerabilities using Clair, Amazon ECS, ECR, and AWS CodePipeline

License Summary

This sample code is made available under a modified MIT license. See the LICENSE file.