Leveraging ECR Cache Manifests with GitHub Actions

Contents

Introduction
Target Audience
Background
Problems to Solve
Implementation Approach
Implementation Details
- 1. GitHub Actions Workflow Configuration
- 2. ECR Lifecycle Policy Configuration
Conclusion
References

Introduction

Have you ever felt that Docker builds in your CI pipeline are too slow? It’s not uncommon for builds to take tens of minutes, especially in large-scale projects or microservice architectures. This article introduces how to significantly reduce build times using ECR cache manifests, a feature recently supported by AWS.

Tech Stack: Docker, AWS ECR, GitHub Actions

Target Audience

Users of CI tools like GitHub Actions or CircleCI
AWS ECR users
Anyone interested in optimizing Docker build speeds

Background

In our project, we build Docker images using GitHub Actions, but installing dependencies and compiling assets was time-consuming, resulting in long CI execution times. The main issue was reinstalling the same dependencies for every build.

CI environments run in fresh containers each time, meaning we couldn’t utilize Docker’s standard layer caching. To solve this problem, we implemented ECR cache manifests.

Problems to Solve

Reducing Docker build time in CI environments
Sharing cache between multiple developers and CI environments
Mitigating security risks from outdated caches
Optimizing storage costs

Implementation Approach

ECR cache manifest is a system that uses BuildKit’s functionality to cache build layers in remote registries. AWS ECR officially started supporting this feature in October 2023.

We modified our GitHub Actions workflow file to configure Buildx to store and retrieve cache from ECR. We also set up ECR lifecycle policies to prevent security risks from excessively old caches.

Implementation Details

1. GitHub Actions Workflow Configuration

- uses: docker/setup-buildx-action@v3

- id: fullsha
  run: echo "sha_full=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT

- name: Build and Push ${{ matrix.component.name }} Image
  uses: docker/build-push-action@v5
  with:
    context: .
    file: ${{ matrix.component.dockerfile }}
    target: ${{ steps.env.outputs.environment }}
    push: true
    build-args: ${{ matrix.component.build_args }}
    tags: |
      ${{ env.ECR_REGISTRY }}/${{ env.REPO_PREFIX }}/${{ matrix.component.name }}:${{ steps.fullsha.outputs.sha_full }}
    cache-from: type=registry,ref=${{ env.ECR_REGISTRY }}/${{ env.REPO_PREFIX }}/${{ matrix.component.name }}:buildcache
    cache-to: type=registry,ref=${{ env.ECR_REGISTRY }}/${{ env.REPO_PREFIX }}/${{ matrix.component.name }}:buildcache,mode=max,image-manifest=true,oci-mediatypes=true

Key points of this workflow:

docker/setup-buildx-action@v3: Step to enable BuildKit
cache-from: Specifies where to retrieve the cache from
cache-to: Specifies where to store the cache
mode=max: Makes all layers cacheable
image-manifest=true,oci-mediatypes=true: Settings to ensure compatibility with ECR

2. ECR Lifecycle Policy Configuration

{
  "rules": [
    {
      "rulePriority": 1,
      "description": "Set 7 days expiration for buildcache tags",
      "selection": {
        "tagStatus": "tagged",
        "tagPrefixList": ["buildcache"],
        "countType": "sinceImagePushed",
        "countUnit": "days",
        "countNumber": 7
      },
      "action": {
        "type": "expire"
      }
    },
    {
      "rulePriority": 2,
      "description": "Keep only the 15 most recent images regardless of tag",
      "selection": {
        "tagStatus": "any",
        "countType": "imageCountMoreThan",
        "countNumber": 15
      },
      "action": {
        "type": "expire"
      }
    }
  ]
}

This lifecycle policy:

Expires buildcache tags after 7 days
Keeps only the 15 most recent images regardless of tag, deleting older ones

Conclusion

Implementing ECR cache manifests has significantly reduced our CI pipeline execution time.

Setting appropriate lifecycle policies is also important to balance cost management and security risks from old caches.

References

Tags: #AWS #Docker #ECR #CI #GitHubActions #Performance #Caching