GitHub Actions: Automate Your CI/CD Pipelines from A to Z
CI/CD & GitOps

GitHub Actions: Automate Your CI/CD Pipelines from A to Z

April 16, 202512 min readGitHub ActionsCI/CDDevOps

GitHub Actions has become the standard for CI/CD on GitHub. Here is a complete guide to building robust, secure, and optimised pipelines.

GitHub Actions Core Concepts

GitHub Actions is an automation engine built into GitHub. It runs workflows defined in YAML, triggered by Git lifecycle events or schedules. Understanding its primitives is essential before moving to advanced patterns.

  • Workflow: YAML file in .github/workflows/, containing one or more jobs. Triggered by events: push, pull_request, schedule, workflow_dispatch, workflow_call
  • Job: execution unit running on a runner. Jobs run in parallel by default; use needs to sequence them
  • Step: shell command or reusable action inside a job. Steps within a job share the filesystem
  • Runner: machine that executes the job — GitHub-hosted (ubuntu-latest, windows-latest, macos-latest) or self-hosted
  • Action: reusable packaged step (uses: actions/checkout@v4) — 20,000+ available on GitHub Marketplace
  • Context: dynamic environment variables exposed by GitHub — github.sha, github.ref, github.event, etc.

Complete Node.js CI/CD Pipeline: Lint → Test → Build → ECR → ECS

Here is a complete production pipeline for a Node.js application deployed on ECS Fargate. It demonstrates essential patterns: npm cache, parallel tests, Docker build with layer caching, AWS OIDC, and deployment with automatic stability check.

# .github/workflows/ci-cd.yml
name: CI/CD Production

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

env:
  AWS_REGION: eu-west-1
  ECR_REPOSITORY: my-app
  ECS_CLUSTER: production
  ECS_SERVICE: my-app-service

permissions:
  id-token: write   # Required for OIDC
  contents: read
  pull-requests: write

jobs:
  # ─── Lint and static checks ────────────────────────────────
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm run lint
      - run: npm run type-check

  # ─── Unit and integration tests ────────────────────────────
  test:
    runs-on: ubuntu-latest
    needs: lint

    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: test
          POSTGRES_DB: testdb
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - name: Run tests with coverage
        run: npm run test:coverage
        env:
          DATABASE_URL: postgresql://postgres:test@localhost:5432/testdb
      - uses: codecov/codecov-action@v4
        with:
          token: ${{ secrets.CODECOV_TOKEN }}

  # ─── Docker build and push to ECR ──────────────────────────
  build:
    runs-on: ubuntu-latest
    needs: test
    if: github.ref == 'refs/heads/main'
    outputs:
      image: ${{ steps.login-ecr.outputs.registry }}/${{ env.ECR_REPOSITORY }}:${{ github.sha }}

    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials via OIDC
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Build and push with layer caching
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: |
            ${{ steps.login-ecr.outputs.registry }}/${{ env.ECR_REPOSITORY }}:${{ github.sha }}
            ${{ steps.login-ecr.outputs.registry }}/${{ env.ECR_REPOSITORY }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

  # ─── ECS Deployment ────────────────────────────────────────
  deploy:
    runs-on: ubuntu-latest
    needs: build
    environment: production

    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials via OIDC
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Deploy to ECS
        run: |
          aws ecs update-service             --cluster ${{ env.ECS_CLUSTER }}             --service ${{ env.ECS_SERVICE }}             --force-new-deployment
          aws ecs wait services-stable             --cluster ${{ env.ECS_CLUSTER }}             --services ${{ env.ECS_SERVICE }}

Reusable Workflows and Composite Actions: DRY Principle

Reusable workflows (workflow_call) allow centralising shared CI/CD logic across multiple repositories. This is the equivalent of reusable functions in programming — essential for organisations with many projects.

# .github/workflows/reusable-deploy-ecs.yml
# Callable from any repository in the organisation
name: Deploy to ECS (Reusable)

on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
      image-tag:
        required: true
        type: string
      ecs-cluster:
        required: true
        type: string
      ecs-service:
        required: true
        type: string
      aws-region:
        required: false
        type: string
        default: "eu-west-1"
    secrets:
      AWS_ROLE_ARN:
        required: true

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment }}
    steps:
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: ${{ inputs.aws-region }}
      - run: |
          aws ecs update-service             --cluster ${{ inputs.ecs-cluster }}             --service ${{ inputs.ecs-service }}             --force-new-deployment
          aws ecs wait services-stable             --cluster ${{ inputs.ecs-cluster }}             --services ${{ inputs.ecs-service }}

---
# From another repository: call the reusable workflow
jobs:
  deploy-staging:
    uses: my-org/devops-workflows/.github/workflows/reusable-deploy-ecs.yml@main
    with:
      environment: staging
      image-tag: ${{ github.sha }}
      ecs-cluster: staging-cluster
      ecs-service: my-app-staging
    secrets:
      AWS_ROLE_ARN: ${{ secrets.AWS_STAGING_ROLE_ARN }}

  deploy-production:
    needs: deploy-staging
    uses: my-org/devops-workflows/.github/workflows/reusable-deploy-ecs.yml@main
    with:
      environment: production
      image-tag: ${{ github.sha }}
      ecs-cluster: prod-cluster
      ecs-service: my-app-prod
    secrets:
      AWS_ROLE_ARN: ${{ secrets.AWS_PROD_ROLE_ARN }}

Matrix Builds: Parallel Tests Across Versions

Matrix builds run tests in parallel across multiple Node.js versions, operating systems, or configurations. They dramatically reduce total CI time.

jobs:
  test-matrix:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        node: [18, 20, 22]
        os: [ubuntu-latest, windows-latest]
        include:
          - node: 22
            os: macos-latest
            experimental: true
        exclude:
          - node: 18
            os: windows-latest
      fail-fast: false  # Continue even if one combination fails

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
          cache: 'npm'
      - run: npm ci
      - run: npm test
      - if: always()
        uses: actions/upload-artifact@v4
        with:
          name: test-results-node${{ matrix.node }}-${{ matrix.os }}
          path: test-results/

Caching Strategies: Cutting CI Time

Caching is the highest-impact lever for reducing pipeline duration. A Node.js pipeline without cache can take 4–6 minutes; with well-configured cache, 45 seconds to 90 seconds.

# Cache node_modules — saves 2-3 min per run
- uses: actions/cache@v4
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

# Docker layer cache via GitHub Actions Cache backend
- uses: docker/setup-buildx-action@v3

- uses: docker/build-push-action@v5
  with:
    cache-from: type=gha
    cache-to: type=gha,mode=max  # Caches all intermediate layers

# Next.js build cache
- uses: actions/cache@v4
  with:
    path: |
      ~/.npm
      ${{ github.workspace }}/.next/cache
    key: ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}-${{ hashFiles('**/*.ts', '**/*.tsx') }}
    restore-keys: |
      ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}-
      ${{ runner.os }}-nextjs-

Environments and Deployment Protection Rules

GitHub Actions environments isolate secrets by deployment target and enforce approval gates before any production deployment. This is the governance control essential for teams with change management requirements.

# Configure in GitHub UI: Settings → Environments → production
# Protection rules:
# [ ] Required reviewers: 2 members of the SRE team
# [ ] Wait timer: 5 minutes (cancellation window)
# [ ] Deployment branches: main only

jobs:
  deploy-staging:
    environment:
      name: staging
      url: https://staging.my-app.com
    runs-on: ubuntu-latest
    steps:
      - run: ./deploy.sh staging

  deploy-production:
    needs: [deploy-staging, integration-tests]
    environment:
      name: production
      url: https://my-app.com
    runs-on: ubuntu-latest
    steps:
      - run: ./deploy.sh production
      # This job waits for reviewer approval before executing steps

OIDC with AWS: Zero Stored Credentials in GitHub

OIDC authentication eliminates the need to store AWS credentials (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY) in GitHub Secrets. GitHub issues a signed JWT that AWS validates directly via STS. This is the most secure method and the one we recommend universally.

# 1. Terraform: create the OIDC provider and IAM role
resource "aws_iam_openid_connect_provider" "github_actions" {
  url             = "https://token.actions.githubusercontent.com"
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = ["6938fd4d98bab03faadb97b34396831e3780aea1"]
}

resource "aws_iam_role" "github_actions_deploy" {
  name = "GitHubActionsDeployRole"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Federated = aws_iam_openid_connect_provider.github_actions.arn }
      Action    = "sts:AssumeRoleWithWebIdentity"
      Condition = {
        StringEquals = {
          "token.actions.githubusercontent.com:aud" = "sts.amazonaws.com"
        }
        StringLike = {
          # Restrict to specific repository and main branch only
          "token.actions.githubusercontent.com:sub" = "repo:my-org/my-repo:ref:refs/heads/main"
        }
      }
    }]
  })
}

# 2. Workflow: OIDC usage (no AWS secrets stored in GitHub)
- name: Configure AWS credentials
  uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: arn:aws:iam::123456789:role/GitHubActionsDeployRole
    aws-region: eu-west-1
    role-session-name: GitHubActions-${{ github.run_id }}
    # No aws-access-key-id or aws-secret-access-key needed!

Self-Hosted Runners on ECS Fargate: Ephemeral and Cost-Effective

GitHub-hosted runners cost $0.008/min. A 10-minute pipeline costs $0.08 — $80 for 1,000 runs. With self-hosted runners on ECS Fargate Spot, the cost drops to ~$0.001/min. For high-volume organisations, the saving is significant. An additional benefit: runners run inside your VPC and access private resources without public exposure.

# Actions Runner Controller (ARC) on EKS
# Runners are ephemeral Kubernetes pods — scale to zero when idle

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: m2c-runners
  namespace: github-runners
spec:
  replicas: 0  # Scale to zero when no jobs pending
  template:
    spec:
      repository: my-org/my-repo
      image: summerwind/actions-runner:latest
      resources:
        requests:
          cpu: "500m"
          memory: "1Gi"
        limits:
          cpu: "2"
          memory: "4Gi"
      serviceAccountName: github-runner-sa

---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: m2c-runners-autoscaler
spec:
  scaleTargetRef:
    name: m2c-runners
  minReplicas: 0
  maxReplicas: 10
  metrics:
    - type: TotalNumberOfQueuedAndInProgressWorkflowRuns
      repositoryNames:
        - my-org/my-repo

Secrets Management: GitHub Secrets vs AWS Secrets Manager

GitHub Secrets are appropriate for small teams and static secrets. For larger teams with secret rotation requirements, secrets shared across environments, or dynamic secrets, AWS Secrets Manager (accessible via OIDC) is preferable:

# Recommended pattern: fetch AWS secrets at runtime via OIDC
# No sensitive secrets stored in GitHub

- name: Configure AWS credentials (OIDC)
  uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
    aws-region: eu-west-1

- name: Fetch secrets from AWS Secrets Manager
  run: |
    DB_PASSWORD=$(aws secretsmanager get-secret-value       --secret-id prod/app/db-password       --query SecretString --output text)
    echo "DB_PASSWORD=$DB_PASSWORD" >> $GITHUB_ENV
    echo "::add-mask::$DB_PASSWORD"  # Mask in logs

# Or use the official action
- uses: aws-actions/aws-secretsmanager-get-secrets@v2
  with:
    secret-ids: |
      prod/app/db-password
      prod/app/api-key
    parse-json-secrets: true

Pipeline Optimisation: Path Filters and Concurrency Groups

For monorepos or projects with independent components, triggering all tests on every commit is wasteful. Path filters and concurrency groups dramatically optimise CI throughput.

# Path filters: only build what changed
on:
  push:
    paths:
      - 'backend/**'   # Trigger only if backend changed
      - '!docs/**'     # Ignore documentation changes
      - '!*.md'

# Concurrency groups: cancel previous runs on the same branch
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

# For main branch: don't cancel (let deployments finish)
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}

# Advanced conditions: differentiate PR vs push to main
jobs:
  deploy:
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    # This job only runs on main branch push, not on PRs

Monitoring: GitHub Actions Dashboard and Datadog CI

GitHub Actions provides a native dashboard (Actions tab — all workflows, durations, success rates). For advanced CI observability, Datadog CI provides detailed metrics: duration per step, flaky test detection, cost per pipeline.

# Datadog CI integration
- name: Install Datadog CI
  run: npm install -g @datadog/datadog-ci

- name: Run tests with Datadog tracing
  run: npm test
  env:
    DD_CIVISIBILITY_AGENTLESS_ENABLED: true
    DD_API_KEY: ${{ secrets.DD_API_KEY }}
    DD_SITE: datadoghq.eu

# GitHub-hosted runner costs (2025):
# ubuntu-latest (2 vCPU) : $0.008/min
# ubuntu-latest-4-cores  : $0.016/min
# ubuntu-latest-8-cores  : $0.032/min
# windows-latest         : $0.016/min
# macos-latest           : $0.08/min

# For 1,000 runs of 10 min on ubuntu:
# GitHub-hosted : 1,000 × 10 × $0.008 = $80/month
# Self-hosted Fargate Spot : ~$8/month (90% saving)

Workflow Security Best Practices

GitHub Actions workflows can become attack vectors if misconfigured. Essential practices:

  • Pin actions by commit SHA: uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 rather than @v4 — prevents supply chain attacks via tag hijacking
  • Minimal permissions: declare permissions: read-all at the workflow level and grant only what each job needs
  • Fork pull requests: workflows triggered by pull_request from a fork do not have access to secrets — this is an intentional protection, do not circumvent it
  • CODEOWNERS on workflows: add .github/workflows/ @team-devops to CODEOWNERS so any workflow modification requires review

Conclusion

GitHub Actions covers the full CI/CD spectrum, from simple test automation to complex multi-environment deployments with governance. Reusable workflows eliminate duplication across projects, matrix builds parallelise tests across all target configurations, and OIDC integration with AWS eliminates the last reason to store cloud credentials in GitHub. For high-volume organisations, self-hosted runners on ECS Fargate Spot reduce CI costs by 80–90%. GitHub Actions is now mature enough to replace Jenkins in virtually all use cases — with an infinitely shorter learning curve and a native Git integration that Jenkins cannot match.

← Previous article
Azure AKS in Production: Complete Guide
April 7, 2025 · 11 min
Next article →
Multi-Cloud: Strategy and Architecture to Avoid Vendor Lock-in
April 25, 2025 · 10 min
← Back to blog