k6 – Running Smoke Tests in CI to Ensure Scenario Quality

Introduction

Continuous testing in CI is extremely important in product development.

Load testing is no exception to this rule, and we want to avoid situations where we run a load script after a long time only to find that it doesn’t work at all.

In this article, we’ll implement continuous automated testing by running k6 load scripts against a mock API server in CI.

This article assumes that you have already set up a mock API server as described in the following article.
Please refer to it as needed.

https://gonkunblog.com/k6-prepare-mock-server/1934/

Implementation Goal

We’ll define a GitHub Actions CI named mockTestForSampleApp that will run automatically, as seen in this PR.

If you just want to see the conclusion, please refer to the following PR:
https://github.com/gonkunkun/k6-template/pull/3

Setting Up CI

Prerequisites

The explanation is based on the following sample repository:
https://github.com/gonkunkun/k6-template

The components in the CI environment are as follows:

k6

• We’re using xk6 to handle extensions
• The execution command is generated by building with Go

Redis

• Used to store user data
  • Detailed in the following article
  • https://gonkunblog.com/k6-how-to-get-options-settings/1914/

Mock API Server

• Instead of calling the actual API server from k6, we call a mock API server

Setup

Prepare an env file for CI (.env.ci):

AMOUNT_OF_INDEX_SIZE_FOR_TEST_DATA=10000
DEBUG=true
ENV=local
SAMPLE_PRODUCT_ENDPOINT=http://localhost:3005
REDIS_ENDPOINT=redis://localhost:6379

Create .github/workflows/mockTest.yml:

name: mockTest

on: [pull_request]

jobs:
  mockTestForSampleApp:
    runs-on: ubuntu-latest
    services:
      redis:
        image: redis
        ports:
          - 6379:6379

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 21
          cache: 'npm'

      - name: Cache npm directory
        uses: actions/cache@v4
        with:
          path: ~/.npm
          key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-npm-
      - name: Install npm dependencies for k6
        run: npm install

      - name: Set up Go
        uses: actions/setup-go@v5
        with:
          go-version: '1.22'
          cache-dependency-path: '**/*.sum'
      - name: Install xk6
        run: go install go.k6.io/xk6/cmd/xk6@latest

      - name: Cache k6 binary
        uses: actions/cache@v4
        id: cache-k6-binary
        with:
          path: k6
          key: ${{ runner.os }}-k6-${{ hashFiles('**/k6.*') }}
          restore-keys: |
            ${{ runner.os }}-k6-
      - name: Build k6 with plugins
        if: steps.cache-k6-binary.outputs.cache-hit != 'true'
        run: |
          xk6 build 
          --with github.com/LeonAdato/xk6-output-statsd@latest 
          --with github.com/grafana/xk6-dashboard@latest 
          --with github.com/szkiba/xk6-enhanced@latest 
          --with github.com/szkiba/xk6-dotenv@latest

      - name: Set up environment variables
        run: cp .env.ci .env
      - name: Start mock endpoint
        run: |
          cd mock
          npm install
          npx ts-node ./src/index.ts 3005 &

      - name: Cache k6 bundle files
        uses: actions/cache@v4
        id: cache-k6-bundle-files
        with:
          path: dist
          key: ${{ runner.os }}-dist-${{ hashFiles('dist/**') }}
          restore-keys: |
            ${{ runner.os }}-dist-
      - name: Bundle k6 scripts
        if: steps.cache-k6-bundle-files.outputs.cache-hit != 'true'
        run: npm run bundle
      - name: Run k6 smoke test
        run: npm run smoke:sample-product

After setting up the environment, we finally run the smoke test scenario:

        run: npm run smoke:sample-product

Now just create a normal PR and verify that the CI works.

Looking at the CI execution logs, we can confirm that k6 is running successfully.

Note that there are some parts where caching isn’t working properly.
Please forgive this as a future challenge.

Conclusion

With CI, we can now verify that k6 scenarios aren’t broken every time.

The repository used in this article is explained in the following article.
If you’re interested, please check it out.

https://gonkunblog.com/k6-project-template/1846/

How much test code should be written for load testing scripts is debatable.
In this example, we can confirm that k6 executes and the scenario runs completely, but we cannot guarantee the behavior of modules and APIs called within the scenario.

If anyone knows a good way to handle this, please contact me quietly…

Thank you for reading this far.