Mocha Integration with Mergify

This guide explains how to integrate Mocha with Test Engine using the @mergifyio/mocha plugin. The plugin uploads your test results to Test Engine when the run ends, with no JUnit file to write or upload, and applies quarantine and flaky detection inside the Mocha run. It takes up no reporter, so Mocha’s output stays the one you read today.

If your CI uploads a JUnit report from Mocha today, see Moving from a JUnit upload.

• Node.js 22 or later in the job that runs the tests. Older versions are not supported: npm and pnpm only warn at install, Yarn 1 refuses.

• Mocha 8.2 or later, the first with the global fixtures the plugin registers. On Mocha 8.2, run your tests serially: its own --parallel mode crashes in its worker pool on current Node.js, with or without the plugin. Mocha 8.3 and later run in either mode. On Node.js 26, Mocha 8.3 to 10.5 crash on start, again with or without the plugin: they pin a yargs release that Node.js 26 loads as an ES module. Move those to Mocha 10.6 or later.

• Mocha started from its command line, which loads the plugin through --require. Code that runs Mocha through its API passes the plugin’s fixtures itself: see Running Mocha through its API.

• Mocha running in Node.js. Mocha in a browser (a page, Karma, web-test-runner, Cypress) is not supported. For Cypress, see the Cypress page.

Install the @mergifyio/mocha package alongside mocha to automatically upload your test results to Test Engine.

Use the package manager version your project pins (packageManager in package.json, mise.toml, .tool-versions), and check that the lockfile diff only adds the new packages: another version rewrites unrelated entries (older npm releases, for example, drop the libc fields of other packages; npx npm@latest install … avoids it).

npm install --save-dev @mergifyio/mocha

yarn add --dev @mergifyio/mocha

pnpm add --save-dev @mergifyio/mocha

Have Mocha require the plugin in your Mocha configuration file:

require:
  - '@mergifyio/mocha'

With a JavaScript configuration:

module.exports = {
  require: ['@mergifyio/mocha'],
};

Or on the command line:

npx mocha --require @mergifyio/mocha

If your configuration already has a require list, add the plugin to it. Mocha merges the require lists of its configuration and of its command line, so a --require of your own keeps the plugin.

The plugin registers Mocha’s global setup and teardown fixtures, which Mocha runs once per run in its main process, whether it runs your files there or in its workers (--parallel). It takes up no reporter: keep spec, mocha-junit-reporter, mochawesome or any other.

The plugin reports nothing, and says so, in watch mode and for a dry run (--dry-run), where Mocha reports every test as passed without running it. --dry-run exists from Mocha 9. Mocha 8 ignores the flag and runs your tests, so the plugin reports them like any other run.

Code that runs Mocha through its API instead of its command line does not read --require. Pass the plugin’s fixtures yourself:

import { mergify } from '@mergifyio/mocha';
import Mocha from 'mocha';

const { mochaGlobalSetup, mochaGlobalTeardown } = mergify();
const mocha = new Mocha({ globalSetup: mochaGlobalSetup, globalTeardown: mochaGlobalTeardown });

Mocha 12 is an ES module: import it, as above. Node.js 22’s require hands it over as a namespace rather than as Mocha’s constructor.

The plugin uploads results only when the CI environment variable is set, or when MERGIFY_CI_ENABLE is on. GitHub Actions, CircleCI and Buildkite set CI for you. Jenkins does not: set CI=true, or MERGIFY_CI_ENABLE=true, in the stage that runs your tests. If your tests behave differently when CI is set (some test suites skip or add tests on CI), set MERGIFY_CI_ENABLE=true instead of CI, so the tests that run stay the same.

CI counts as set unless it is empty or a false value (false, no, off, 0, f or n), so true, 1 and a CI system’s name (Woodpecker sets CI to woodpecker) all count. MERGIFY_CI_ENABLE is stricter: only true, yes, on, 1, t or y turn it on. Case does not matter for either.

Without MERGIFY_TOKEN the plugin uploads nothing and your tests run as usual. Pull requests from forks and Dependabot pull requests on GitHub Actions get no repository secrets, so they report nothing: no guard is needed in the workflow. (The RSpec gem logs an authentication error when the variable is set but empty, as on those pull requests; the tests are not affected.) To have Dependabot pull requests report too, store the key as a Dependabot secret as well (gh secret set MERGIFY_TOKEN --app dependabot --repo <owner>/<repository>).

Every test is identified by its name and the name of the CI job that ran it. By default the job name is the job’s identifier in your CI:

Set MERGIFY_TEST_JOB_NAME when that default is not one stable name per set of tests:

• GitHub Actions matrix: every leg shares one job id. Set it per leg from the matrix values, for example MERGIFY_TEST_JOB_NAME: test (${{ matrix.python-version }}, ${{ matrix.os }}).

• Two test runs in one job: set a distinct name on each step.

• Reusable workflows: a called workflow reports under the caller’s workflow name, so two called workflows whose jobs share an id (often test) collide. Set a distinct name in each.

• Shards of one test suite are the same tests split across machines: give all shards the same name. Never put a shard index in it.

In a Jenkins multibranch pipeline, every branch and every pull request of the project reports its tests under the same pipeline name and job name: the multibranch project (my-repo, not my-repo/PR-123). MERGIFY_TEST_JOB_NAME replaces the job name but never the pipeline name, so it cannot file the tests of two Jenkins pipelines together. Inside a container, pass BRANCH_NAME through as listed below, or the tests are filed under JOB_NAME.

tox, docker run and Docker Compose start the tests in an environment that does not inherit the CI job’s variables. Pass through MERGIFY_TOKEN, CI, MERGIFY_TEST_JOB_NAME if you set it, and every variable the plugin reads to identify the run. tox’s pass_env accepts globs (GITHUB_*); Docker Compose, docker run -e and the Buildkite Docker Compose plugin need each name:

GitHub Actions: GITHUB_ACTIONS GITHUB_BASE_REF GITHUB_EVENT_NAME GITHUB_EVENT_PATH
  GITHUB_HEAD_REF GITHUB_JOB GITHUB_REF_NAME GITHUB_REF_TYPE GITHUB_REPOSITORY
  GITHUB_REPOSITORY_ID GITHUB_RUN_ATTEMPT GITHUB_RUN_ID GITHUB_SERVER_URL
  GITHUB_SHA GITHUB_WORKFLOW RUNNER_NAME
CircleCI: CIRCLECI CIRCLE_BRANCH CIRCLE_BUILD_URL CIRCLE_JOB
  CIRCLE_REPOSITORY_URL CIRCLE_SHA1 CIRCLE_WORKFLOW_ID
Buildkite: BUILDKITE BUILDKITE_AGENT_NAME BUILDKITE_BRANCH BUILDKITE_BUILD_ID
  BUILDKITE_BUILD_URL BUILDKITE_COMMIT BUILDKITE_LABEL BUILDKITE_PIPELINE_SLUG
  BUILDKITE_PULL_REQUEST BUILDKITE_PULL_REQUEST_BASE_BRANCH BUILDKITE_REPO
  BUILDKITE_RETRY_COUNT BUILDKITE_STEP_KEY
Jenkins: JENKINS_URL BUILD_ID BUILD_URL JOB_NAME BRANCH_NAME NODE_NAME GIT_URL
  GIT_BRANCH GIT_COMMIT CHANGE_ID CHANGE_TARGET CHANGE_BRANCH CHANGE_FORK
  TAG_NAME

Pass them by name, never by writing a value into a file. In a container on GitHub Actions, also mount the file GITHUB_EVENT_PATH points to, at the same path: without it the run is recorded against the merge commit instead of the pull request’s head commit.

Depending on its version and persona, zizmor can report secrets-outside-env on the MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }} line. Run the repository’s own zizmor configuration on the changed workflow, and if it reports the finding, follow the convention the repository already uses for it: a deployment environment on the job, or an inline # zizmor: ignore[secrets-outside-env] comment.

Your workflow should run your tests as usual while exporting the secret MERGIFY_TOKEN as an environment variable.

Add the following to the GitHub Actions step running your tests:

env:
  MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}

For example:

- name: Run Tests 🧪
  env:
    MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
  run: npx mocha

Set MERGIFY_TOKEN in the environment of the agents running your tests. The step itself then needs no Mergify-specific configuration:

steps:
  - label: "Run Tests 🧪"
    command: npx mocha

The plugin automatically collects your test results and sends them to Test Engine.

Before any test runs, the plugin fetches the list of tests quarantined in Test Engine. When one of them fails, Mocha counts it as passed: it does not fail your run or stop it under --bail, and your reporter shows it passed. The test reaches Test Engine as failed and quarantined. This works the same whether Mocha runs your files in its main process or in its workers (--parallel). Once the run is over, the plugin’s Mergify CI section says what the quarantine did.

A failing hook does not fail your run when every test it kept from running is quarantined. A failing after hook keeps no test from running, so quarantine never covers it.

The exit code of your test step already accounts for quarantine, so the step needs nothing more:

• Do not add continue-on-error: true. The recipes that upload a JUnit report with the mergifyio/gha-mergify-ci action need it because the action decides the job’s result after the tests. Here nothing does, so it would let every real failure through.

• There is no step id to set and no test_step_outcome to pass: both belong to that action, which this setup does not use.

A crash cannot pass for a green run either. An error no test raised still makes Mocha exit non-zero and fail the step, and so does Mocha dying mid-run. The plugin uploads results when the run ends, so a process killed outright before then, such as by the out-of-memory killer, sends nothing to Test Engine.

If the plugin cannot fetch the quarantine list, it quarantines nothing for that run, and a quarantined test that fails makes the step fail as usual.

Every test of the run reaches Test Engine as it ended: passed, failed with its error, or skipped.

A failing hook stops Mocha from running tests, and Mocha reports only the hook. The plugin reports each test the hook kept from running as failed, with the hook’s error:

• A failing before hook: every test of its describe, nested ones included, that --grep would have run.

• A failing beforeEach hook: the test it ran for, and the rest of its describe.

• A failing afterEach hook: the test it ran after, and the rest of its describe.

• A failing after hook fails no test. It fails the run.

A test that Mocha’s retries ran again reaches Test Engine as its first run, with how it ended and what it raised. Every run after it is counted, and the test’s duration covers them all.

The plugin reruns tests within the budgets set under Native Flaky Detection:

• On a pull request, a test the target branch does not have yet is new. The plugin runs it again right after its own run, with its beforeEach and afterEach hooks around each run, as many times as fit in the Prevention Budget. When one of those runs fails, the test is flaky: Mocha reports it failed, with that run’s error, so the pull request does not merge. See Prevention.

• On a push, the plugin reruns the tests Test Engine knows as unhealthy, within the Detection Budget, to report whether they still are. The run’s result does not change. See Rerunning unhealthy tests.

The runs Mocha’s retries made of a new test count too: a new test that retries rescued is flaky, and Mocha reports it failed with its first run’s error. Those runs do not use up the budget. Your reporter hears of no rerun, and Mocha’s retries do not apply to the reruns. The Mergify CI section says what the reruns found.

In a parallel run (--parallel), the reruns wait until the run’s tests are over. Then, before Mocha’s global teardown, Mocha’s workers run again in full each file that holds a test to rerun, and run that test as many times as fit. Your reporter has printed the run by then, so it shows a new test found flaky as passed: the Mergify CI section names it, and the run fails. A parallel run that --bail stopped reruns nothing.

Each test reaches Test Engine named by its file, relative to the directory Mocha runs in, the describe blocks around it, and its own title:

test/api.spec.js > users > creates one

The file is part of the name because tests declared outside any describe often share a title across files (works), and would otherwise be one test in Test Engine. Test Engine tracks a test’s history under that name, so renaming the file, a describe block or the test starts a new test.

The plugin uploads each run’s results under your CI job’s name. On GitHub Actions, that name is the job’s identifier (GITHUB_JOB, recorded with the workflow name), which every leg of a matrix shares. The job name is part of a test’s identity in Test Engine, so when the legs run the same suite on several Node.js versions, their results merge into one test.

Set MERGIFY_TEST_JOB_NAME on each leg and include the matrix value, so each leg reports under its own name:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [22, 24]
    steps:
      # ...
      - name: Run Tests 🧪
        env:
          MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
          MERGIFY_TEST_JOB_NAME: test (node ${{ matrix.node-version }})
        run: npx mocha

Setting MERGIFY_TEST_JOB_NAME on a job that already reports changes its name, so its tests start a new history in Test Engine.

If your CI writes a JUnit report with Mocha’s xunit reporter or mocha-junit-reporter and uploads it with the mergifyio/gha-mergify-ci action, the mergifyio/mergify-ci Buildkite plugin or mergify ci junit-process, switch to @mergifyio/mocha:

1. Install @mergifyio/mocha and add it to Mocha’s require, as shown in Configuration.

2. Keep your reporter if another tool still reads the JUnit file. Otherwise, go back to the reporter you read in your terminal, and drop mocha-multi-reporters if you only added it to keep that output. The plugin needs no reporter.

3. Remove the upload: the mergifyio/gha-mergify-ci step, the mergifyio/mergify-ci plugin on the Buildkite step, or the mergify ci junit-process command. @mergifyio/mocha sends the results itself. If you ran mergify ci junit-process, also drop || exit_code=$? from your test command, so its exit code is the step’s again.

4. On GitHub Actions, remove continue-on-error: true from the test step. If you added a step for pull requests without the secret that fails the job when no report was uploaded, remove it too.

5. If your upload set a job_name, as a matrix setup does, move its value to MERGIFY_TEST_JOB_NAME on the step that runs the tests, as shown in Matrix runs, so your tests keep the job name they report under.

6. Set MERGIFY_TOKEN on the step that runs the tests, as shown in Update your CI workflow.

What changes in Test Engine once the plugin uploads:

• Your tests get new names, so their history starts again under those names.

• Quarantine entries made from your JUnit uploads still apply when the upload named a test the way xunit does, or the way mocha-junit-reporter does with its default options. With jenkinsMode on and suiteTitleSeparatedBy set to a space, mocha-junit-reporter gives the same name as xunit.

• Flaky detection judges whether a test is new by its name. Until your target branch has run with the plugin, it has none of the new names, so a pull request reruns its tests as new ones within the Prevention Budget, and a test that was already flaky can fail the job as a new flaky test.

After pushing these changes, your next CI run reports its Mocha results automatically.

You can then review your test results, including any failures or flaky tests, directly in the Test Engine dashboard.

To check the setup without uploading anything, run your tests with MERGIFY_CI_ENABLE=true MERGIFY_CI_DEBUG=true: the plugin prints the spans it would send.