Playwright Integration with Mergify

This guide explains how to integrate Playwright with Test Insights using the @mergifyio/playwright reporter. Once installed, test results are automatically uploaded to Test Insights without any extra workflow changes.

Install the @mergifyio/playwright package alongside @playwright/test to automatically upload your test results to Test Insights.

npm install --save-dev @mergifyio/playwright

yarn add --dev @mergifyio/playwright

pnpm add --save-dev @mergifyio/playwright

Setting up the reporter takes two steps.

Wrap your playwright.config.ts with withMergify:

import { defineConfig } from '@playwright/test';
import { withMergify } from '@mergifyio/playwright';

export default withMergify(
  defineConfig({
    // ... your existing configuration
  })
);

withMergify adds the Mergify reporter while preserving any reporters, globalSetup, and globalTeardown you already have configured.

In your test files, import test and expect from @mergifyio/playwright instead of @playwright/test:

import { test, expect } from '@mergifyio/playwright';

test('logs in', async ({ page }) => {
  // ...
});

This import enables test quarantine: when a quarantined test fails, its outcome is reported as passing so it doesn’t block your pipeline.

Your workflow should run your tests as usual while exporting the secret MERGIFY_TOKEN as an environment variable.

Add the following to the GitHub Actions step running your tests:

env:
  MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}

For example:

- name: Run Tests 🧪
  env:
    MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
  run: npx playwright test

Set MERGIFY_TOKEN as an environment variable in your pipeline step:

steps:
  - label: "Run Tests 🧪"
    command: npx playwright test
    env:
      MERGIFY_TOKEN: "${MERGIFY_TOKEN}"

The reporter automatically collects your test results and sends them to Test Insights.

Check the Test Insights dashboard afterward to view execution metrics, detect flaky tests, and review test trends.

If your playwright.config.ts defines multiple projects, such as one per browser (chromium, firefox, webkit), the same test runs once per project. By default, Test Insights identifies each test by name only, so every project’s copy of a test shares the same identity and is treated as a single test. A test that passes on chromium but fails on webkit then looks flaky instead of consistently broken on one browser.

To keep each project’s tests separate, set PLAYWRIGHT_MERGIFY_INCLUDE_PROJECT_IN_TEST_NAME to true. The reporter then prefixes each test name with its project, for example [chromium] > login.spec.ts > logs in, so Test Insights tracks flakiness and quarantine per project:

env:
  MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
  PLAYWRIGHT_MERGIFY_INCLUDE_PROJECT_IN_TEST_NAME: "true"

This option is opt-in (off by default) to preserve the history of tests that already report without a project prefix.