CI Insights

CI Insights helps you monitor your pipelines, detect flaky tests and jobs, and optimize your CI efficiency. This guide walks you through enabling CI Insights via GitHub and covers basic configuration steps.

CI Insights provides monitoring and analysis for two key components of your CI pipeline:

Jobs are the individual tasks that run as part of your CI pipeline (e.g., build, test, deploy steps). CI Insights tracks job-level metrics and health status.

Tests are the individual test cases that run within your test suites. CI Insights captures detailed test-level data to help identify problematic test cases.

For each job and test, CI Insights computes a health status based on the last 10 runs:

• Healthy: The job or test works fine and produces consistent results
• Broken: The job or test consistently fails and needs attention
• Flaky: The job or test is unstable and its outcome is not reliable

This health status helps you quickly identify which components of your CI pipeline need attention.

CI Insights automatically detects flaky behavior in both jobs and tests using the same criteria:

A flaky job or flaky test is one that has 2 different conclusions on the same SHA1. For example, if a job or test runs twice on the same commit and once fails while the other succeeds, it’s considered flaky because the outcome is not consistent with the same code.

1. Enable CI Insights on your repositories by visiting the GitHub Integration page (docs).

2. In order to upload your test results, you’ll need to configure your test. Before doing so, make sure you set your MERGIFY_TOKEN in GitHub Actions secrets.

   To find your token in the Mergify dashboard, go to Settings > CI Insights:

   

   Once copied, paste it into your GitHub Actions secrets:

   

   This token will be used to upload test reports.

3. Click on CI Insights in the Mergify dashboard navigation. You should start seeing your GitHub Actions job runs appear:

   

CI Insights supports many test frameworks and captures test-level data from them. Each framework requires a quick setup to ensure test results are parsed and uploaded properly.

Refer to the dedicated page for your specific framework to configure and verify your test output.