--- title: Fail First description: When a batch fails, Fail First makes the search for the pull request that broke it faster and cheaper, without ever skipping a test. --- When a batch fails, Mergify has to find the pull request that broke it, and it does so by retesting smaller splits of the batch. Fail First makes that search faster and cheaper: every split starts with the tests that already failed, and stops early when they fail again. On a suite that takes 40 minutes, a split holding the culprit still stops after the few tests that had failed. No test is ever skipped to turn a split green. ## What it looks like A batch of 8 pull requests failed, with 3 of its 96 tests red. Mergify [split it](/merge-queue/batches#handling-batch-failure-or-timeout) in two rounds to find the culprit, and Fail First ran those 3 tests first in every split: ```dot class="queue" title="A real run: the splits holding PR5 stopped after 3 of 96 tests, the others ran all 96 and went green" digraph { node [shape=box]; batch [label="Batch PR1 to PR8\n3 of 96 tests failed", class="failed"]; subgraph cluster_round_1 { class="batch"; label="Round 1, all at once"; s2 [label="PR1 to PR2\nall 96 tests pass · 67 s", class="merged"]; s4 [label="PR1 to PR4\nall 96 tests pass · 72 s", class="merged"]; s6 [label="PR1 to PR6\nstopped after 3 tests · 15 s", class="failed"]; s7 [label="PR1 to PR7\nstopped after 3 tests · 17 s", class="failed"]; } subgraph cluster_round_2 { class="batch"; label="Round 2, once PR1 to PR4 passed"; s5 [label="PR1 to PR5\nstopped after 3 tests · 14 s", class="failed"]; } culprit [label="PR5 broke the batch", class="failed plain", shape=plaintext]; batch -> s2; batch -> s4; batch -> s6; batch -> s7; s6 -> s5; s5 -> culprit [class="chrome"]; } ``` The splits without the culprit ran the whole suite and went green, as they would without Fail First. The splits holding it cost a few seconds each instead of a full run. ## How it works Fail First reorders the tests in a split: ```dot class="queue" title="Tests 2, 5 and 7 failed in the batch: Fail First runs them first, the others follow in their usual order" digraph { nodesep=0.15; node [shape=box, width=0.9]; // Graphviz stacks the second row on top: the usual order is declared last // so that it reads first. subgraph cluster_fail_first { class="batch"; label="With Fail First"; f2 [label="test 2", class="failed"]; f5 [label="test 5", class="failed"]; f7 [label="test 7", class="failed"]; f1 [label="test 1", class="config"]; f3 [label="test 3", class="config"]; f4 [label="test 4", class="config"]; f6 [label="test 6", class="config"]; f8 [label="test 8", class="config"]; f2 -> f5 -> f7 -> f1 -> f3 -> f4 -> f6 -> f8 [class="chrome"]; } subgraph cluster_usual { class="batch"; label="Usual order"; u1 [label="test 1", class="config"]; u2 [label="test 2", class="failed"]; u3 [label="test 3", class="config"]; u4 [label="test 4", class="config"]; u5 [label="test 5", class="failed"]; u6 [label="test 6", class="config"]; u7 [label="test 7", class="failed"]; u8 [label="test 8", class="config"]; u1 -> u2 -> u3 -> u4 -> u5 -> u6 -> u7 -> u8 [class="chrome"]; } } ``` If one of the tests that failed in the batch fails again, the run stops early: that split holds the culprit, and it gives its CI runner back without running the rest. If they all pass, the run goes on with the rest of the suite, so a green split has run every test. Frameworks that run tests by file put first the files that hold those tests, so the other tests of those files run early too. In a sharded job, each shard starts with the failed tests it holds, and a shard that holds none runs in its usual order. Pull request CI, the batch's own run and Mergify's retries keep your usual order. ## What you need Fail First is on by default. A job gets it when: - your merge queue uses [batches](/merge-queue/batches); - the job runs the Mergify plugin for your [test framework](/test-engine#test-framework-configuration), version 2026.10.8 or later, set up with your `MERGIFY_TOKEN` as for the rest of Test Engine. There is nothing else to add to your jobs. Jobs that only upload JUnit reports keep their usual order. To turn it off on a job, set `MERGIFY_FAIL_FIRST_ENABLE` to `false` there. ## In the dashboard The Fail First section of **Test Engine > [Configuration](https://dashboard.mergify.com/test-engine/configuration)** says whether it runs on your repository and on which jobs. Over the last 30 days, it counts the splits that got it and the ones that stopped early, and estimates the CI time it saved. It also offers to install or update the plugin on the jobs that cannot use it yet. ## In your CI log The plugin says what Fail First did at the end of the job log, where a split is called a bisection step. A split that holds the culprit stops early, so its log lists the tests it did not run: ```text ⏩ Fail First 3 tests failed in the previous bisection step, so they ran first. 3 of them failed again, so Mergify stopped the run early: the merge queue can split this batch again without waiting. 93 tests did not run. Failed again: - tests/test_taxes.py::test_taxes_adds_vat_to_net_price - tests/test_taxes.py::test_taxes_adds_vat_to_small_amounts - tests/test_taxes.py::test_taxes_adds_vat_to_order_total ``` ## Limits ### Tests that depend on their order Fail First moves the tests that failed in the batch, or the files that hold them, to the front of the suite; the others keep their order. A test that only passes once other tests have prepared something for it, such as a database row, a global setting or a file, can then fail just because it runs first, even though no pull request broke it. The split goes red, and the merge queue can remove the wrong pull request. If your suite has such tests, set `MERGIFY_FAIL_FIRST_ENABLE` to `false` on the jobs that run them. ### A summary check that waits for every job Some CI setups add a summary job that depends on all the others, such as an "all checks passed" job, and make it the only check the [merge conditions](/merge-queue/rules#queue-vs-merge-conditions) require. That job starts only once every other job has finished, so a split goes red when its slowest job ends, even if a test job stopped after a few seconds. Fail First still saves the CI time of the tests that did not run, but the culprit is found no sooner. To get both, require the checks of your test jobs themselves in your merge conditions: the split then goes red as soon as one of them fails.