Merge Queue Batches
Increase your merge queue throughput and decrease your CI usage.
Mergify's batch merging feature is a powerful tool that enhances productivity and efficiency in your development workflow. Instead of merging pull requests one by one, which can be time-consuming, especially for larger projects, batch merging allows you to combine and merge multiple pull requests at once.
The batch merging process works by setting a
batch_size option on your merge
queue. This option determines the number of pull requests that Mergify will
combine into a single draft pull request. The draft pull request is then
validated by your Continuous Integration (CI) system. If the CI validation
passes, Mergify merges all the pull requests in the batch and closes the draft
If a failure occurs, Mergify identifies the problematic pull request(s) automatically and removes them from the queue, allowing the rest of the queue to be processed as usual.
This feature is especially useful for large-scale projects with many pull requests to merge, as it can significantly reduce the time required to merge them all.
Batch merging works by utilizing the
batch_size option in your Mergify queue
configuration. This option tells Mergify how many pull requests it should try
to combine and validate at once. The
batch_size can be adjusted according to
the needs of your project, with a higher number indicating a larger batch size.
For example, if you set
batch_size to 3, Mergify will create a draft pull
request that includes the changes from the next three pull requests in the
queue. This draft pull request is then validated by your CI system. If all
checks pass, then the three pull requests included in the batch are deemed
ready to be merged.
queue_rules:- name: defaultbatch_size: 3...
In case the draft pull request fails CI validation, it means one or more pull requests in the batch have introduced changes that are causing the failure. Mergify handles this scenario by using a binary search method to identify the failing pull request. It does this by systematically reducing the batch size and re-running the CI checks until it finds the problematic pull request. Once found, the failing pull request is removed from the queue and the rest of the pull requests are processed as usual.
This batch merging process allows multiple pull requests to be validated and merged more efficiently than if they were handled individually.
Configuring batch merging involves adjusting your Mergify configuration file to
batch_size option according to your project's needs.
Open your Mergify configuration file.
queue_rules, set the
batch_sizeoption to the number of pull requests you want to be tested together in a batch.
queue_rules:- name: defaultbatch_size: 3...
In the above example, batches of up to 3 pull requests will be created and tested together.
You can configure the delay that Mergify will use to wait for the batch to be
filled up using the
queue_rules:- name: defaultbatch_size: 5batch_max_wait_time: 5 min...
With the configuration above, Mergify waits up to 5 minutes for 5 PR to enter the queue before creating a batch. This allows you to pick the right trade-off between latency and minimal CI usage.
Batch merging and speculative checks are two powerful features that work in synergy to improve the efficiency of your merge queue.
Batch merging allows Mergify to test multiple pull requests together as a single unit, reducing the amount of time waiting for individual pull request tests to complete. On the other hand, speculative checks allow for multiple batches to be tested in parallel, further speeding up the merge process.
When both these features are enabled, Mergify creates multiple batches of pull
requests (according to the
batch_size option) and then runs tests on several
of these batches at the same time (as defined by the
option). If any pull request within a batch fails, Mergify identifies the
culprit through a binary search, removes it from the queue, and continues
processing the rest of the queue.
queue_rules:- name: defaultspeculative_checks: 2batch_size: 3...
In the above example, Mergify will create up to 2 batches, each containing up to 3 pull requests, and test them in parallel.
Combining these two features allows you to optimize the throughput of your merge queue. You can increase the batch size to merge more pull requests concurrently, while also increasing the number of speculative checks to test more batches in parallel. This minimizes idle time and makes full use of your CI resources.
Suppose your queue has 7 pull requests waiting, and your CI pipeline takes
about 10 minutes to complete. If you set
batch_size to 3 and
speculative_checks to 2, Mergify would create 2 batches, each containing 3
pull requests. These batches are then tested in parallel.
With this configuration, even if your CI time is 10 minutes, you can merge the first 6 pull requests in only 10 minutes, as opposed to the 1 hour it would typically take to test each pull request individually.
Check out our performance page for more information.
By default, Mergify creates temporary branches and draft PRs for testing batches. However, the original PRs are the ones merged, not these temporary branches. This ensures the integrity and traceability of the original pull requests.
However, there might be scenarios where you want to merge the temporary branches instead. One advantage of this is maintaining the same SHA1, which might be important for some workflows and for traceability.
Also, if you are deploying after a merge, this can also make sure that you trigger only a single deployment once a batch of pull request is fully tested and passes the CI.
Mergify provides a setting called
queue_branch_merge_method for this purpose.
fast-forward, Mergify will merge
the temporary branches instead of the original PRs. The default setting is
none, which means the temporary branches are not merged.
Here is an example configuration:
queue_rules:- name: defaultbatch_size: 10queue_branch_merge_method: fast-forwardmerge_method: merge
Please note that this setting only works if
merge_method is set to
This is necessary so GitHub can mark the original PR as "merged" once the
temporary branch is merged. This configuration makes sure that the original
SHA1 from the PRs are included in the temporary branch and thus are detected
as merged. Importantly, the final merged result is the exact SHA1 of what has
been tested by CI, ensuring consistency and traceability.
If GitHub branch protections are enabled, fast-forward requires some additional configuration in branch protection settings, you also need to allow Mergify to "bypass the required pull requests" to merge.
This is mandatory since Mergify pushes the temporary branch to the base branch without going through a pull request in order to keep the same SHA1.
When a batch fails, Mergify does not remove all its pull requests from the queue. Instead, it takes additional steps to identify the problematic pull request and remove it from the queue.
This is how it works:
Splitting the batch: If a batch fails, all subsequent batches are deemed to fail as well, are canceled and put back into the queue. The system splits the failed batch to isolate the problematic pull request. The size of these new batches is determined by the
speculative_checksparameter. By default, the batch is split into two; however, if
speculative_checksis set to a value greater than 1, it dictates the number of batches the failed batch should be divided into.
Testing the new batches: After splitting, the first new batch is immediately retested, while others are queued. If
speculative_checksis greater than 1, the system will also test subsequent batch split at the same time. This continues until the split is done.
Handling the result: If the first batch split succeeds, it is merged and the next split is scheduled for testing. If the batch fails, Mergify splits this batch, going back to step 1.
Pin-pointing the failed batch: If a batch contains only one pull request and still fails, it is deemed to be the culprit and is removed from the queue.
Note that this system is completely automatic and there is no need to
intervene. The number of maximum splits can be controlled by
Let's assume that we have a batch of 6 pull requests:
[PR1 + PR2 + PR3 + PR4 + PR5 + PR6]. During the initial testing, Mergify first tests the batch with all
6 pull requests together.
As the batch fails, this could be due to any of the PRs or a specific
combination of them. Mergify learned that the combination
[PR1 + PR2 + PR3 + PR4 + PR5 + PR6] does not work; it now needs to test parts
of this combination.
speculative_checks set to 3, the system will aim to split the batch into
- First part:
[PR1 + PR2]
- Second part:
[PR1 + PR2 + PR3 + PR4]
- Third part:
[PR1 + PR2 + PR3 + PR4 + PR5]
Mergify already tested
[PR1 + PR2 + PR3 + PR4 + PR5 + PR6] in the
original batch and knows it does not work. No need to test it again!
The system will immediately retest the different parts since the number of splits matches the number of speculative checks allowed.
If the first part
[PR1 + PR2] passes, the system will merge it.
If the first part
[PR1 + PR2] fails, the system will split it again and test
each PR individually, applying the algorithm again.
If there's a consecutive failure in the subsequent parts, the system will continue to split and isolate the problematic PR(s) and retest until the split contains a single pull request.
While using batch merging and speculative checks together can significantly speed up your merge queue processing, it's crucial to consider the following points for an optimal setup:
Since pull requests are tested within temporary ones, the branch protection
Require branches to be up to date before merging needs to be
disabled. If your team requires a linear history, you can set the queue option
This does not mean that Mergify will test outdated PRs, but it will merge the original pull requests once its speculative checks are finished. The original PR won't be up-to-date according to GitHub, which means using this setting would block the merge.
Remember that changes to PRs or the queue can disrupt the batch process. If a
PR is updated or changed in a way that it no longer meets the
will be removed from the queue, and the order of checks will be updated. In
such cases, the process resets, and the remaining PRs are rechecked in their
If you rely on labels for some reason (e.g., triggering workflows) and would like to have those copied on the draft PRs that Mergify creates, you can use the mergify-merge-queue-labels-copier GitHub Action to copy the labels from the original pull requests to the draft PR.