View as Markdown

Integrating Jenkins with Mergify

Run CI Insights via the Mergify Jenkins plugin, and Job Selection and Scopes through the Mergify CLI.


Jenkins is an open-source automation server. Mergify ships a Jenkins plugin for CI Insights and supports Job Selection and Scopes through the Mergify CLI. Any Jenkins status check reported to GitHub can also be referenced from your Mergify conditions via check-success.

  1. Jenkins is properly set up and is building your projects. See the Jenkins Documentation for help with setting up Jenkins.

  2. Jenkins is configured to report build statuses to GitHub. This is typically done via plugins such as the GitHub plugin or the GitHub Branch Source Plugin.

  3. The Mergify GitHub App is installed in your repository.

  • CI Insights: install the Mergify Jenkins plugin to collect job metrics, detect flaky tests, and get Slack notifications for CI failures.

  • Job Selection: skip unaffected stages on pull requests to cut CI spend in monorepos.

  • Scopes: batch related pull requests in the merge queue. Scopes from a build tool are reported via the Mergify CLI.

Job Selection is scopes applied to your pipeline. The same scopes block that lets the merge queue batch by scope also tells each stage whether it has work to do, so a pull request that only touches the frontend does not pay for the API and docs test suites.

This section wires it into a Jenkins pipeline; Job Selection explains how it works on any CI. For smarter batching or parallel mode in the merge queue, scopes already do that on their own. See Scopes.

Running GitHub Actions or Buildkite instead? The GitHub Actions and Buildkite integrations cover the same ground. Scopes themselves are CI-agnostic, so for any other provider let us know and see other build tools.

mergify ci scopes reads the scopes block of your .mergify.yml to know which areas of the repository map to each scope name:

.mergify.ymlyaml
scopes:
source:
files:
frontend:
include:
- apps/web/**/*
api:
include:
- services/api/**/*
docs:
include:
- docs/**/*

A Jenkins pipeline driven by scopes has two parts:

  1. Detect scopes in a first stage, with mergify ci scopes.

  2. Guard each stage with a when condition on its scope, so Jenkins skips the stages whose scope the build does not touch.

PR Pull request changes Detect Detect scopes stage (mergify ci scopes) PR->Detect Config Scopes config (.mergify.yml) Config->Detect Frontend Frontend tests (run) Detect->Frontend scope: frontend Docs Docs build (run) Detect->Docs scope: docs API API tests (skipped) Detect->API scope: api (false) E2E End-to-end tests (skipped) Detect->E2E scope: merge-queue (false) cluster_tests Tests PR Pull request changes Detect Detect scopes stage (mergify ci scopes) PR->Detect Config Scopes config (.mergify.yml) Config->Detect Frontend Frontend tests frontend: run Detect->Frontend Docs Docs build docs: run API API tests api: skipped E2E End-to-end tests merge-queue: skipped
On a pull request build, each test stage runs only when the pull request touches its scope: orange runs, grey is skipped. End-to-end tests runs only when merge-queue is listed, on a merge queue batch.

The pipeline reads the scopes with the readJSON step of the Pipeline Utility Steps plugin, and binds credentials with the withCredentials step of the Credentials Binding plugin: install both on the controller.

groovy
pipeline {
agent any
stages {
stage('Detect scopes') {
steps {
withCredentials([gitUsernamePassword(credentialsId: 'github')]) {
sh 'mergify ci scopes --write scopes.json'
}
script {
env.SCOPES = readJSON(file: 'scopes.json', returnPojo: true).scopes.join(',')
}
}
}
stage('Tests') {
parallel {
stage('Frontend tests') {
when { expression { touches('frontend') } }
steps { sh 'npm test' }
}
stage('API tests') {
when { expression { touches('api') } }
steps { sh 'pytest tests/api/' }
}
stage('Docs build') {
when { expression { touches('docs') } }
steps { sh 'make docs' }
}
stage('End-to-end tests') {
when { expression { touches('merge-queue') } }
steps { sh 'make e2e' }
}
}
}
}
}
// Whether the build touches `scope`, from the list Detect scopes wrote.
def touches(String scope) {
return env.SCOPES.tokenize(',').contains(scope)
}

In this pipeline:

  • Detect scopes runs mergify ci scopes, which matches the files the build changes against your .mergify.yml and writes the scopes they touch to scopes.json, such as {"scopes":["docs","frontend"]}. readJSON puts them in SCOPES for the stages after it. Keep returnPojo: true: it makes scopes a plain list, and the script sandbox rejects join on the JSON array readJSON returns without it.

  • Each test stage runs only when touches finds its scope in SCOPES, and Jenkins marks the others as skipped.

  • End-to-end tests is guarded by the merge queue scope, which mergify ci scopes lists only on a build of a merge queue batch, so the suite runs on the batches the queue tests instead of on every push to a pull request. Its name comes from scopes.merge_queue_scope, which defaults to merge-queue: if you set it to another name, guard the stage on that one.

Install the latest release of the Mergify CLI on the agent. mergify ci scopes needs no Mergify application key: it matches the files on the agent, and Mergify matches each pull request’s files itself to batch by scope. On a branch build that is not testing a merge queue batch, such as a build of main, it compares the last commit only: when the build covers several commits, the stages only the earlier ones need are skipped.

withCredentials lets the step fetch from origin what the clone lacks. On a merge queue batch, that is the Git note holding the batch’s scopes, whichever strategy your Discover pull requests behaviors check pull requests out with. Under The current pull request revision, it is also the pull request’s target branch, which Jenkins does not fetch. github stands for the ID of the credentials your GitHub source checks out with, and the Git plugin’s Git Username and Password binding hands them to the git commands the step runs. A public repository needs no credentials.

On a build of a merge queue batch, as its draft pull request or as a branch, mergify ci scopes compares no files. It returns the scopes the merge queue decided for the batch, read from the Git note on the batch’s head commit, and adds the merge queue scope. The stages that run are those the batch’s pull requests need, not those of the batches queued ahead of it. When Jenkins builds the queue branch both as the draft and as a branch, both builds run them. Merge queue batch builds covers when it does.

A stage skipped by its when condition leaves the build result alone, and a failed Detect scopes stage fails the build, which skips every stage after it. Require the status Jenkins reports to GitHub for the build: it passes only when every stage that ran passed.

A files source needs no Jenkins stage: Mergify matches each pull request’s changed files itself. Report scopes from Jenkins when your source is manual, alone or beside files.

In a multibranch pipeline, the Mergify CLI reads the pull request from the variables the GitHub Branch Source plugin sets on a pull request build: mergify ci git-refs takes its base from the pull request’s target branch (CHANGE_TARGET), and mergify ci scopes-send reads the pull request number (CHANGE_ID). Install the latest release of the Mergify CLI on the agent, and pass the pull request’s head commit with --head-sha:

groovy
stage('Report scopes') {
// Branch builds have no pull request to report scopes for.
when { changeRequest() }
environment {
MERGIFY_TOKEN = credentials('MERGIFY_TOKEN')
}
steps {
sh '''
REFS=$(mergify ci git-refs --format shell) || exit 1
eval "$REFS"
# One scope per line, from the files the pull request changes.
my_monorepo_command_to_get_impacted_projects \
"$MERGIFY_GIT_REFS_BASE" "$MERGIFY_GIT_REFS_HEAD" > scopes.txt
mergify ci scopes-send --scopes-file scopes.txt --head-sha "$GIT_COMMIT"
'''
}
}

MERGIFY_TOKEN is a Secret text credential holding an application key granted CI Reporting. MERGIFY_GIT_REFS_BASE is the target branch’s commit, fetched from origin when the clone lacks the branch, and MERGIFY_GIT_REFS_HEAD is HEAD, the commit Jenkins checked out. Compare them from their merge base, as git diff BASE...HEAD and nx affected --base --head do, to cover what the pull request adds. A merge queue batch gets references of its own, described in Merge queue batch builds.

GIT_COMMIT is the pull request’s head commit when the Discover pull requests from origin and Discover pull requests from forks behaviors use the strategy The current pull request revision, the default for a GitHub source created with a current plugin release. Older projects, and a behavior added by hand, default to Merging the pull request with the current target branch revision: whenever the target branch has commits the pull request lacks, GIT_COMMIT is the merge commit Jenkins built, and scopes reported for it reach no pull request. Switch those behaviors to The current pull request revision. As GIT_COMMIT can be that merge commit, the CLI does not read it on its own: without --head-sha, it reports by pull request number, which is deprecated and counts only for a manual source alone.

Under The current pull request revision, Jenkins fetches the pull request alone by default, so git-refs fetches the target branch from origin, and the stage fails when it cannot. On a private repository, give the sh step the credentials to fetch with, such as the Git plugin’s Git Username and Password binding.

On a build of the merge queue’s draft pull request, git-refs reads the batch from the Git note Mergify attaches to the batch’s head commit. MERGIFY_GIT_REFS_BASE is the commit the batch was built on, its checking_base_sha, and MERGIFY_GIT_REFS_HEAD is the batch’s head commit, under either checkout strategy. The diff then holds the pull requests of the batch, not those of the batches queued ahead of it. When a shallow clone lacks the base, git-refs fetches it, and the stage fails when it cannot.

M M B B M->B  #1   H H B->H  #2   _branch0 main _branch0->M _branch1 queue branch _branch1->H M M B B M->B  #1   H H B->H  #2   _branch0 main _branch0->M _branch1 queue branch _branch1->H
Batch 2 is tested on top of batch 1, which is still being tested ahead of it. On batch 2's build, the base is B, the commit holding main and PR #1, and the head is H, so the diff holds PR #2 alone. Orange is being tested.

Jenkins can also build the queue branch as a plain branch, beside the draft or instead of it, depending on how the project discovers branches. A Discover branches behavior set to All branches or to Only branches that are also filed as PRs builds it both ways, and the default, Exclude branches that are also filed as PRs, can too when Jenkins handles the branch before the draft opens. git-refs reads the same note on that build, with HEAD as the head. The Report scopes stage skips it, since a branch build has no pull request to report scopes for. A stage without the changeRequest() guard, such as one that picks the tests a build runs, gets the batch’s references there too.

git-refs fetches the note from origin on both builds, whichever checkout strategy the project uses, so a private repository needs the credentials described above even when Jenkins merges the pull request with its target branch. Without them, git-refs finds no note and, with no error, handles the build as any other. Under that merge strategy, it compares a draft from its target branch, which adds the batches queued ahead; under The current pull request revision, it cannot fetch the target branch either, and the stage fails. A queue branch built as a plain branch is compared from its last commit, which drops the batch’s other pull requests. On a batch build that read the note, MERGIFY_GIT_REFS_SOURCE is merge_queue: check it to catch the others.

Was this page helpful?