Integrating Jenkins with Mergify
Run CI Insights via the Mergify Jenkins plugin, and Job Selection and Scopes through the Mergify CLI.
Jenkins is an open-source automation server. Mergify
ships a Jenkins plugin for CI Insights
and supports Job Selection and Scopes through the Mergify CLI. Any
Jenkins status check reported to GitHub can also be referenced from your Mergify
conditions via check-success.
Prerequisites
Section titled Prerequisites-
Jenkins is properly set up and is building your projects. See the Jenkins Documentation for help with setting up Jenkins.
-
Jenkins is configured to report build statuses to GitHub. This is typically done via plugins such as the GitHub plugin or the GitHub Branch Source Plugin.
-
The Mergify GitHub App is installed in your repository.
Mergify features for Jenkins
Section titled Mergify features for Jenkins-
CI Insights: install the Mergify Jenkins plugin to collect job metrics, detect flaky tests, and get Slack notifications for CI failures.
-
Job Selection: skip unaffected stages on pull requests to cut CI spend in monorepos.
-
Scopes: batch related pull requests in the merge queue. Scopes from a build tool are reported via the Mergify CLI.
Job selection
Section titled Job selectionJob Selection is scopes applied to your pipeline. The
same scopes block that lets the merge queue batch by scope also tells each
stage whether it has work to do, so a pull request that only touches the
frontend does not pay for the API and docs test suites.
This section wires it into a Jenkins pipeline; Job Selection explains how it works on any CI. For smarter batching or parallel mode in the merge queue, scopes already do that on their own. See Scopes.
Running GitHub Actions or Buildkite instead? The GitHub Actions and Buildkite integrations cover the same ground. Scopes themselves are CI-agnostic, so for any other provider let us know and see other build tools.
Declare your scopes
Section titled Declare your scopesmergify ci scopes reads the scopes block of
your .mergify.yml to know which areas of the repository map to each scope
name:
scopes: source: files: frontend: include: - apps/web/**/* api: include: - services/api/**/* docs: include: - docs/**/*Pipeline outline
Section titled Pipeline outlineA Jenkins pipeline driven by scopes has two parts:
-
Detect scopes in a first stage, with
mergify ci scopes. -
Guard each stage with a
whencondition on its scope, so Jenkins skips the stages whose scope the build does not touch.
Example pipeline
Section titled Example pipelineThe pipeline reads the scopes with the readJSON step of the
Pipeline Utility Steps
plugin, and binds credentials with the withCredentials step of the
Credentials Binding plugin:
install both on the controller.
pipeline { agent any stages { stage('Detect scopes') { steps { withCredentials([gitUsernamePassword(credentialsId: 'github')]) { sh 'mergify ci scopes --write scopes.json' } script { env.SCOPES = readJSON(file: 'scopes.json', returnPojo: true).scopes.join(',') } } } stage('Tests') { parallel { stage('Frontend tests') { when { expression { touches('frontend') } } steps { sh 'npm test' } } stage('API tests') { when { expression { touches('api') } } steps { sh 'pytest tests/api/' } } stage('Docs build') { when { expression { touches('docs') } } steps { sh 'make docs' } } stage('End-to-end tests') { when { expression { touches('merge-queue') } } steps { sh 'make e2e' } } } } }}
// Whether the build touches `scope`, from the list Detect scopes wrote.def touches(String scope) { return env.SCOPES.tokenize(',').contains(scope)}In this pipeline:
-
Detect scopesrunsmergify ci scopes, which matches the files the build changes against your.mergify.ymland writes the scopes they touch toscopes.json, such as{"scopes":["docs","frontend"]}.readJSONputs them inSCOPESfor the stages after it. KeepreturnPojo: true: it makesscopesa plain list, and the script sandbox rejectsjoinon the JSON arrayreadJSONreturns without it. -
Each test stage runs only when
touchesfinds its scope inSCOPES, and Jenkins marks the others as skipped. -
End-to-end testsis guarded by the merge queue scope, whichmergify ci scopeslists only on a build of a merge queue batch, so the suite runs on the batches the queue tests instead of on every push to a pull request. Its name comes fromscopes.merge_queue_scope, which defaults tomerge-queue: if you set it to another name, guard the stage on that one.
Install the latest release of the Mergify CLI on the agent.
mergify ci scopes needs no Mergify application key: it matches the files on
the agent, and Mergify matches each pull request’s files itself to batch by
scope. On a branch build that is not testing a merge queue batch, such as a
build of main, it compares the last commit only: when the build covers
several commits, the stages only the earlier ones need are skipped.
withCredentials lets the step fetch from origin what the clone lacks. On a
merge queue batch, that is the
Git note holding the batch’s
scopes, whichever strategy your Discover pull requests behaviors check pull
requests out with. Under The current pull request revision, it is also the
pull request’s target branch, which Jenkins does not fetch. github stands for
the ID of the credentials your GitHub source checks out with, and the Git
plugin’s Git Username and Password
binding
hands them to the git commands the step runs. A public repository needs no
credentials.
Scopes of a merge queue batch
Section titled Scopes of a merge queue batchOn a build of a merge queue batch, as its draft pull
request or as a branch, mergify ci scopes
compares no files. It returns the scopes the merge queue decided for the batch,
read from the Git note on the batch’s head commit, and adds the merge queue
scope. The stages that run are those the batch’s pull requests need, not those
of the batches queued ahead of it. When Jenkins builds the queue branch both as
the draft and as a branch, both builds run them.
Merge queue batch builds covers when it does.
Protecting the branch
Section titled Protecting the branchA stage skipped by its when condition leaves the build result alone, and a
failed Detect scopes stage fails the build, which skips every stage after it.
Require the status Jenkins reports to GitHub for the build: it passes only when
every stage that ran passed.
Reporting scopes from Jenkins
Section titled Reporting scopes from JenkinsA files source needs no Jenkins stage: Mergify matches each pull request’s
changed files itself. Report scopes from Jenkins when your source is manual,
alone or beside files.
In a multibranch pipeline, the Mergify CLI reads the pull request from the
variables the GitHub Branch Source plugin sets on a pull request build:
mergify ci git-refs takes its base from the
pull request’s target branch (CHANGE_TARGET), and
mergify ci scopes-send reads the pull
request number (CHANGE_ID). Install the latest release of the
Mergify CLI on the agent, and pass the pull request’s head commit
with --head-sha:
stage('Report scopes') { // Branch builds have no pull request to report scopes for. when { changeRequest() } environment { MERGIFY_TOKEN = credentials('MERGIFY_TOKEN') } steps { sh ''' REFS=$(mergify ci git-refs --format shell) || exit 1 eval "$REFS" # One scope per line, from the files the pull request changes. my_monorepo_command_to_get_impacted_projects \ "$MERGIFY_GIT_REFS_BASE" "$MERGIFY_GIT_REFS_HEAD" > scopes.txt mergify ci scopes-send --scopes-file scopes.txt --head-sha "$GIT_COMMIT" ''' }}MERGIFY_TOKEN is a Secret text credential holding an application
key granted CI
Reporting.
MERGIFY_GIT_REFS_BASE is the target branch’s commit, fetched from origin
when the clone lacks the branch, and
MERGIFY_GIT_REFS_HEAD is HEAD, the commit Jenkins checked out. Compare them
from their merge base, as git diff BASE...HEAD and nx affected --base --head
do, to cover what the pull request adds. A merge queue batch gets references of
its own, described in Merge queue batch builds.
GIT_COMMIT is the pull request’s head commit when the Discover pull requests
from origin and Discover pull requests from forks behaviors use the
strategy The current pull request revision, the default for a GitHub source
created with a current plugin release. Older projects, and a behavior added by
hand, default to Merging the pull request with the current target branch
revision: whenever the target branch has commits the pull request lacks,
GIT_COMMIT is the merge commit Jenkins built, and scopes reported for it reach
no pull request. Switch those behaviors to The current pull request
revision. As GIT_COMMIT can be that merge commit, the CLI does not read it on
its own: without --head-sha, it reports by pull request number, which is
deprecated and counts only for a manual source alone.
Under The current pull request revision, Jenkins fetches the pull request
alone by default, so git-refs fetches the target branch from origin, and the stage fails
when it cannot. On a private repository, give the sh step the credentials to
fetch with, such as the Git plugin’s Git Username and Password
binding.
Merge queue batch builds
Section titled Merge queue batch buildsOn a build of the merge queue’s draft pull
request, git-refs reads the batch from the
Git note Mergify attaches to the batch’s head commit. MERGIFY_GIT_REFS_BASE is
the commit the batch was built on, its
checking_base_sha, and
MERGIFY_GIT_REFS_HEAD is the batch’s head commit, under either checkout
strategy. The diff then holds the pull requests of the batch, not those of the
batches queued ahead of it. When a shallow clone lacks the base, git-refs
fetches it, and the stage fails when it cannot.
Jenkins can also build the queue branch as a plain branch, beside the draft or
instead of it, depending on how the project discovers branches. A Discover
branches behavior set to All branches or to Only branches that are also
filed as PRs builds it both ways, and the default, Exclude branches that are
also filed as PRs, can too when Jenkins handles the branch before the draft
opens. git-refs reads the same note on that build, with HEAD as the head.
The Report scopes stage skips it, since a branch build has no pull request to
report scopes for. A stage without the changeRequest() guard, such as one that
picks the tests a build runs, gets the batch’s references there too.
git-refs fetches the note from origin on both builds, whichever checkout
strategy the project uses, so a private repository needs the credentials
described above even when Jenkins merges the pull request with its target
branch. Without them, git-refs finds no note and, with no error, handles the
build as any other. Under that merge strategy, it compares a draft from its
target branch, which adds the batches queued ahead; under The current pull
request revision, it cannot fetch the target branch either, and the stage
fails. A queue branch built as a plain branch is compared from its last commit,
which drops the batch’s other pull requests. On a batch build that read the
note, MERGIFY_GIT_REFS_SOURCE is merge_queue: check it to catch the others.
Was this page helpful?
Thanks for your feedback!