Using Multi-branch Pipelines in the Apache Maven Project

This is a post about how using Jenkins and Pipeline has enabled the Apache Maven project to work faster and better.

Most Java developers should have at least some awareness of the Apache Maven project . Maven is used to build a lot of Java projects. In fact the Jenkins project and most Jenkins plugins are currently built using Maven.

After the release of Maven 3.3.9 in 2015, at least from the outside, the project might have appeared to be stalled. In reality, the project was trying to resolve a key issue with one of its core components: Eclipse Aether . The Eclipse Foundation had decided that the Aether project was no longer active and had started termination procedures .

Behind the scenes the Maven Project Management Committee was negotiating with the Eclipse Foundation and getting all the IP clearance from committers required in order to move the project to Maven. Finally in the second half of 2016, the code landed as Maven Resolver.

But code does not stay still.

There had been other changes made to Maven since 3.3.9 and the integration tests had not been updated in accordance with the project conventions.

The original goal had been to get a release of Maven itself with Resolver and no other major changes in order to provide a baseline. This goal was no longer possible.

In January 2017, the tough decision was taken .

Reset everything back to 3.3.9 and merge in each feature cleanly, one at a time, ideally with a full clean test run on the main supported platforms: Linux and Windows, Java 7 and 8.

In a corporate environment, you could probably spend money to work your way out of trying to reconstruct a subset of 14 months of development history. The Apache Foundation is built on volunteers. The Maven project committers are all volunteers working on the project in their spare time.

What was needed was a way to let those volunteers work in parallel preparing the various feature branches while ensuring that they get feedback from the CI server so that there is very good confidence of a clean test run before  the feature branch is merged to master.

Enter Jenkins Pipeline Multibranch and the Jenkinsfile .

A Jenkinsfile  was set up that does the following:

1. Determines the current revision of the integration tests for the corresponding branch of the integration tests repository (falling back to the master  branch if there is no corresponding branch)
2. Checks out Maven itself and builds it with the baseline Java version (Java 7) and records the unit test results
3. In parallel on Windows and Linux build agents, with both Java 7 and Java 8. Checks out the single revision of the integration tests identified in step 1 and runs those tests against the Maven distribution built in step 2, recording all the results at the end.

There's more enhancements planned for the Jenkinsfile  (such as moving to the declarative syntax) but with just this we were able to get all the agreed scope merged and cut two release candidates.

The workflow is something like this:

1. Developer starts working on a change in a local branch
2. The developer recognizes that some new integration tests are required, so creates a branch with the same name in the integration tests repository.
3. When the developer is ready to get a full test run, they push the integration tests branch (integration tests have to be pushed first at present) and then push the core branch.
4. The Apache GitPubSub event notification system sends notification of the commit to all active subscribers.
5. The Apache Jenkins server  is an active subscriber to GitPubSub and routes the push details into the SCM API  plugin's event system.
6. The Pipeline Multibranch plugin creates a branch project for the new branch and triggers a build
7. Typically the build is started within 5 seconds of the developer pushing the commit.
8. As the integration tests run in parallel, the developer can get the build result as soon as possible.
9. Once the branch is built successfully and merged, the developer deletes the branch.
10. GitPubSub  sends the branch deletion event and Jenkins marks the branch job as disabled (we keep the last 3 deleted branches in case anyone has concerns about the build result)

The general consensus among committers is that the multi-branch project is a major improvement on what we had before. 

Notes

• While GitPubSub itself is probably limited in scope to being used at the Apache Software Foundation, the subscriber code that routes events from source control into the SCM API plugin's event system is relatively small and straightforward and would be easy to adapt if you have a custom Git hosting service, i.e. if you were in the 4% on this totally unscientific poll I ran on twitter:

  If you use Git at work, please answer this poll. The git server we use is:

  
  - Stephen Connolly (@connolly_s) March 17, 2017

• There is currently an issue whereby changes to the integration test repository do not trigger a build. This has not proved to be a critical issue so far as typically developers change both repositories if they are changing the integration tests.