A New Way to Do Continuous Delivery with Maven and Jenkins Pipeline

Note: for a more up to date take on this, please see my follow on post from March 2019.

Like any build tool, Maven offers multiple ways to do things. One thing that differentiates Maven from other build tools is that Maven is opinionated. What this means is that Maven will make the preferred path the easy path. 

Now the question you need to ask yourself is: Does this mean that the preferred path is the best path?

I'll put you out of your misery... the answer is NO. The preferred path is just the path that should suit most people's common requirements.

So, for example, when we look at how Maven suggests you should release things, we have the Maven Release plugin...

First things first, this is not so much of a plugin as a toolkit for building plugins that release and a sample plugin that matches the needs and requirements of the Apache Maven project for releasing. If you have different requirements the Apache Maven project expects that you would create your own release plugin that uses the Maven Release API to complete your needs.

Next, the plugin itself will build your project twice, running your tests twice... because most people prefer to only create tags that should build, and most people want to ensure that the tag actually built. So the first build will verify that the code to be released should build. The second build is from a clean checkout of the tag and verifies that the actual tag is what the release was built from (e.g. no need to worry about files in the workspace that were hidden by a .gitignore file)

Now when we look at how people try to apply continuous delivery we see lots of people not stepping back and seeing what assumptions they are not giving Maven the option of breaking.

So let's look at one of the first impedance mismatches that comes up: "Should we deploy - SNAPSHOT s?"

In continuous delivery, every commit should be potentially releasable, so the idea that most people try is to use the -SNAPSHOT versions and just deploy them...

There is an important aspect of continuous delivery that keeps on getting forgotten. With continuous delivery, every commit is potentially releasable. Obviously if the commit fails the tests we won't release it. But more importantly, even if the release does pass all the tests we may not deploy that commit. For example even in a fully automated continuous deployment pipeline a later commit may have got to run its tests on a faster build agent and won the race to get deployed. In pipelines that rely on a human intervention before actually proceeding with the final step of pushing to production you may have many commits that were potentially releasable and do not actually end up getting released.

So there is now a traceability problem that we need to solve. How do we know what code is in production right now? - SNAPSHOT s are not great for helping you with that problem. Never mind that there are issues with ensuring that the correct -SNAPSHOT is actually resolved from the Maven Repository.

Well the next option would be to make a release. Hmm, so every commit is going generate a release? Ok, so that solves the traceability issue, we have a unique version number and a tag... but oh dear me, the git history now has the [maven-release-plugin] prepare release ... and [maven-release-plugin] prepare for next development iteration commits for every ordinary commit... and we had to work to stop Jenkins from deciding to trigger a build after the [maven-release-plugin] prepare for next development iteration commit... and the developers are constantly hitting merge conflicts due to changes in the pom.xml files... and we are now building everything twice.

Well that's crappy. So then people start to fight that and either have a separate release step and have moved away from the "every commit should be potentially releasable" principle of continuous delivery... or they try to hack around the release plugin (because people seem to love hacking around Maven rather than stop for a second and actually do what Maven wants you to do... write a plugin)

Well there is a different way.

First off, if we are doing continuous delivery, we don't want to build any commit more than once. If the commit doesn't build then it will go no further. If it builds then we will run it as far through the testing pipeline as it can go until it gets to the end.

So what we really want to do is actually run a release build for every commit, but have the preparation for a no-op, and only push the tags if the release goes anywhere.

Here's a rough sketch of a new way to approach this problem:

1. Do a local checkout in a detached head
2. Use the Maven release plugin to prepare a release with pushChanges=false (we are not going to push the release commits back to master) and preparationGoals=initialize (we don't care if the tag is bad as we will only push tags that are good)
3. Run your stages release through your test pipeline
4. When you are ready to push to production (or if you prefer when ready to push to testing) you push the tag and release the staging repository

This will give you

• A clean master branch with none of the [maven-release-plugin] prepare ... commits.
• Good traceability from the release artifacts to the tagged source code
• No merge conflicts for the developers in their pom.xml

Now there are still issues that you will need to solve, for example you will possibly have inter-project dependencies that you will need to keep up-to-date... and if both are running continuous delivery your release process will have to find some way to pick those updates up...

But here is a starter pipeline script that can be used to show the way:

node {
  // Mark the code checkout 'stage'....
  stage 'Checkout'
  // Get some code from a GitHub repository

  git url: '...'
  // Clean any locally modified files and ensure we are actually on origin/master
  // as a failed release could leave the local workspace ahead of origin/master
  sh "git clean -f && git reset --hard origin/master"
  def mvnHome = tool 'maven-3.3.9'
  // we want to pick up the version from the pom
  def pom = readMavenPom file: 'pom.xml'
  def version = pom.version.replace("-SNAPSHOT", ".${currentBuild.number}")
  // Mark the code build 'stage'....
  stage 'Build'
  // Run the maven build this is a release that keeps the development version 
  // unchanged and uses Jenkins to provide the version number uniqueness
  sh "${mvnHome}/bin/mvn -DreleaseVersion=${version} -DdevelopmentVersion=${pom.version} -DpushChanges=false -DlocalCheckout=true -DpreparationGoals=initialize release:prepare release:perform -B"
  // Now we have a step to decide if we should publish to production 
  // (we just use a simple publish step here)
  input 'Publish?'
  stage 'Publish'
  // push the tags (alternatively we could have pushed them to a separate
  // git repo that we then pull from and repush... the latter can be 
  // helpful in the case where you run the publish on a different node
  sh "git push ${pom.artifactId}-${version}"
  // we should also release the staging repo, if we had stashed the 
  //details of the staging repository identifier it would be easy

}

---

Learn More

Organizations have been orchestrating pipelines with Jenkins for years. As Jenkins and continuous delivery experience deepens, organizations want to move beyond simple pipelines and chart complex flows to maps to their specific software delivery processes. For Jenkins users, creating and managing complex pipelines just became easier with Jenkins and the Pipeline plugin. Learn what it is, what it does for you and how to set up your delivery pipelines using it.

Download your copy and you’ll learn how you can deliver better software, faster.