Organizations are constantly striving to release software faster, to get their product into users' hands sooner and get feedback for improvement or correct problems. Software is never going to be perfect in its first iteration and the end users might actually want something different than what is produced. There is great value to the business if new features can be delivered and bugs fixed in a timely fashion; or a full-course change is required. Working on a minimal viable product (MVP) and using Agile practices, development teams can, in theory, produce a new working product at the end of every sprint. However, there is a big difference in continuously developing a product and continuously delivering that product to users.
Software is a world of interdependencies and all of those interdependencies have to be validated at various stages before a product is released. Are the external library files consistent? Is the database version the same? Are all the required packages installed on the target host OS? There are countless things that can go wrong when moving from development to testing to production.
Tools like Jenkins, Chef, and Puppet have helped to automate the flow of software through various stages and ensure a consistent environment. By continuously integrating all software dependencies and standardizing the configuration management of the environments, teams have reduced the number of variables in a delivery pipeline and eliminated potential problems allowing for more automation and, thus, expediting the delivery of the software.
The emergence of Docker and containers has further reduced the variables present in a delivery pipeline. With Docker, a single image can move from development to testing and finally to production without changing the application or the underlying configuration. As long as the Docker host is consistent then all containers with that image should work across all environment stages.
What is Docker?
Docker is an open-source project that provides a platform for building and shipping applications using containers. This platform enables developers to easily create standardized environments that ensure that a testing environment is the same as the production environment, as well as providing a lightweight solution for virtualizing applications.
Docker containers are lightweight runtime environments that consist of an application and its dependencies. These containers run “on the metal” of a machine, allowing them to avoid the 1-5% of CPU overhead and 5-10% of memory overhead associated with traditional virtualization technologies. They can also be created from a read-only template called a Docker image.
Docker images can be created from an environment definition called a Dockerfile or from a running Docker container which has been committed as an image. Once a Docker image exists, it can be pushed to a registry like Docker Hub and a container can be created from that image, creating a runtime environment with a guaranteed set of tools and applications installed to it. Similarly, containers can be committed to images which are then committed to Docker Hub.
The Interdependency Problem
The immutability of the Docker container goes a long way towards facilitating continuous delivery but it does not completely solve the problem of interdependencies. Docker containers are built upon images, both parent and base images. An application can run on an Apache parent image with a base image of CentOS. These images, and the containers they are used in, are all uniquely identified and versioned to account for change over time much like binary artifacts or gems.
In addition to image dependencies an application is not always contained in a single container; Dockerized applications are increasingly deployed as microservices . As Martin describes, breaking up monolithic applications into discrete functional units that interoperate is a great way to help teams continuously deliver parts of an application without requiring a release cycle of the entire application and every team involved. Not only are there image dependencies but we now have microservice dependencies. The level of abstraction has moved up a rung.
Traceability with Fingerprinting and Docker
Despite, or because of, all of the automation inherent in a continuous delivery pipeline things still break. When they do it is necessary to quickly identify and correct the problem across all of the dependencies that go into a running application. Visibility and traceability across all dependencies in an application are paramount to continuously delivering and running that application. To that end, Jenkins allows teams to track artifacts with a "Fingerprint" , letting users see what went into a build and where that build is being used. Combined with the Deployment Notification Plugin this fingerprint can be used to track when and where a package has been deployed by Chef or Puppet. This traceability is very useful for both developers and operations. If a bug is found in development it can be quickly traced to everywhere it has been deployed. Conversely, if a problem occurs in production the operations team can easily find the deployed build in Jenkins and see all components included.
The addition of the CloudBees Docker Traceability plugin enables Jenkins to now extend this same traceability to Docker images, showing the build and deployment history of each container and the related images. This plugin requires the Docker Commons plugin which provides the fingerprints for all Docker images and it is available to everyone in the Jenkins community.
The CloudBees Docker Traceability plugin provides both an overall view from the Jenkins sidebar of all containers currently registered and deployed and a detailed view of the container build from the build page. The Docker image ids are provided for all parent images and the base image used. In addition the Docker image ID is searchable in Jenkins to quickly find where and when it is deployed and how and when it was built.
Using this information, it is possible to determine if something changed in the code for a container or if one of the parent images or base image of a container changed from one build to another helping to determine the root cause of any problems in the overall application.
Where do I start?
The CloudBees Docker Traceability Plugin is an open-source plugin, so it is available for download from the open-source update center or packaged as part of the CloudBees Jenkins Platform.
Other plugins complement and enhance the pipelines possible with this plugin. Read more about their uses cases in these blogs:
Docker Agents with the CloudBees Jenkins Platform
More information can be found in the newly released Jenkins Cookbook
Patrick Wolf
Product Manager
CloudBees
Patrick Wolf is a product manager for CloudBees and is based in San Jose.