Speeding Release Cycles and Overcoming Regulatory Hurdles with CloudBees CI

Contributed by Geir Engebakken, Chief Consultant / DevOps Engineer, Tietoevry

When I started coding in 1990, there were no source or access control tools. Developers had individual copies of source code on their computers, which they manually shared and merged with the work of their peers. Automation wasn’t a part of the conversation, either.

At the time, I was part of a company called Fellesdata. I helped cobble together a GUI-based source control system—built atop a file-based version control system—that sorted sources into projects and modules. This project was the start of automating daily builds and was a crude predecessor of today’s CI systems, but it wasn’t robust, and we eventually abandoned the project.

Over 30 years of mergers and acquisitions later, we evolved into Tietoevry, a leading IT services and software provider in the Nordic countries with nearly 24,000 employees.

I am currently a Chief Consultant and DevOps Engineer in Tietoevry Banking. Tietoevry Banking is the market leading financial services software and solution partner in the Nordics. We deliver a modular, pre-integrated banking as a service solution, as well as market leading software for core, payments, credit, cards, financial fraud and wealth. With unmatched scale, expertise and specialization, we drive the industry forward by enabling financial institutions to become more agile and efficient, accelerating their digital transformation journey.

My DevOps team comprises eight engineers who provide continuous integration (CI) and source control tools and services for our pool of developers. Their job is to produce a comprehensive portfolio of products, including internal database applications, online services, and customer self-service solutions.

One of our biggest hurdles is regulatory compliance. A small change in the rules that govern banking and financial transactions can wreak havoc on our development cycles as we retool our products to comply with new standards.

About four years ago—before the Tieto merger—EVRY was at an impasse. We were developing 100 different financial applications, and our continuous integration (CI) tools couldn’t keep up with our release schedule and constant changes to regulatory frameworks. We invested in Jenkins as our DevOps platform, but our access and source control, versioning, project organization, and support tools were in disarray.

We had one open source Jenkins controller and five agents with two or three threads each. As a result, we could only build 15 projects in parallel, which severely impacted our productivity. We also had plugin and dependency problems. When we first adopted Jenkins, we gave every developer unfettered access, so everyone started using the plugins of their choice. This open access led to version and dependency issues. There was no standard set of plugins, and we lacked a plugin update strategy, which created chaos as we approached our delivery dates.

To standardize our Jenkins environment, create a uniform set of DevOps and CI practices, and ensure timely technical support, we chose CloudBees CI, an enterprise version of open source Jenkins. We started by deploying the Role-Based Access Control (RBAC) plugin to standardize access to the Jenkins environment. We defined developer, administrator, and other security roles, which we assigned to groups of users. We also configured Jenkins access to use the same authorization credentials as other CI and DevOps tools.

Next, we used the Folders plugin to organize development projects. We created folder hierarchies that included top-level folders for project categories such as credit cards and bank access channels. We then set up subfolders and permissions to control specific jobs within each project. Finally, we implemented the Pipeline and Pipeline Multibranch plugins to manage complex jobs.

Using these CloudBees plugins, each developer creates an individual source code branch that triggers an automatic build on our Jenkins instance. This approach has vastly expanded our capacity to work on multiple projects. We have grown our developer pool from 100 to 600 coders and now have four Jenkins controllers that can build 300 projects in parallel.

Automating building and testing 300 simultaneous projects generates a lot of value for the Tietoevry Banking development team. We have eliminated errors triggered by manual builds and have supercharged our ability to detect and correct coding errors. CloudBees provides almost immediate feedback to my developers, reducing our time to fix mistakes and recompile our work. Jenkins also starts up faster, so we can get to work right away.

These combined efficiencies allow our developers to work on more than 2,000 different projects a year.

CloudBees also provides 24/7 technical support to my team. Although we know how Jenkins works, we don’t always have the time and resources to troubleshoot the system itself. CloudBees will step in and help us in such circumstances. Their people are well-versed in CI pipelines and DevOps environments, and even help us manage pipeline administration and conflicts with CI tools from other vendors.

When we open a support ticket, they go to work immediately to offer the fastest-possible resolution. CloudBees is so good at supporting us that my team is modeling our internal support efforts on their approach. I like to think of my eight-person DevOps support squad as a mini-CloudBees.

We’re delighted with the quality of the platform and the level of support, and we recently renewed our CloudBees contract for another three years. The Tietoevry merger is only two years old, and we have been busy with tasks connected to the merger, such as consolidating our processes and migrating Tietoevry users and applications—including Jenkins authentication—to Microsoft Azure cloud-based infrastructure.

But we can’t neglect the work we still have to do in our CI environment. If we want to grow our environment and have more than the four Jenkins controllers that we have currently, we have to figure out a way to manage those controllers. Manually managing these controllers hampers our capacity, stunts our growth, and takes valuable time.

One of the things that we could do is implement the CloudBees Kubernetes plugin to integrate a Kubernetes cluster and run Docker containers in our CI environment. Doing so would automate the management of our Jenkins controllers using the CloudBees Operation Center. By automating the deployment and maintenance of our Jenkins controllers, we can further increase the number of parallel builds.

CloudBees is helping Tietoevry rein in our Jenkins environment. We have standardized processes, implemented access and source control mechanisms, automated workflows, and increased production capacity. My team spends less time managing our Jenkins environment and our developers output more code faster. The result is shorter development cycles, faster updates, more new products, and quicker reactions to changes in regulatory frameworks and trends in finance and banking.

Thanks to CloudBees, Tietoevry Banking continues to create innovative and secure products for over 150 financial institutions and banks that serve individual and institutional customers in 12 international markets. We jumped on the CloudBees wagon and haven’t looked back.