Scaling Jenkins Horizontally with Jenkins Operations Center by CloudBees

Written by: Harpreet Singh

4 min read

The recent Jenkins survey [1] showed that 82% of users consider Jenkins critical for their organization.

Approximately 91.5% of Jenkins support 2+ projects and 42% support 10+ projects. Furthermore, 30% of respondents worked for companies with 100+ developers.

So what do these numbers mean for Jenkins administrators toiling away to keep the agile factory running?

Admins are constantly adding more and more teams to the software factory. Thus, admins are in the business of making their instances resilient to failures and scaling them to onboard more teams.

Jenkins is scaled by adding build nodes to the Jenkins instance while beefing the machine that runs the Jenkins controller. Said differently, admins scale their Jenkins controller vertically. Although, Jenkins scales well vertically, there is a limit on what can be done on one box [2].

And as we know, horizontal scaling trumps vertical .

Jenkins Operations Center by CloudBees makes it possible to scale Jenkins horizontally. The product (affectionately known at CloudBees as "Jockey") is the operations dashboard for Jenkins in the organization.

The Jenkins Operations Center server (which is just a Jenkins controller with some plugins on it) effectively becomes the central "cloud" provider for build nodes for all downstream controllers attached to it.

Downstream controllers rent a build node when they want to run a build. Thus with Jenkins Operations Center, admins can add a new controller when a team comes onboard or just create a new controller if performance dips on a particular Jenkins controller.

About 32% of organizations have more than one controller floating around (usually one controller per group) and about 65% of these have more than one build node. Since the Jenkins Operations Center server consolidates build nodes, it drastically cuts down CAPEX costs entailed as admins can move away from the model where each group buys more build nodes to bolster their controller.

The Jenkins Operations Center server becomes the central launching pad to jump into and browse jobs on downstream controllers. There is SSO setup, facilitating easy access to downstream jobs.

One other point, the Jenkins Operations Center by CloudBees server can optionally push down security realm and authorization roles to downstream controllers. Thus, it effectively becomes the single point of security configuration, as opposed to setting up configurations on each controller separately. To take it further, you can also configure plugin centers per controller and manage them from the Jenkins Operations Center server.

Watch some videos [3] to see a quick overview of the product and how to set it up.

I believe Jenkins Operations Center by CloudBees is going to fundamentally change how admins architect scalability and resiliency for their Jenkins instances. No more single point of failure due to an overloaded Jenkins controller!

There is more (Jenkins Enterprise by CloudBees)...

We have also released a new version of Jenkins Enterprise by CloudBees (November 2013 release). The release has two main customer RFEs: a) capability to create templates per folder and general improvements in the Templates plugin, b) CLI support for the Role-based Access Control plugin and the ability to quickly setup security with pre-defined roles. If you want to know more, there is additional information in the user guide and release notes. [3]

Next steps:

I suggest reading through the excellent tutorial [5] on how to set up Jenkins Operations Center by CloudBees. Note: this sets up a highly available Jenkins Operations Center server and highly available downstream controllers. Download the free 30-day evaluation copies for Jenkins Operations Center and Jenkins Enterprise [6].

Key Takeaways:

  1. Scale Jenkins controllers horizontally making CI infrastructure resilient to failures

  2. Share build nodes between controllers. Reduce CAPEX by sharing instead of buying build nodes

  3. Share security configuration between controllers in the organization

References:

[1] Jenkins 2013 survey results (link ). All graphs in this blog are from the survey
[2] View Kohsuke'spresentation for JUC2013 where he talks about the X1Kgoal of making it feasible to run 1kexecutors on one controller
[3] Jenkins Operations Center videos: two-minute introduction , comprehensive overview , attach a client controller , attach a shared agent
[4] Jenkins Enterprise by CloudBees release notes and user guide
[5] Jenkins Operations Center by CloudBees tutorial
[6] Free 30-day evaluation links for Jenkins Enterprise by CloudBees and Jenkins Operations Center by CloudBees

- Harpreet Singh
Senior Director, Product Management
CloudBees
cloudbees.com

Harpreet CloudBees

Harpreet has 15 years of experience in the software industry. Prior to CloudBees, he was at Oracle and Sun for 10 years in various roles, including leading the marketing efforts for Java EE 6 and GlassFish 3.1. He was also product manager for Hudson, launching it within Sun's GlassFish portfolio.

Stay up-to-date with the latest insights

Sign up today for the CloudBees newsletter and get our latest and greatest how-to’s and developer insights, product updates and company news!