Tuning the Engine - CI/CD Platform Monitoring with CloudBees DevOptics Run Insights

Written by: Ben Williams

4 min read

Today we’re excited to announce the availability of an entire new set of capabilities in CloudBees DevOptics for continuous integration (CI) and continuous delivery (CD) platform monitoring.

Designed to help Shared Services teams and Jenkins administrators ensure that their CI and CD platforms are optimally supporting the demands of their product teams, the all new Run Insights capabilities in CloudBees DevOptics provide both real-time and historic views of job and pipeline activity across your entire software delivery infrastructure.

CloudBees DevOptics works with a heterogeneous mix of CloudBees Core and Jenkins, and allows you to connect any number of controllers to provide aggregate views across your entire organization.

As a trusted partner in the software delivery space, we engage with many organizations from the largest multinationals to startups in their infancy, and we identified a number of common challenges in optimizing CI and CD platforms that we’re now solving. Let’s take a look at a few of those challenges now.

Past, present and future - what’s happening right now, what has been and what should we expect to be?

Previously, it’s been difficult to understand CI/CD platform activity in real-time. Getting answers to questions such as those listed below has been challenging:

  • How many active jobs or pipelines are running right now?

  • How many build executors are idle and available to serve requests?

  • How many jobs or pipelines are in the queue waiting for executor resources?

  • When will the jobs or pipelines on the queue be able to start?

  • How long will it be before all currently active jobs or pipelines are completed?

The challenges are only magnified when you want the answers to aggregate across multiple different installations across groups or companies.

And for all of the above questions, it’s also important to understand what the expected answers are typically at the current (or any given) day and time so that anomalies can be easily detected.

Similarly, it hasn’t been easy to understand what has been happening with the platform historically. Why were queue lengths higher than normal yesterday at 16:00 UTC? Or why do jobs and pipelines have longer execution times on Fridays?

CloudBees DevOptics Run Insights historical charts . CloudBees DevOptics Run Insights historical charts

Balancing supply and demand for build resources

It’s critical for Shared Services teams and admins to know the answers to these questions to help them tune the platform so that it is neither wastefully using resources, nor needlessly constraining product teams and introducing waste into their Value Streams by forcing them to wait for resources. Having online and idle executors means resources are ready to serve teams. But having too many idle executors will create unnecessary infrastructure costs. Getting the balance right is important.

Teams have different demand at different times

Geographical location, working patterns and development and release cycles all contribute to the patterns of demand that teams create on the CI/CD platform. Knowing these patterns is an essential ingredient in platform optimization.

Breaking it down - not all resources are created equally

But supply and demand over time has yet another dimension. Different job or pipeline workloads need different types of resources to serve them. It’s important to match the availability of the right type of resources to the demand for that type as and when it happens.

Minimizing value stream waste

When the queue length is large, developers have to wait longer for feedback on the jobs and pipelines that they have triggered, which not only adds waiting waste into value streams and increases Mean Lead Times (MLT) delaying time to value, but also has a magnified effect of increasing the cost of any failures. Getting this right is vital for any business seeking to maximize their DevOps ROI. For more information on how CloudBees DevOptics provides real-time visualization of your software value streams, read this blog .

Finding the right maintenance windows

Have you ever needed to perform some urgent system or application level maintenance and wanted to know when currently running pipelines will be finished? Or wanted to schedule time for application or OS level upgrades and wanted to plan the best time to do that with least disruption and impact to the product teams? It’s important to have the data you need to make these decisions effectively.

Schedule a demo today

The Run Insights capabilities in CloudBees DevOptics help you with all of these challenges. To get more detail on how all of the exciting new features can help you tune your CI/CD engine, as well as to find out more about the end to end Value Stream Management and DevOps Performance Metrics solutions in CloudBees DevOptics, schedule a demo today .

Stay up-to-date with the latest insights

Sign up today for the CloudBees newsletter and get our latest and greatest how-to’s and developer insights, product updates and company news!