Intro to Analytics Reports

The following blog briefly describes the out-of-the-box analytics reports available within the CloudBees platform. These reports address three main pain points for our target end users: platform engineering teams, SREs, and developers. 

1. Engineering efficiency
   Developers, engineering managers, and CTOs always try to identify engineering bottlenecks and delays and reduce workflow inefficiencies. These users rely on data to identify issues and drive towards optimization and streamlining the engineering process. Our current reports for software delivery activity, flow metrics, and DORA metrics reports will help the users.

2. Risk mitigation
   Developers, engineering managers, and many CISOs of the organization are interested in early detection of potential vulnerabilities or bottlenecks so that they can proactively work on risk mitigation. Additionally, CISOs would like to establish global organizational SLAs and track compliance with these SLAs. Our current report for security insights will help the users in this regard.

3. Quality assurance
   Developers, engineering managers, QAs, and CTOs want to track the quality of work by measuring defects and issues. This helps them to improve product quality, thereby increasing customer satisfaction. Our current reports for flow metrics and DORA metrics will help the users.

As of February 2024, we offer four analytics reports. Before diving into each, let’s quickly discuss common elements shared across each report:

• Filtering: Use the filters to choose the component and the duration for which I want to see flow metrics down to the component level. When filtering, please note that weeks run from Monday to Sunday. 

• Drill downs: You can click any data point in the bold blue font for a deeper dive. These drill-downs will present you with all relevant Jira links for further investigation.

• Hovering: Each report has a tooltip explaining what it covers. You can also hover over each graph type to get a breakdown for that point in time.

• Viewing: all pages within the CloudBees platform can be viewed in light or dark mode. For this blog, we use dark mode.

This report provides detailed insights into the performance and health of software engineering. It provides CI/CD and engineering metrics to users. Teams can optimize their software delivery process and improve the overall quality of their software. This consists of multiple widgets, including: 

• Component, workflow, and run totals.

• Commits and pull request trends.

• Code churn (code change rate).

• Percentages of successful builds and deployments.

• Average development cycle times.

• Average deployment time.

DORA metrics are a benchmark for your software processes' effectiveness and a cornerstone for value stream management.  This report provides detailed insights into the continuous delivery of software.

• Deployment frequency: Track total successful deployments per day.

• Deployment lead time: Provide insights into the deployment lead time and streamline the process.

• Change failure rate: The percentage of changes that fail when deployed to production. 

• Mean time to recovery: The average time it takes to recover from an incident or outage.

Currently, our DORA reports track performance over time. It will soon extend this functionality to include a comparison with other teams in an organization for how good the DORA metrics are. Click here to see how your organization matches industry benchmarks for elite, high, medium, and low performers.

Our flow metrics report provides detailed insights into the productivity of work items for software engineering teams. It also measures what types of work items, durations, and bottlenecks exist for various work items. Monitoring flow metrics can improve predictability, bottleneck identification, risk management, and cost-effective software delivery.

The CloudBees platform calculates five flow metrics that measure how value flows through a product value stream, as follows:

• Workload: Number of flow items currently in progress

• Work items distribution: Proportion of each flow item type

• Velocity: Number of completed flow items

• Cycle time: The time it takes to complete a flow item from start to finish

• Efficiency: Percentage of active time versus total cycle time 

Users must create a project within the Analytics Configuration navigation to configure Flow metrics.

By using DORA and Flow metrics together, teams can understand the software delivery process comprehensively, from overall performance and impact to detailed workflow. These insights enable teams to make data-driven decisions and take action to continuously improve the delivery process, leading to better quality software, faster time-to-market, and improved business outcomes.

Our security insights report provides detailed insights into the results of security scans. It provides a measure of how vulnerable the systems are. Engineering teams and CISOs can set SLAs to resolve vulnerabilities promptly. 

It includes the CWE Top 25, a vulnerability list compiled by MITRE that lists the common security vulnerabilities with the most severe impact. This widget lets you instantly identify which components are impacted, how many occurrences, and the SLA status to help troubleshoot.

All reports are powered by a scalable and flexible Cassandra DB, which ingests data from workflows, scans, and integrations like JIRA. This data is indexed by an open search cluster, where an analytics service provides some pre-computation services so that the data is already pre-computed for all the widgets. 

When a user opens one of the reports, it connects to the report service via a secure API gateway for quick access. 

You should now understand what analytics reports are available within the CloudBees platform—try it now for free.

To dive deeper into each of these reports, continue reading our blog series:

• Software delivery activity

• Security insights

• Flow metrics

• DORA metrics