Customer Success Stories of Faster, Smarter Releases with Launchable by CloudBees’s AI-Augmented QA
Speed is everything for developers. As teams push to build, test, and release software faster, the demand for high-quality code only grows—but thorough quality assurance (QA) can sometimes slow things down.
Traditional testing processes often create bottlenecks that stall productivity and delay feedback loops. Long test queues, slow build times, and overwhelming cloud resource costs are just a few of the hurdles standing between developers and efficient workflows.
These challenges are especially pronounced when scaling projects, adding new features, or dealing with increasingly complex test suites.
Many organizations are turning to AI-augmented QA to streamline the testing process and ease the manual work involved in getting code from commit to production. By automating key parts of quality assurance, developers can ship releases faster without sacrificing quality.
While AI-powered tools like GitHub Copilot have already gained popularity, there's still a huge opportunity for specialized AI tools to elevate engineering workflows, from DevSecOps to QA, according to ICONIQ’s 2024 State of AI Report.
With the power of AI, Launchable by CloudBees helps QA teams and developers use Predictive Test Selection (PTS) to identify which tests are most likely to fail and prioritize them. AI is like a smart filter, sorting through the noise of test failures and allowing only the most important issues to rise to the surface, making test management more efficient.
By running smaller, more focused subsets of tests, developers can cut down on test execution times, optimize cloud resources, and improve developer feedback loops.
In this blog, we’ll explore how three distinct Launchable by CloudBees customers—GoCardless, BMW, and Delphix—are leveraging AI-augmented QA to overcome these challenges and achieve impressive results.
1. GoCardless: Cutting Cloud Costs and Reducing Test Times By 50%
GoCardless is a global leader in direct bank payments, processing over $35 billion annually across 30+ countries. With such a large volume of transactions and a growing number of tests in their pipeline, the GoCardless team was facing a common challenge in the world of development: slow CI times that were limiting productivity. Their Developer Enablement team needed an AI-augmented QA solution that would help reduce testing times while simultaneously cutting down on cloud resource costs.
The Challenge
GoCardless had a unit test suite with around 60,000 test cases. Running this suite on every push function across multiple branches created significant strain on cloud resources, and the team could not afford to scale up further. They needed to reduce testing time by at least 40% to meet their service-level objectives (SLO).
The Solution
By implementing Launchable’s AI-powered Predictive Test Selection, GoCardless was able to intelligently subset tests, focusing on those most likely to fail and running them first. This not only improved developer productivity by drastically reducing test times but also led to significant cost savings in cloud resources.
The Results
With an AI-augmented QA solution, GoCardless cut machine hours by 50% per test run, saving 8,500 hours in their first full month of implementation. Testing times dropped from over 300 minutes to just 48 minutes per run, dramatically improving the CI pipeline's speed and test efficiency.
“Thanks to Launchable by CloudBees, we’ve managed to increase the speed of our pipeline for our engineers, reducing waiting time and costs and increasing productivity and satisfaction all around. The product is thorough and easy to integrate, which allowed us to get it on production in just one month.”
Bastian Zamorano
Product Manager for Developer Enablement at GoCardless
2. BMW: Optimizing Hardware Resources and Speeding Up Feedback Loops with AI-Augmented QA
BMW’s DevSecOps team faced a unique challenge. Testing against physical hardware, such as head units (HUs), was slowing down their builds and increasing queue times. As more tests were added, hardware constraints grew, and resource costs began to spiral out of control. With the pressure to provide fast feedback to developers, BMW turned to Launchable by CloudBees to solve their hardware usage problems while still maintaining test quality.
The Challenge
BMW’s test suite was designed to run on real physical hardware, but the limited number of testing stations meant developers had to wait long periods to see if their code changes had passed. As the team continued to scale, this issue worsened, leading to increased queue times and resource usage. AI-Augmented QA was crucial in addressing these scaling issues by helping the team prioritize and streamline testing processes.
The Solution
Launchable’s PTS allowed BMW to focus on the tests most likely to fail, reducing the amount of time and hardware resources needed for each run. By utilizing its split subset feature, they were able to evenly distribute tests across available hardware resources, optimizing test execution.
The Results
The result was a dramatic reduction in hardware usage, as well as more efficient parallel testing. By prioritizing tests based on their likelihood of failure, BMW saved significant time and reduced the costs of running tests on physical hardware. This approach seamlessly integrated the power of AI and ML into their team's workflow with minimal effort.
3. Delphix: Reducing Costs and Saving 40,000 Hours of Test Execution with AI-Augmented QA
DevOps data platform Delphix (now Perforce) provides on-demand data for development, testing, and analytics environments. With a focus on accelerating software delivery, Delphix wanted to streamline its testing processes to facilitate monthly releases and speed up its go-to-market time, all while maintaining data integrity and product quality.
The Challenge
Delphix was facing long-running test suites that could take days to complete. This was problematic as the team relied heavily on regression tests to ensure the quality of their releases. The process was not only time-consuming but also costly in terms of cloud resources.
The Solution
Launchable’s PTS helped narrow down their test cases and execute them in parallel, saving time and reducing cloud infrastructure costs. This AI-augmented QA solution allowed them to run only the most relevant tests, ensuring faster feedback loops and optimize resource usage.
The Results
Delphix saved over 40,000 hours of test execution annually, which translated into reduced cloud costs and faster release cycles. The improvements helped them achieve their goal of delivering monthly product releases with greater confidence. This allowed them to ship products faster while maintaining high-quality standards.
How AI-Powered QA Solves Common Challenges to Boost Productivity Across Industries
While GoCardless, BMW, and Delphix serve different industries, they all share common challenges: slow testing times, resource inefficiencies, and the need for faster feedback to developers. These challenges are not unique to any one business, but Launchable by CloudBees’ AI-augmented QA solutions have helped them tackle these issues head-on.
By implementing AI-powered Predictive Test Selection, all three teams were able to:
Reduce Testing Times: Whether it’s cutting machine hours or reducing the number of tests executed, each team saw faster feedback cycles.
Optimize Hardware/Cloud Costs: By running only the necessary tests, each team saved valuable resources and reduced infrastructure costs.
Improve Developer Productivity: With reduced waiting times, developers could focus on coding instead of waiting for tests to complete, resulting in quicker deployments and improved job satisfaction.
Boost Customer Satisfaction and Revenue: Faster, reliable releases enhance customer experience, drive loyalty, and optimize revenue by ensuring high-quality products reach the market more efficiently.
As software complexity and speed demands rise, AI-powered testing is now essential for teams looking to deliver quality code faster and stay ahead of the competition.
Launchable by CloudBees helps teams achieve real ROI with AI-augmented QA, offering a flexible, all-in-one approach to optimize your test suites.
Book a demo and start driving faster, more efficient releases.