Editor’s Note: This blog post is the first in a series of software development design patterns for continuous everything in DevOps. It’s modeled after the infamous Gang of Four book for object-oriented design patterns.
You’ve all heard how software developers are becoming the “kingmakers” of the industry. It’s no surprise when nowadays just about everything is powered by code. But how developers can improve their skills in the DevOps world (and any other industry for that matter) can stem from a variety of avenues -- write more and more code, read software engineering books to learn new techniques, learn from fellow developers writing blogs about how they solve a problem or simply watch a YouTube video.
In this series of blogs, we’ve chosen a set of software development design patterns to help developers keep up to date on the patterns most important to the DevOps industry. This first one focuses on value stream management (VSM). This concept enables developers to see where blockages and DevOps waste occur so that decisions can be made on how to improve the software delivery process and make it an efficient operation.
Name
Value Stream Management
Intent
Value stream management (VSM) helps optimize the software delivery process by reducing waste and improving efficiency. The scenarios below describe situations that might have become ingrained into your system so much so that they appear normal. Beauty is in the eyes of the beholder but to protect the customer, the beholder might need to refine their vision and wear a stricter pair of lenses.
To understand better, let’s evaluate a couple of scenarios which appear normal at the onset but can be wasteful from the customer’s perspective. Can you relate to one or more of these scenarios?
Motivation
In their zeal to automate, engineers often fall prey to automating waste inside their pipelines, alongside codifying the value-adding steps. VSM visualizes the wasteful portions of your processes and to help you reduce or eliminate waste and thereby improve your company’s bottom line. As if that isn’t enough, you also discover the root causes and the sources of the waste and can take measures to prevent waste from recurring.
For example, risk-averse organizations institute periodic reviews between engineers and a Change Approval Board (CAB) to inspect deployments to production. While the intent might be to protect the customer, the reality is that CAB members introduce friction and delay by manually reviewing and approving every change. In a vain attempt to optimize the process, engineers automate the generation of a report that summarizes the various changes for the CAB to review. When you zoom in, this sounds like a useful optimization. When you zoom out with VSM, both the automated report generation and the CAB process can be seen as wasteful steps.
Scenario 1
Development writes new features and hands off to QA for testing. QA tests the code and hands off to release engineering and infosec. If a defect is found, the guilty artifacts are hurled back at development. Once every department (silo) certifies the versioned artifacts, the signed-off artifacts are handed over to operations.
While this sounds like a perfectly feasible development and delivery workflow, visualizing the value streams clearly reveals that there’s delay introduced at each hand-off amounting to hours or even days. Moreover, these manual handoffs occur via meetings and emails, which aggravate the drag. Value stream management enables us to analyze and optimize the development and delivery process in a surgical fashion.
Scenario 2
As soon as QA discovers a defect, they send the offending artifact back to development, who put in a fix and send it back to QA for a second iteration. QA verifies that the first bug was fixed but now discovers a second one and throws the guilty artifact back at Development. And so in a lively game of ping-pong.
A VSM would clearly expose wasteful behavior and wait times. QA might consider running all or most tests instead of getting blocked and tackled on the first unit test that fails. It is more efficient if they know and report all known issues at once instead of one by one.
Even more important is that when you analyze using VSM, you establish the business case for continuous everything. Code should move along pipelines via pre-meditated and automated steps like unit tests, static code analysis, builds, code reviews, artifact versioning, deployment, functional tests, performance tests, security tests and others. These pipelines gnaw through the silos mentioned in scenarios 1 and 2 and eradicate the culture of handoffs and sign-offs.
Applicability
Value stream management is a lean manufacturing technique that analyzes, designs and manages the flow of materials and information required to bring a product to a customer. Teams map steps as either value-adding or non-value-adding from the customer’s standpoint, with the purpose of rooting out items that don’t add value for the customer. These maps improve the efficiency of any process in which there are repeatable steps and multiple handoffs.
The application of value stream management isn’t limited to manufacturing and the assembly line. Lean VSM is gaining momentum in knowledge work and has become the de-facto solution to optimize software development and delivery workflows. Organizations love to optimize this workflow to the fullest simply because they get paid once their customers are satisfied.
The figure illustrates a simplified workflow where code committed by developers at the left-most end is consumed by customers as finished products at the right-most end. As depicted in the figure, engineering organizations can be fragmented into dev, QA, tools, infrastructure, platform, release, infosec, operations and others, thereby leading to multiple silos and handoff delays. By analyzing the steps in this workflow, VSM helps teams get rid of silos and bottlenecks that impede the speed and predictability of delivery.
This results in better communication and effective collaboration. Much of the software development and delivery waste in knowledge work occurs in the handoffs (or wait time) between individuals and teams. Inefficient handoffs not only lead to low productivity and poor quality but also inflate the cost of delay .
Continuous everything (integration | testing | delivery | deployment | analytics | governance) does not mean that everything you do today needs to be (blindly) automated. Why? First, there are some steps that are wasteful for the customer. Reduction or even elimination of wasteful steps should be prioritized in your backlog since they have a super high return on investment (ROI). Second, you should pick the lowest hanging fruit first where you can make the biggest impact. Once you demonstrate a positive RoI from this exercise, you can tackle more intense optimization problems.
Enter VSM! VSM helps identify waste and streamline the product development and delivery processes of not just software products but also hardware, firmware, embedded systems and IoT. By definition, continuous everything doesn’t need to make use of VSM but small, mid-sized and large organizations have reaped rich dividends from this exercise while accelerating their digital transformation initiatives.
Consequences
Digital transformation initiatives like continuous everything can ignite fear and uncertainty in the minds of people who fear they will lose their jobs to automated agents. VSM can have a similar effect since it uncovers wasteful processes that have grown roots over a period of time. When these wasteful steps are eliminated, people who own execution of these steps can also get affected by workforce reduction. Hence conducting VSM can often be met with some resistance.
Another unintended consequence of value stream management is that teams tend to waste a lot of time learning industry-standard VSM symbols and formalizing a map with those standard symbols. Remember, the map is not the deliverable. Focus on the insights that stem from the map and not so much on decorating the map. Strike the right balance and by keeping a hawk eye on RoI (return on investment).
It’s worthwhile to emphasize that during the VSM exercise, you are trying to cut waste and not create any more than you already have. Use of professional charts, tools and symbols is good but not mandatory while mapping your value stream. To get started, you can sketch with a pencil or use a whiteboard.
Implementation
Major players have joined hands to bring forth sophisticated solutions around value stream management. Some of the notable leaders are:
VSM can be as effective when done in a rudimentary fashion on whiteboards and with sticky notes when it is done with sophisticated tools as mentioned above. As discussed under “Consequences,” focus less on formalizing the value stream map itself and concentrate more on the insights revealed by the map. As you become more familiar with VSM, you can formalize your maps and format them more professionally.
Known Uses
Uses of VSM is diverse and widespread across a multitude of industries. Be it software development, supply chain, manufacturing, services, or healthcare, processes need to be optimized to streamline value creation and delivery to customers. Let’s look at a few instances where value streams are prevalent.
In software development, VSM weeds out steps like CAB (Change Approval Board) meetings that occur periodically to review and approve changes made to production.
In the supply chain, a VSM can root out costly delays leading to a finished product.
In manufacturing, a VSM helps in finding waste by analyzing each step of material handling and information flow. The process items that flow through the value stream are materials.
In service industries, a VSM facilitates effective and timely services for external customers, whereas inside administration and offices, it facilitates services for internal customers.
In healthcare, a VSM ensures patients are effectively treated with high-quality care. The process items that flow through the value stream are customer needs.
Here are a few VSM case studies:
Related Patterns
There are other design patterns that aren’t directly related to VSM but are based on similar concepts.
Lean: VSM has its roots in Lean.
Process as code: VSM helps in the optimization of processes by identifying waste. A process workflow, when codified, enables automation. Codification also helps in visualization and that in turn helps users in assessing the various work streams.
Pipeline as code
References
Here’s more information on VSM.
Additional resources
Dive into the Gang of Four Design Patterns model