How to Disable Code: The Developer's Production Kill Switch

The following is a guest post written by Carlos Schults.

Being able to disable code in production is a power that many developers aren’t aware of. And that’s a shame. The ability to switch off some portions—or even complete features—of the codebase can dramatically improve the software development process by allowing best practices that can shorten feedback cycles and increase the overall quality.

So, that’s what this post will cover: the mechanisms you can use to perform this switching off, why they’re useful and how to get started. Let’s dig in.

Here's an outline of what we'll cover:

• Why Would You Want to Disable Code?

• How Do I Disable a Section of Code? Feature Flags To The Rescue!

• Disable Code in Production 101

• Feature Flags Are CI/CD Enablers

Before we take a deep dive into feature flags, explaining what they are and how they’re implemented, you might be asking: Why would people want to switch off some parts of their codebase? What’s the benefit of doing that?

To answer these questions, we need to go back in time to take a look at how software was developed a couple of decades ago. Time for a history lesson!

Historically, integration has been one of the toughest challenges for teams trying to develop software together. 

Picture several teams inside an organization, working separately for several months, each one developing its own feature. While the teams were working in complete isolation, their versions of the application were evolving in different directions. Now they need to converge again into a single, non conflicting version. This is a Herculean task.

That’s what “integration hell” means: the struggle to merge versions of the same application that have been allowed to diverge for too long. 

“If it hurts, do it more often.” What this saying means is that there are problems we postpone solving because doing so is hard. What you often find with these kinds of problems is that solving them more frequently, before they accumulate, is way less painful—or even trivial.

So, how can you make integrations less painful? Integrate more often.

That’s continuous integration (CI) in a nutshell: Have your developers integrate their work with a public shared repository, at the very least once a day. Have a server trigger a build and run the automated test suite every time someone integrates their work. That way, if there are problems, they’re exposed sooner rather than later.

One challenge that many teams struggle with in CI is how to deal with features that aren’t complete. If developers are merging their code to the mainline, that means that any developments that take more than one day to complete will have to be split into several parts. 

How can you avoid the customer accessing unfinished functionality? There are some trivial scenarios with similarly trivial solutions, but harder scenarios call for a different approach: the ability to switch off a part of the code completely.

There are many names for the mechanisms that allow developers to switch a portion of their code off and on. Some call them “feature toggles” or “kill switches.” But “feature flags” is the most popular name, so that’s what we’ll use for the remainder of this post. So, what are feature flags?

Put simply, feature flags are techniques that allow teams to change the behavior of an application without modifying the code. In general, flags are used to prevent users from accessing and using the changes introduced by some piece of code, because they’re not adequate for production yet for a number of reasons.

We’ll now cover some of the most common use cases for disabling code in production.

As you’ve seen, one of the main use cases for feature flags is preventing users from accessing features that aren’t ready for use yet.

That way, programmers developing features that are more complex and take a longer time to complete aren’t prevented from integrating their work often and benefiting from it.

The adoption of feature flags enables the use of several valuable practices in the software development process, one of which is A/B testing. 

A/B testing is a user experience research technique that consists of comparing two versions of a website or application to decide which one to keep. It entails randomly splitting users into two groups, A and B, and then delivering a different version of the application to each group. One group might receive the current production version, which we call the “control,” whereas the second group would receive the candidate for the new version, called the “treatment.” 

The testers then monitor the behavior of both groups and determine which of the versions achieved better results. 

Feature flags are a practical way to enable A/B testing because they allow you to quickly and conveniently change between the control and treatment versions of your application.

If you deliver the new version of your app to your entire userbase at once, 100 percent of your users will be impacted if the release is bad in some way. What if you could gradually roll out the new version instead? You’d first deploy to a small subset of users, monitoring that group to detect issues. If something went wrong, you could roll it back. If everything looked fine, you could then gradually release the version for larger groups. 

That’s a canary release in a nutshell. It’s another powerful technique that feature flags might help with.

It’s not uncommon to have to customize your application according to the needs of specific users, and there are several ways in which software teams can accomplish that—some more efficient, and others less so (companies that create separate branches or entire repositories for each client come to mind).

This is another area where feature flags could help, allowing teams to dynamically switch between different versions of the same functionality.

How do you go about disabling code? That’s what we’re going to see now, in three increasingly sophisticated phases.

We start with an approach that’s so primitive, it maybe shouldn’t be considered a feature flag at all. Consider the pseudocode below:

calculateAdditionalWorkHours(Employee employee, Date start, Date end) {

// return calculateAdditionalWorkHoursSameOldWay(employee, start, end);

return calculateAdditionalWorkHoursImproved(employee, start, end);

}

In the code above, we’re just commenting out the old version of some method and replacing it with a new version. When we want the older version to be used, we just do the opposite. (Well, I said it was primitive.) This approach lacks one of the most fundamental properties of a feature flag—the ability to change how the application behaves without changing its code.

However, it plants the seed for more sophisticated approaches.

The previous approach didn’t allow developers to select the desired version of the feature without changing the code. Fortunately, that’s not so hard to do. First, we introduce a logical variable to determine which version we’re going to use:

calculateAdditionalWorkHours(Employee employee, Date start, Date end) {

    var result = useNewCalculation

? calculateAdditionalWorkHoursImproved(employee, start, end)

: calculateAdditionalWorkHoursSameOldWay(employee, start, end);

    return result;

}

Then, we use some mechanism to be able to assign the value to the variable from an external source. We could use a configuration file:

var useNewCalculation = config[newCalculation];

Passing arguments to the application might be another option. What matters is that we now have the ability to modify how the app behaves from the outside, which is a great step toward “true” feature flagging.

Keep in mind that the code examples you see are all pseudocode. Using your favorite programming language, there’s nothing stopping you from starting with this approach and taking it up a notch. You could, for instance, use classes to represent the features and design patterns (e.g., factories) to avoid if statements.

The previous approach might be enough when your application has only a small number of flags. But as that number grows, things start to become messy.

First, you have the issue of technical debt. Manually implemented feature flags can create terribly confusing conditional flows in your codebase. That only grows worse with new flags being introduced each day. Additionally, they might make the code harder to understand and navigate, especially for more junior developers, which is an invitation for bugs.

Another problem is that as the number of flags grows, it becomes more and more common to forget to delete old, obsolete ones. 

The main problem of a homegrown approach is that it doesn’t give you an easy way to see and manage all of your flags at once. That’s why our third and final stage is a single piece of advice: Instead of rolling out your own feature flags approach, adopt a third-party feature flag management system. 

We’ve covered the mechanisms developers can use to disable portions of their codebase in production without having to touch the code. This capability is powerful and enables techniques such as A/B testing and canary releases, which are all hallmarks of a modern, agile-based software development process.

The names for the techniques might vary—feature flags, feature toggles, feature flipper, and so on. The way in which the techniques are implemented can also vary—from a humble if statement to sophisticated cloud-based solutions.

But no matter what you call them, you can’t overstate the benefit these mechanisms offer. They’re an enabler of Continuous Integration, which is essential for any modern software organization that wants to stay afloat.

Carlos Schults is a .NET software developer with experience in both desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices that help you improve app health, such as code review, automated testing, and continuous build.