The Top Questions About Feature Flags, Answered

When we first introduce people to the concept of feature flags, they often wonder the same things. Don’t feature flags just make testing more complicated? How do you know when to retire  a flag? How is an enterprise tool any different than a homegrown system? Aren’t feature management tools just a glorified way to do A/B testing? The list goes on and on.

I spoke with Elliott Landon and Justin Pflughaupt, solution architects for CloudBees Feature Management with deep knowledge of the tool as well as of feature flags in general. They outlined a few of the most common questions people ask them in product demos and other conversations. This article collects and summarizes their answers to these frequently asked questions. Who knows—you may find your own questions about feature flags answered here.

A/B testing generally happens on the user experience side, and doesn’t test technical feature performance at all (i.e., what’s going on behind the scenes). Feature flags, on the other hand, are a technical tool originally created to help engineering teams test and deliver features much faster. With feature flags, you can manage features and scale releases based on a wide range of criteria, which can include data from the user experience side. Feature flags enable A/B testing and experimentation, but A/B testing tools can’t do what feature flags do. 

CloudBees Feature Management can also integrate with best-in-class tools like Google Analytics to enable experimentation on the user experience side. Feature flags can be turned on and off based on the results of experiments conducted using analytics. With feature flagging tools like CloudBees Feature Management, you get the technical benefits for your engineering team as well as access to the analytics tools you already use for A/B testing.

Third-party, enterprise feature management tools are built to help you scale. Homegrown systems can be useful for a few simple flags that aren’t much more than “if else” statements in a single language. But if you don’t have the time and resources to introduce more complexity, an enterprise tool is going to be the way to go. 

Tools like CloudBees Feature Management offer multivariate flags, for those times when you need to dynamically configure two or more variants—if you wanted to change text in multiple places on the screen, for example. We also support all of the most popular programming languages, so you don’t need to develop support for every language your team uses. We’re hosted, so you don’t need to think about infrastructure management. You won’t get audit insights with a homegrown system, either, and you’re going to have to rely on a developer to make any changes. With the CloudBees Feature Management dashboard, though, someone less familiar with the code can enable a feature without re-deploying.

All of these capabilities would take a lot of development work to build on your own, which adds up to more developer time taken away from revenue-generating activities. When you need to scale, an enterprise tool like CloudBees Feature Management is more cost-effective than homegrown.

Whether your old system is homegrown or one of our competitors, you can use a mapping file. It’s fairly easy—you can map the names of your old flags to the names of our new flags, and we’ll pick them up.

Yes, feature flags do introduce more complexity. We can’t know whether a flag will be turned on or off in production, so it’s important to test both ways. Introducing a single feature flag theoretically doubles the amount of testing that needs to be done, and as you might imagine, validating both states for every single feature flag you use would be an astronomical amount of work. 

Fortunately, you don’t need to test every possible permutation. Many feature flags will never interact with each other, and most releases will only involve changing the properties of a single feature flag. It’s most important to test the configuration that you expect to go live in production, followed by your fallback configuration. To make this work, as Pete Hodgson recommends in his landmark article on feature flags, “a good convention is to enable existing or legacy behavior when a Feature Flag is Off and new or future behavior when it's On.”

It’s important to establish conventions for naming feature flags and for deciding how and when to remove them. Otherwise, you risk accumulating technical debt. Naming conventions include a namespace that identifies how a flag will be used, a label to show which teams are using it and a description to include details like its creation date and any dependencies. Enterprise feature management tools like CloudBees Feature Management can help you enforce naming conventions.

The CloudBees Feature Management dashboard has a flag status column that provides live updates on the kind of impressions your flag is getting. Impressions measure how many times a flag has been evaluated in a specific time frame and the value that users get. If a flag has been turned on or off for all your users for a certain duration, it will be automatically marked as “stale.” You can delete that flag and merge the feature to your main production code path.

There are also some circumstances where you may need to implement long-running feature flags—such as offering different features to the premium tier of your app versus the free tier. The dashboard also has a “permanent” flag status you can mark so that impressions aren’t counted.

For details on lifecycling feature flags, you can check out this blog post.

Enterprise flag management tools like CloudBees Feature Management have audit logs that show when each flag was created, who created it and when the values are changed for flags in different environments. You can search by environment and by date range. If you’re doing a config file or database change in a homegrown system, you may not have insight into who makes the changes and when they happen. This is another example of why having an enterprise tool makes things easier for the engineering team.

We support a wide variety of client-side and server-side SDKs. You can check our documentation for the complete list.

We don’t capture any kind of personally identifiable information. In fact, with our architecture, it’s impossible for us to collect it. We issue the rules for any given flag from the SaaS platform and the actual decision takes place on the application—essentially, each device decides for itself whether it belongs to the target group for each flag. All customer data stays local, with the customer.

It’s a quick process to get started. You don’t have to install anything. We walk you through example source code in the documentation. You’ll need to make sure that a container class is defined with a few flags and an environment key. The docs also describe how to set up a handshake, and once you do so, you’ll start to see those few flags appear in your code. The flags will also show up your dashboard, verifying that you’re fully integrated. The entire process generally takes less than half a day.

Jump in and get started with a demo of CloudBees Feature Management. Sign up for a free, full-access 14-day trial.