How to Test in Your Local Environment When Using a Feature Flag Management System

Feature flag management systems are wonderful things, leagues above the normal "throw all my feature flags into a config file" technique. They let you change them at runtime, sometimes with a fancy dashboard. Some of them support more advanced flags that may take in the current request context. However, it is reasonable for you to be concerned about taking a dependency or (likely another) third-party dependency. Every third-party tool runs the risk of making your system harder to maintain and harder to test. Feature flag management systems are no different. I want to hone in on the concern that a feature flag management system will make your system more difficult to test, especially when writing automated unit tests. When we write unit tests, we don't always get the luxury of spinning up our entire system, and even if we do, having to depend on a network connection just to experiment can lengthen your feedback loop, slowing you down. So today we are going to look at some ideas that help us mitigate or eliminate the testing cost of using a feature flag management system. But before we start, let's be honest: "feature flag management system" is a mouthful. Too much of a mouthful. Let's shorten it to something more reasonable. Now this being the software industry, you might think, "How about FFMS?" I say, nah, that's too stiff. Let's call it your feature flagger.

Before we move on, identify exactly what you're trying to test and why. Are you looking to test how your app interacts with your feature flags, or are you questioning whether or not the flagger itself is working? You may think I should not have to ask, but you'd be surprised. I just want to take a quick moment of review to ensure that you are testing the former and not the latter. If you're questioning whether the flagger is working, please stop. Someone else has likely spent a lot of time and money to test it themselves before delivering it to your doorstep. Okay, great. Onto the good stuff.

The easiest way to locally test may be to see if your feature flagger has a local development version. You can spin that sucker up and it's ready to go, waiting for flags while you run your app. Bootstrap it to your command line, Gradle, NuGet manifest, etc. Or if you are feeling especially clever, drop it in a container you use for local development. Now, that's fancy. If this is the case, you can stop here. No need to strain your eyes further than you have to. But maybe your flagger—yeah, that name keeps getting shorter—doesn't have a local development version. Or perhaps its local development version is slow, and you don't want to deal with the pain of interacting with it during your fast, test-driven feedback loop. A lot of solutions these days exist in the cloud. In this case, read on.

Ideally, you don't have to test your feature flag directly at all. At its core, a feature flag is a piece of data saying whether or not you should run one set of code or another set of code. See below:

public void doStuff() {
 if(featureFlagger.getFlag("doTheThing")) {
   doThing();
 } else {
  dontDoThing();
 }
}

More advanced flags may have more than two paths, like a flag that returns an enumeration instead of a Boolean. But you can push the results to the boundary, making them a parameter of whatever method you are testing. This is a key technique in functional programming. With the flagger itself out of the picture, it's back to testing business as usual, as the code below demonstrates.

public void doStuff(bool doTheThing) {
 if(doTheThing) {
   doThing();
 } else {
  dontDoThing();
 }
}

[Fact]
public void testTehThingshouldDoTheThing() {
   sut.doStuff(true);

   assert.that(thingDone());
}

As convenient as it is to push your flags to the boundary of your test, this is not always possible. Perhaps you need to dynamically query the flag every so often. Or perhaps it requires input, like user region, that the code you are testing needs to pass to it. There are many reasons why the above tactic may not work. Where does that leave us? We have to go back into our toolbox of patterns and practices and pull out a classic. In a healthy application, the idea of dependency injection, also known as inversion of control, lets us give up control of the lifecycle of dependencies. If you are not familiar with the idea, there are quite a few good resources on it. Instead of carrying around the bulk of instantiating our flagger within the code we are testing, we can push it out. We are not pushing out the results of a flag like we did above, but are instead letting some other piece of code deal with the heavy lifting and configuration of the system itself. See the example below. Before dependency injection:

public void doStuff() {
 flagger = new FeatureFlagger(new FeatureFlaggerConfig());

  if(flagger.getFlag("doTheThing")) {
   doTheThing();
  } else {
   dontDoTheThing();
  }
}

After dependency injection:

public void doStuff(FeatureFlagger flagger) {
  if(flagger.getFlag("doTheThing")) {
   doTheThing();
  } else {
   dontDoTheThing();
  }
}

Once we do this, we can make it easy to test by optimizing the configuration for testing locally or even stubbing out the flagger, like below:

[Fact]
public void shouldDoStuff() {
 var flagger = mock(typeof(FeatureFlagger));
 when(flagger.getFlag("doTheThing")).thenReturn(true);

 sut.doStuff(flagger);

 assert.that(didTheThing());
}

Now we can completely control what the flagger will give us while testing.

These are some useful techniques, but we can use them in combination or instead use something else. We actually don't need to get the flagger involved at all. We and just wrap it in a thin, nifty adapter. Once we do this, we can create our own test version using a mocking framework to do what we like. See below:

public interface IFlagFeatures {
  bool getFlag(String flagName);
}

public class FeatureFlaggerWrapper : IFlagFeatures {
   inner = new FeatureFlagger(new FeatureFlaggerConfig());

  public bool getFlag(String flagName) {
    return inner.getFlage(flagName);
  }
}

public class TestFeatureFlagger : IFlagFeatures {
  Dictionary<String, bool> flagSetups = new Dictionary<String, bool>();

  public void setupFlag(String flagName, bool result) {
   flagSetups.add(flagName, result);
  }

  public bool getFlag(String flagName) {
    return flagSetups.get(flagName);
  }
}

public void doStuff(IFlagFeatures flagger) {
 if(flagger.getFlag("doTheThing")) {
  doTheThing();
 } else {
  dontDoTheThing();
 }
}

Why would we do this when the other techniques are available? Well, remember that we can use this alongside the other techniques. This is also useful in situations where the flagger may be hard to test or the interface is too jarring to work with directly.

Testing your feature flag management system doesn't have to be hard. Remember that in the end, you just want to test how you interact with the system. Someone else already tested the system itself. By baking in healthy software practices such as dependency injection and wrapping third-party dependencies, you can make testing a breeze.