Decoupling Deployment From Release With Feature Toggles

Deploying new features is a nerve-racking task, but it must be done.

Here’s an issue we’ve all faced or will inevitably face in our careers. Allow me to set the scene…

You’ve finished your work on some features, fixed some bugs, etc. You’re feeling good because some features were a bit more complicated. You decide it’s high time to let your users experience these changes. You bundle up all your teams’ changes and deploy them to the production server. A couple of hours later, disaster strikes! There’s a critical bug on a feature you’ve worked on. Now, you have to revert to the old version of the application or deploy a new one, but without your changes.

As these situations can have a huge impact, reacting quickly is key. Unfortunately, developers usually learn how to handle this situation once it happens, so it only prolongs the agony.

Deploying and releasing at the same time isn’t such an uncommon practice because, at the beginning of a project, you want your users to see the changes immediately. Many projects continue using this process, but as the project grows, it makes sense to decouple the deployment of the code and the release process (activating changes to users).

Even if this hasn’t happened to you yet, we can all agree that it’s a very stressful situation, and establishing a process for handling this is something you want to do ahead of time. 

But how much ahead of time are we talking about here?

There are many ways to handle these types of outages, and it’s important to establish a rollback procedure. In addition to that, we also use something called feature toggles. It’s not something unique to us, but it’s a pretty sweet thing to have in your toolbox. For those unfamiliar with feature toggles, it’s a mechanism that allows you to turn a feature on or off for a user without needing to deploy the whole application. It’s really simple, but it’s huge!

Although it’s called a “feature toggle”, it doesn’t necessarily mean that you have to use them for features only. You can use them basically for anything. Whether it’s A/B testing, refactoring a part of the screen, or navigating a user to a different page entirely! I’m sure you can think of a couple of more examples that would be useful to you.

This also allows us to oversee which users can access a certain change. This is helpful when trying to collect feedback or just observing how users interact with something you’ve developed. Once you’ve established your feature toggle system, you don’t need to put in continuous work to keep it going. It’s also a great way to manage the lifecycle of the change by progressively rolling it out to your canary, beta, and all users! 

You gain a lot more control over who sees what, and the best part is that turning toggles on or off can be done by anyone! Someone like a product manager or customer support person could decide to turn a toggle on because it’s just something that the customer needs. After a feature has been fully developed, the toggle can be globally released to all customers, and everyone can enjoy the stable version.

The number of annoyed users will decrease, and valuable feedback on changes will increase. 

It’s also worth mentioning that you can base your whole QA process on the toggles. We can test a change by turning a toggle on, gathering feedback, and then turning the toggle off. When you have such fine-grained control, testing on production isn’t a big deal! We also use these toggles in other environments, so you can isolate a change to a specific set of users on any server.

It’s a win-win for everyone!

The main question that’s probably going around in your head is how the feature toggles affect the development of a feature.

Feature toggles do complicate things a bit when writing code, but if you spend some time on planning, it can be done elegantly in most cases. You must be careful to add every change behind the toggle so nothing leaks when the toggle isn’t turned on. Other people working on the feature must also be aware of using the toggle since it probably will span through multiple files and can sometimes be forgotten.

Sometimes, it’s not that complicated, but you gain a lot. As I’ve already mentioned, it can be used for basically anything, and complexity may increase, but it really is worth it:

The added complexity isn’t such a big drawback. You’re getting a lot more out of feature toggles, and if that means that developers have to be a bit more careful when writing and reviewing code, so be it. It’s much less stressful for a developer when they know a feature can be turned off if it’s causing problems. This doesn’t mean we approve of sloppy code, but sometimes, these things happen.

To conclude, feature toggles (once established) don’t affect the development process as much, but they offer a lot in return.

The first one does introduce technical debt, but if we create a task for deleting a toggle after it’s been globally released, we can be sure that it will be deleted as soon as possible. The person who created the toggle is responsible for deleting it, which is a great way to handle this efficiently. In any case, deleting a toggle usually takes significantly less time than developing it!

As for the other issue, there are two ways to handle this. We can do either of these two:

It’s not an ideal situation, but it can happen. The solution mostly depends on the state of the features, so it’s impossible to determine the best way without knowing all the facts. Either way, both solutions can simplify things for development.

For us, implementing feature toggles was a no-brainer. Considering all of the benefits like the control, progressive rollouts, and others, it seriously outweighs the downsides it introduces. Even the downsides can be managed to an extent.

It’s important that everyone in the company understands the whole process and embraces why we’re doing it in such a way. It’s an essential part of a company like ours since every department is involved in the release process of product updates. Once people are aware of this, building an admin system to control this is up to you and your company’s needs. 

Here’s an example of our own. We call it the “Backoffice”:

As a developer, I’m grateful we can release features quickly and stress-free, knowing we have some wiggle room.

As for you, I hope I’ve shown you why it’s a good idea to think about something like this if you don’t use a similar mechanism. If you need a bit more convincing, you can read more about it in an article by Martin Fowler! Even if you’re not in a position to make your feature toggle system, there are a lot of external tools and libraries that might help you implement the same!