Unleash the Power of Feature Flagging in Product Management

In a previous blog post our Engineers discussed the technical implementation details about Feature Flags. In this blog post I discuss what product management problems can be managed and solved with feature flagging.

Product development takes time

Every Product Manager dreams of a world in which their features would come to life quick and easy. Code would be 100% bug-free. End users would completely understand the new feature and have zero suggestions for change. You would just build-and-deploy, build-and-deploy, over and over, with a continuous string of problem-free releases that give your users joy. Anybody relate to that world? Me either.

See if this sounds more familiar: software development is challenging. Technical challenges can hold up development. User requirements can be hard to pin down and get correct. Stakeholders may ask you to change a feature and then when reviewing the change ask you to change it back. Everything takes longer than planned. Any of this more relatable?

There are other hurdles, too. Priorities can change midway through a project and the business may require the team to pivot to a newly emerging need or opportunity. Features may be put on the shelf for a month and then picked back up.

All of these realities mean that you need to have flexibility built into your product to manage the messiness. Creating a new feature should always begin with creating a new feature flag and wrapping all the new code in the flag. This allows you to deploy your product to production anytime you want without concern about the current status of the new feature because it’s all hidden.

Releases shouldn’t be scary

One of the key mantras of feature flagging advocates is the “separation of deployment and releases”. Once you get your mind around this it’s such a paradigm change!

• Deployment is the process of pushing a new version of the code to production
• Release is the process of making new features available to your users

Historically these have been synonymous. How do you release a new feature to your users? You deploy the new version of the code of course!

So, what can go wrong? Let me count the ways…

• Defects in the software can cause impact to users
• Missed requirements that weren’t identified during UAT may come to light
• Users may not understand how to use the new feature
• A multitude of other nuances… that all end with “put it back the way it was for now”.

In the traditional format, the only option is to “un-deploy” the code and rollback to the previous version. With complex software this can be a challenging process that is often more difficult than the first deploy.

Feature flagging these processes allow them to be separated and become far easier to manage.

First you Deploy the code to production. As long as no fundamental technical issues occur then the new version of the code is now in place. However, no change has impacted the users yet because the feature is hidden behind a flag.

To Release the feature you activate the feature flag. Does that mean all the things I mentioned above can’t ever happen? Of course not! The fact that those things will happen is exactly why we use feature flagging. You can never eliminate risk from a project – all you can do is manage it.

Since you have separated deployment and release, there is no complex and risky code rollback. You simply deactivate the feature flag. Then in a calm and controlled way you work through the issues to get your feature to 100%. At some future point you deploy the changes, and then at another point in time you re-release the feature.

Change Management is a real thing

In the above section about things that can go wrong, one of the items revolves around user training and adoption. In the old days of “release the feature” being synonymous with “deploy the code”, that also means every release is big bang – once you deploy the code all users now have the new feature.

A key aspect of feature flags is that you can selectively target users with flags. After doing the deploy step, activating the feature flag can be a selective process that only impacts a small subset of users that will be the early adopters. Those users can vet out the real-life performance of the feature in production without impacting the majority of the user base. Once any findings have been smoothed out, the circle can be widened to another group of users activated in a managed way until all the feature has been released to all users.

Product Managers should own the feature flags

Here at MercuryWorks our product managers and delivery leads are the feature flag owners and are responsible for the entire flag lifecycle. This is because they are such an inherent aspect of the product strategy.

Because of this, MercuryWorks recommends a feature flag management tool like Launch Darkly. While it is certainly possible to build your flags strictly in code, having the flags managed in an admin interface that a non-technical product manager can control really allows you to leverage the power of the concept.

Taking this a step farther, remember our goal of separating deploy and release. With an admin interface you are also able to separate the responsibilities and areas of concern. Deployment is now an engineering-owned activity, and release is completely owned by product.

Feature Flags best practices

Are you convinced and ready to get started? Here’s some key rules to live by:

Name your flags well

Product Managers own the entire feature flag process here at MercuryWorks and that starts with creating and naming your flag. It seems minor but when you get multiple flags going at the same time within a large complex product, you’ll be glad you took the extra 5 minutes to be thoughtful here.

Examples of terrible feature flag names:

• feature-flag (you’d be surprised…)
• new-feature (this only works until you make a 2nd new feature and even if you don’t it’s still terrible)
• new-invoice-process (seems good, but terrible if this flag controls anything that isn’t related to the new invoice process)
• controls-use-of-new-invoice-process-and-associated-screens (too long and has unnecessary words – if the associated screens are fundamentally part of the new invoice process then they are redundant)
• Anything that doesn’t succinctly describe the feature

Keep your names short and to the point, but they need be clear and complete. Examples:

• invoicing-module (for an application that hasn’t previously had one)
• address-validation (for an application that hasn’t previously had any form of validation and the team is now implementing a new API to notify users when a bad address is entered)
• new-payment-workflow (for an application that has an existing payment process but you are implemented a new revised UI on how to process payments)

Spin them up, use them well, and then retire them

Feature flags are great but they do add complexity to the code. Once the feature is fully rolled out to all users, a critical final step is to write a user story for the team to remove the feature flag check and also remove the alternate logic that is now dead code from the solution.

Since Product Managers own the feature flags, it’s important for the PM to be driving this effort. The challenge is that it’s additional work for the team that if done correct produces no visible benefit for the users. But trust me: leaving old flags lingering and dead code hanging out in the solution will come back to haunt you one day. When you fully adopt the feature flag life, the most rewarding moment is when you ask the team to retire the flag because that is the true meaning of being complete!

Recycling is great but not when it comes to feature flags

This is a major no-no. Feature flags are cheap – always start with a brand new flag. If you have an old flag that you are tempted to re-use, go back and read the last paragraph first. If it’s unused it should be deleted.

Still don’t believe me? Ask the folks at Knight Capital what they think about reusing old flags and “dead code”.

Specify feature flag functionality as precise as any other functionality

Product Managers are well versed in the importance of writing strong Acceptance Criteria for their team on all the nuances of a new feature. You must be equally precise in communicating the exact interactions a flag controls.

Telling your team “wrap all this in a feature flag” and expecting to get the right results is just asking for a bad release experience. As an example, consider the “New Payment Workflow” process I suggested above. Specifications for this work might look like this:

• Post Payment button exists on three screens. All three should be directing to the same logic that initiates the payment process. In that shared logic section implement the feature flag to direct users to the new workflow when on and the old workflow when it is off.
• The “View Payment Status” navigation menu item needs to respond to the flag and take the user to the new view screen. When the flag is off it should continue to take user to the same screen it does today.
• The main dashboard “Payment Widget” also needs to respect the flag. In this case we are building a completely new widget so the screen should render the appropriate widget based on the flag setting.

Of course those likely end up being multiple user stories, but that level of specificity is required to ensure proper outcomes.

Feature flags are not always the right answer

To a man with a hammer, everything looks like a nail

Abraham Maslow

Don’t let feature flagging become your hammer. Here are some scenarios when feature flagging is not recommended.

Standard maintenance changes

For a product that is live and in production there are certain changes that you should just make. If at the top of your Account Detail page you currently show the Account Name and you now want to append the Account Status to the end of it, very likely the right thing to do is to just make that change. Spinning up flags for trivial changes will quickly bloat your feature flag list and your code.

Low-Impact changes to isolated functionality

If your application is primarily external facing but you have an internal admin screen that is only used by a handful of people, it may be an appropriate strategy to simply make those changes. Even still you will note I said “may”. In this case my assumption is that as a PM you work closely with the entire admin group and will have a high level of confidence by the time you go to production that there is no risk to manage.

By comparison, if the admin group is a small but powerful team that wields much influence and you do not have a strong relationship with them then by all means protect yourself with a flag!

Certain architectural changes

There are times when feature flagging may not be possible, or at least pragmatic. A prime example would be fundamental architectural changes in the underlying database of the application.

Consider an application that only supports one email address on the Customer record, and in the database this is represented as a single column on the customer table. The business has now requested the ability to support multiple emails. The database tables and relationships are going to have to be restructured to support one-to-many relationships, and a data migration will be required to move all existing data to the new location. In this scenario you don’t want some users accessing/updating the old emails on the main table while others are using the new table.

Conclusion

I hope this post allowed me to demonstrate the flexibility given to a Product Manager feature flagging as a strategic product management approach. My experience is that the process has been nothing less than transformative in the way I think about long term product development.