Skip to content
← Feature Experimentation

Welcome! đź‘‹

Feature Experimentation

Categories

JUMP TO ANOTHER FORUM

(thinking…)

Visibility into flag usage

Depending on the number of teams operating in FX, tech debt accumulates quickly if not managed. Help customers determine how and when to consider tech debt in their FX endeavors.

Some customers struggle with outdated flags that have not been used in a while, but it is hard for them to determine which ones are outdated.

Help customers understand how their users are reacting to a new feature rollout.

4 votes

We're glad you're here

Please sign in to leave feedback
Signed in as (Sign out)
Close
Close

We’ll send you updates on this idea

Chris Vafiadis shared this idea  ·   ·  Report…  ·  Admin →
How important is this to you?

We're glad you're here

Please sign in to leave feedback
Signed in as (Sign out)
Close
Close
New  · 

We're glad you're here

Please sign in to leave feedback
Signed in as (Sign out)
Close
Close
Submitting...
An error occurred while saving the comment
  • Simon Born commented  ·   ·  Report

    Referencing Sarah's answer here - my point was not about the performance of the variants connected with a flag (that would be an A/B test) but more about governance and usage of flags. In a decentralized setting, the tool is in danger of becoming cluttered.
    One is blind to what flags are still being used which hinders cleaning up.

  • Simon Born commented  ·   ·  Report

    We are currently tackling this topic and it is some headache. We run a decentralized experimentation setting.

    Problem is on the one hand that flags stay in the code although not in use anymore. What is worse - they also stay in the datafile, thereby constantly increasing its size.

    Finding out which flags are still in use and which are not is very complex in a decentralized experimentation setting with independent codebases.

    The solution I could imagine on Optimizely side would be to send sampled pings whenever a flag is evaluated (eg. 1/1000). Then, there should be an option to auto-retire flags that have not been called for more than x days.

    The "flag usage" events should not count against any budget (impression or MAU budget).

    To make this even more efficient (and put less strain on clients executing Optimizely FX) would be that Optimizely does recognize the standard frequency a flag is called and automatically adjust the sampling per flag - as in essence one ping per day suffices. Thinking about it this way, the datafile could be automatically updated eg. every hour and make sure to only send pings for those flags that had not been seen that day yet.

    I do find the point "Help customers understand how their users are reacting to a new feature rollout." strange. That is an A/B test and not a Targeted Delivery then.

  • Sarah Ager commented  ·   ·  Report

    I like this idea. Similar to our Metric Impact Reports and Program Reports in the Reporting tool, I'm curious if we could have some type of Flag Impact Report.