Provide SRM monitoring on the results page
Hey!
As Team Lead Web Analytics I often see teams struggling to solve SRM issues.
It would be immensely helpful if the results interface would provide a graph depicting the user distribution between variants. This seems like a low-hanging fruit for Optimizely and would facilitate debugging SRMs strongly.
Thanks!
Simon Born
shared this idea
-
Simon Born
commented
Hey! So I got this answer (not sure why it is not showing up here):
We currently do offer "visitors over time" graph that is shown on the results page for unique count, total count, revenue per user and value per user metric types.
Are there any specific adjustments you have in mind that would help your team to debug SRM issues?Yes, the "visitors over time" usually look s.th. like in the attached screenshot. However, even with experiments with an SRM, these graphs look very much the same, and the difference between visitor counts in A and B usually are not spottable with the naked eye.
So instead of showing the time series of users like this it would be better to show a distribution graph of users where one could identify kinks in the graph and from that know when an SRM hit.
