You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Q: How to get the bucket of combined_experiments ? #500
After you run ab_combined_test how can you know whether you are in the blue or green bucket (and decide what color to actually draw the button) ?
In this snippet, thing gets the value of ["button_color_on_signup", "button_color_on_login"] , which is actually the result of the each method here
Unfortunately from my usage, it doesn't look like ab_combined_test works properly. Using ab_combined_test locally will generate the first combined experiment in the combined_experiments array properly but will generate new versions of subsequent experiments for each participant.
So in your example, you'd get a consistent test for button_color_on_signup but each participant in button_color_on_login would cause the version for that test to bump.
It isn't immediately clear to me and it is certainly possible I'm doing something wrong but I don't see what that would be.
Maybe @daviddening has some thoughts on the functionality since he originally wrote it?
Aha. the issue is line 18 above. Only the selected alternative is passed in (with weight of 1) but since not all alternatives are represented, it appears to be a new test and the version is bumped.
Activity
semanticart commentedon Sep 19, 2017
It looks like this should be just a matter of returning
alternativeafter this block:split/lib/split/combined_experiments_helper.rb
Lines 9 to 20 in 87fbd83
That + a test is a pretty quick PR.
Unfortunately from my usage, it doesn't look like
ab_combined_testworks properly. Usingab_combined_testlocally will generate the first combined experiment in thecombined_experimentsarray properly but will generate new versions of subsequent experiments for each participant.So in your example, you'd get a consistent test for
button_color_on_signupbut each participant inbutton_color_on_loginwould cause the version for that test to bump.It isn't immediately clear to me and it is certainly possible I'm doing something wrong but I don't see what that would be.
Maybe @daviddening has some thoughts on the functionality since he originally wrote it?
semanticart commentedon Sep 19, 2017
Aha. the issue is line 18 above. Only the selected alternative is passed in (with weight of 1) but since not all alternatives are represented, it appears to be a new test and the version is bumped.
I'll try to get a PR together to fix this.
Fix combined experiments (#502)