Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

webconnectivity LTE: A/B testing using richer input #2555

Closed
bassosimone opened this issue Oct 10, 2023 · 2 comments
Closed

webconnectivity LTE: A/B testing using richer input #2555

bassosimone opened this issue Oct 10, 2023 · 2 comments
Assignees
@bassosimone
Labels
funder/drl2022-2024 ooni/probe-engine priority/high Important issue that needs attention soon research prototype technical task technical tasks e.g. deployment testing

Comments

@bassosimone
Copy link
Contributor

This issue is about performing A/B testing between webconnectivity v0.4 and LTE using richer input. We have already this functionality in tree since ooni/probe-cli#1123. This issue is about using the check-in API to perform the actual A/B testing. We would collect data for either a selected subset of users or time. The goal would be to figure out the differences between these two versions, which then would inform which extra QA test to write and bugs to fix.
@bassosimone bassosimone self-assigned this Oct 10, 2023
@bassosimone bassosimone mentioned this issue Oct 19, 2023
Closed
@bassosimone bassosimone added priority/high Important issue that needs attention soon priority/medium Normal priority issue and removed priority/medium Normal priority issue priority/high Important issue that needs attention soon labels Nov 21, 2023
@bassosimone bassosimone mentioned this issue Nov 21, 2023
Closed
@bassosimone
Copy link
Contributor Author

We're tentatively going to run this A/B testing around the next week for ooniprobe-android-unattended 3.8.6 only and take advantage of the staged rollout to observe changes. We're going to limit ourselves to such a specific version because it contains ooni/[email protected], which ensures v0.4 and v0.5 generate the same results for our QA testing suite.

bassosimone added a commit to ooni/backend that referenced this issue Jan 23, 2024
@bassosimone
feat(api): prepare for WebConnectivity A/B testing
925667c 
See ooni/probe#2555
@bassosimone bassosimone mentioned this issue Jan 23, 2024
Open
@bassosimone bassosimone added priority/high Important issue that needs attention soon and removed priority/medium Normal priority issue labels Jan 29, 2024
@bassosimone bassosimone mentioned this issue May 20, 2024
Open
@hellais hellais added this to Roadmap Jan 7, 2025
@hellais
Copy link
Member

hellais commented Jan 23, 2025

We have dropped the richer input plan in favour of shooting for ooni/backend#927

@hellais hellais closed this as completed Jan 23, 2025
@github-project-automation github-project-automation bot moved this to Done in Roadmap Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bassosimone bassosimone

Labels
funder/drl2022-2024 ooni/probe-engine priority/high Important issue that needs attention soon research prototype technical task technical tasks e.g. deployment testing
Projects
Roadmap
Archived in project
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bassosimone @hellais