During my senior spring semester, Harvard's COVID 19 testing schedule required each student to complete a COVID test kit from Color, as well as fill out a lengthy and tedious online form every single time. Early that semester, I built and launched autocolor — a service that automated the electronic parts of this testing process. It operated entirely over Twilio SMS — users would onboard by signing up and sending their credentials to my backend, which would then use them to sign into Color's website and complete their questionnaires on demand.
Within the first two weeks of launch, I had 160 users signed up from word of mouth, and they had done 731 successful activations using my service.
By the time I shut down the service at the end of the semester, I had completed over 1700 activations for 177 users. (Harvard dramatically relaxed its testing frequency requirements over the course of the semester, and eventually dropped them altogether, so the rate of activations dropped sharply over time.)
After doing some tests with my friends, having them race to complete the questionnaire as quickly as humanly possible and comparing it to autocolor's times, I estimate that I saved Harvard undergraduates, grad students, and faculty about 40 hours over the course of the Spring 2022 semester.
The questionnaire in the COVID test kits was demographic information that would almost never change (ethnicity, date of birth, etc.) and it contained some silly combinations of questions (like asking if you were male, and then on a later screen whether you were pregnant). Critically, though, the form auto-populated any previous answers, which meant that knowing a user's credentials was enough to fill out the questionnaire on their behalf, since it simply required logging into Color's (very slow) site and clicking through the pages of questions.
The final step of the form was to manually type in two numbers that were printed on the Color test kit with a barcode, like so:
(My camera roll from that time is littered with test images of these kits.)
The tedium of going through this process for each test, and the added burden of having to manually transcribe a barcode on a mobile device with a camera (!), was too much to bear, leading to the development of autocolor.
I had initially built a computer vision pipeline that extracted the numbers and barcodes from an image that the user texted to my Twilio number, but the night before I was initially about to launch the initial version of autocolor, Color themselves added an API endpoint that would do the extraction. Theirs was a bit more robust (and CV pipeline was expensive to run on my server!), so I quickly swapped over to their endpoint.
The actual backend itself (code in /backend) ran spread over three Docker images:
- a Postgres database of credentials and activations
- one image (which could run multiple copies in parallel) with a puppeteer instance that would navigate the Color website, log in, fill out the survey, extract the barcodes, and submit everything
- finally, a self-upgrading backend written in Go that connected to Twilio, launched the Puppeteer instances, and texted me with any errors or feedback that users sent to the bot
After a simple onboarding flow (I couldn't find any screenshots, sorry!), you could simply text a picture of a barcode to the server, and it would respond back once your activation was confirmed:
If something failed, it would log an error in the database and respond appropriately, along with a message to reach out to me:
Separately, the backend would text me whenever someone texted the bot something that wasn't an image or an expected component of the onboarding flow. This let me collect feedback from users without them necessarily reaching out to me directly (see the last text in the above screenshot).
Similarly, whenever the server restarted, or deployed a new version of the backend, I'd get a text message notifying me.
Building something people used and loved was so rewarding to me — I decided to keep some of the feedback I got and save it.
When I shut down the service before graduating, I sent out a text blast to anyone who had used autocolor (also called SUPERFAST at the time) encouraging them to reach out if they liked the bot. I actually made a couple of friends that way!
I also asked for a couple bucks to help offset the server costs if people liked the service and wanted to buy me a coffee or a beer.
(For clarification, students no longer had to take COVID tests after testing positive once and recovering.)
In the end, I got a lot more money for beer than for coffee. I drank both :)