Any plans on dockerizing this project?

[rvalenciano]

Hey 👋

Very cool idea and project, and useful. I was wondering if there are any plans on dockerizing this, as testing with different images would be easier, and also we could share resources easily between benchmarks. I also see some open issues regarding installation that might benefit from it.

Best!

[noahgibbs]

I've looked at it a bit. Docker has some good points - reproducible installs, mainly. It also has some difficult points: the performance numbers are significantly affected by it, which means it's not measuring how most people actually deploy Rails apps.

It actually makes it harder to configure most one-off benchmarks since you have to write a Docker file to make changes in an automated way instead of just making the change once (e.g. new Ruby version - okay, but now you're maintaining an install script for that Ruby.) It's the classic problem with automation, where doing it once gets harder. So for the articles I was writing for AppFolio, Docker would have made all of that more difficult and a lot less would have happened.

Docker would probably be the right choice going forward since I'm now putting a lot less effort into RRB. And Docker would be a lower maintenance burden for a benchmark that would no longer be used much for interesting new results. But Docker is a substantial rewrite of the existing Packer infrastructure, and I'm currently putting a lot less effort into RRB.