bundler snafu

[hsitter]

Bundler 0.16 screws up deployment

Why?

Depends on the specific calls involved. Generally speaking all our deployment is split into an install portion (on the host), a package portion (on the host) vendoring the relevant gems, and other install operations inside containers which installs the gems in Gemfile.lock from the packaged gems.

Tooling (in particular on slaves) is deployed using calls similar to bundle install --jobs=nproc --no-cache --local --frozen --system --without development test. This command installs all gems excluding the groups we do not need for use at runtime. This in turn creates a Gemfile.lock with only a subset of all possible gems listed (compared to simply running bundle install, which would install everything).

For deployment in containers we'll then bundle package the relevant gems into the working directory to eliminate duplicative network IO across the numerous containers we build concurrently.

At this point the problem appears. We now have a Gemfile with (e.g.) 100 gems it wants, and a Gemfile.lock which only specifices (e.g.) 50 gems out of the 100. A bug inside bundler makes various commands (foremost bundler package) want to resolve gems which are not in the set of the (e.g.) 50 locked ones which causes error like these

Some gems seem to be missing from your vendor/cache directory.
Could not find vcr-4.0.0 in any of the sources

Because we do not install the development test groups the vcr gem was not bundler packaged, yet bundler wants it to be there making the deployment fail.

The cause is probably (as in: AFAIK no one has verified that this is in fact the cause):
rubygems/bundler#6074