Move images used in tests off of Docker Hub

[adamnovak]

Docker Hub no longer allows the images used in the conformance tests to be fetched an arbitrary number of times by our CI for Toil. This causes conformance tests to fail because they need to fetch copies of the images and are denied access by Docker Hub. THis is especially a problem for Toil because it can in the worst case fetch a copy of each image every time it executes a CWL job that uses the image.

The images used in the tests should be hosted somewhere where they are freely accessible.

• dockerPull: docker.io/alpine:latest
• dockerPull: docker.io/bash:4.4
• dockerPull: docker.io/debian:stable-slim
• dockerPull: docker.io/python:3-slim

[adamnovak]

A workaround on the toil-cwl-runner/cwltool side might be to add --docker-login support to the singularity build calls that cwltool knows how to make, and just say that anyone who wants to run CI on a CWL runner needs to pay Docker Hub to get at the images the tests need.

[mr-c]

@adamnovak Can you try the following for the toil-cwl-runner CI?

singularity remote login --username [email protected] --password FOO docker://docker.io
(or use --password-stdin instead of --password)

I did look at moving to quay.io hosted images, but I had difficulties finding equivalents

[mr-c]

I added a list of the 4 containers we use to the original comment.

[adamnovak]

@mr-c We're looking at getting a Docker Hub account we can use on CI for Toil.

Is there a place in cwltool we would hook in to get it to issue that command before it issues other Singularity commands? Probably cwltool expects Singularity to just already be logged in, and so Toil would be responsible for doing the login for each job.

Those particular images all look to be library images, though, so they should be mirrored on Google's mirror.gcr.io which doesn't limit pulls:

mirror.gcr.io/library/alpine:latest
mirror.gcr.io/library/bash:4.4
mirror.gcr.io/library/debian:stable-slim
mirror.gcr.io/library/python:3-slim

[adamnovak]

Looks like bash:4.4 isn't on the mirror, but bash:latest is. Those would all be drop-in replacements.

[mr-c]

Hey @adamnovak

I agree that finding a free resource would be better. I'm not against using these mirror URLs, but I'm having trouble finding public documentation or an commitments from Google. While the hashes match docker.io/library, it would be nice to know who is providing these images.

Anyhow, feel free to send a PR against https://github.com/common-workflow-language/cwl-v1.2/tree/main and https://github.com/common-workflow-language/cwl-v1.2/tree/1.2.1_proposed

[adamnovak]

This has been less of a problem for the Toil team lately, since our cache has been working properly.