weldr api delayed response after first /compose call

[jkozol]

As part of releasing cockpit-composer to f35/36 it became apparent that the api calls following the first /compose call are delayed. After the POST /compose call there is a long delay and then next expected calls/response suddenly all submit (We receive multiple status responses at once).

[bcl]

Ah, is this the first request sent after rebooting the system? If that's the case it is probably downloading new dnf metadata. We've discussed ways to fix that in #2887 (basically do a depsolve on startup to preload the metadata cache).

[achilleas-k]

The startup metadata preload added in #2941 has a little nasty side-effect: because it uses the same base solver to depsolve for all distros, they all share the same top-level cache, which dnf locks, so they're essentially running in order.

This isn't an issue when no depsolve is needed immediately after starting composer but, if it is, it will block waiting for the preload, which can take minutes.

We could create a separate solver for each distro in the registry, with its own cache as a subdirectory of the main cache dir. I'm not sure what the memory load of that would be, given how much we cache in memory for each solver instance. It might also be a bit wasteful depending on how much users do cross-distro builds.

Maybe we can instead only preload the host distro?

[bcl]

See #3317 for a possible solution to this.