Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize the build time by doing manual docker image builds #4

Open
elprans opened this issue May 1, 2020 · 4 comments · Fixed by edgedb/edgedb#1518
Open

Optimize the build time by doing manual docker image builds #4

elprans opened this issue May 1, 2020 · 4 comments · Fixed by edgedb/edgedb#1518
Assignees

Comments

@elprans
Copy link
Member

elprans commented May 1, 2020

Github does not support expressions in the Docker URI, so we explicitly unroll the build matrix into a bunch of steps gated by a conditional expression instead. This works, but the downside is that Github builds all docker images in every matrix iteration, effectively building images n^2 times. All that without using any build cache. The result is that nightly builds easily spend more time in preparing the build images than actually building the packages.

This can be fixed by pre-building all necessary docker images in a separate workflow step, preferably using some sort of build cache (see actions/cached-docker-build-push). The build steps would then use pushed images.

@ambv
Copy link
Contributor

ambv commented Jul 8, 2020

As discussed, we will rather drop usage of GHA's matrix and treat all specific Linux distro builds as separate jobs, just as macOS is a separate job.

ambv added a commit to ambv/edgedb that referenced this issue Jul 8, 2020
Use Jinja for the same purpose. This allows for different Linux distributions
to independently publish their artifacts when their own build and test was
green.

It also speeds up execution by not building all Docker images on every matrix
iteration.

Fixes edgedb/edgedb-pkg#4
@ambv ambv reopened this Jul 8, 2020
@ambv
Copy link
Contributor

ambv commented Jul 8, 2020

Let's keep this open until we evaluate caching.

@fmoor fmoor closed this as completed Apr 5, 2021
@tailhook
Copy link
Contributor

tailhook commented Apr 6, 2021

Looks like this was accidentally closed.

@tailhook tailhook reopened this Apr 6, 2021
@elprans elprans assigned RobertBlumen and unassigned ambv Sep 8, 2023
@elprans
Copy link
Member Author

elprans commented Sep 8, 2023

The strategy here should be to implement an action that automatically builds all images and pushes them to this repository's container registry. Consuming workflows would then refer to the ghcr.io image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants