Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/boulder #87

Merged
merged 43 commits into from
Dec 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
222fdb6
Move config to new crate and enable user scope
tarkah Nov 21, 2023
088081c
Add configurable boulder dirs
tarkah Nov 22, 2023
7437dd1
Add boulder cli entry
tarkah Nov 22, 2023
124ce77
Add profile config
tarkah Nov 22, 2023
2917363
Enable explicit repo configs w/ moss
tarkah Nov 22, 2023
cc4320e
Drop async runtime before entering container
tarkah Nov 27, 2023
413aefd
Rename vars
tarkah Nov 27, 2023
9c805e7
Add cache module & refactor away client
tarkah Nov 28, 2023
75bc38f
Add container bind mounts & options & refactor
tarkah Nov 28, 2023
4d397c5
Refactor dependency calculation
tarkah Nov 28, 2023
ec7362c
Fetch upstreams
tarkah Nov 29, 2023
453ddf4
Remove runtime internally by converting to async functions
tarkah Nov 29, 2023
3180e59
Add indicatif progress to upstream sync
tarkah Nov 30, 2023
9a23977
Deser stringy bool
tarkah Nov 30, 2023
f369ef7
Mount recipe and allow ro mounts
tarkah Nov 30, 2023
a81bf50
Rename to job and refactor
tarkah Nov 30, 2023
dcc9af3
More refactoring
tarkah Nov 30, 2023
43c3b7b
Add macros to stone_recipe
tarkah Nov 30, 2023
52d51ea
Load macros from data dir
tarkah Nov 30, 2023
4bead2b
Add initial script parsing
tarkah Dec 1, 2023
b9af7c3
Add Script and Parser
tarkah Dec 1, 2023
98df246
Support adding entire Macros type
tarkah Dec 1, 2023
452d8e2
Build scripts and exec them (very much WIP)
tarkah Dec 1, 2023
4f81b58
Use builddir as workdir in prepare step
tarkah Dec 4, 2023
c60922c
Prevent deadlock
tarkah Dec 4, 2023
fb2bd33
Unpack into named dir
tarkah Dec 4, 2023
168e300
Set jobs to num_cpus
tarkah Dec 4, 2023
f577cd3
Remove deprecated commands from meson install macro
tarkah Dec 4, 2023
12da279
Fix unpack & rename logic
tarkah Dec 4, 2023
424a727
Prepare git upstream
tarkah Dec 4, 2023
75e5f2c
Fix extra whitespace in logging
tarkah Dec 4, 2023
b43b10b
Add Builder and job for each build target
tarkah Dec 5, 2023
d20fbce
Handle pgo
tarkah Dec 6, 2023
6a1367f
Update log format
tarkah Dec 6, 2023
9e395ff
Refactor pgo away from job step
tarkah Dec 6, 2023
89102d1
Rename method
tarkah Dec 6, 2023
2b78444
Add tuning flags
tarkah Dec 6, 2023
b11f48e
Better container logging & sigint capture
tarkah Dec 7, 2023
76f3cc8
Set umask to 022
tarkah Dec 8, 2023
5554320
More robust idmap for rootless container
tarkah Dec 8, 2023
e0ae246
Update logging output, thanks @ermo
tarkah Dec 8, 2023
de2a710
Resetup signal handler when leaving container
tarkah Dec 11, 2023
56f99de
Recreate work_dir for each new pgo stage
tarkah Dec 11, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,14 @@ bytes = "1.5.0"
chrono = "0.4.30"
clap = { version = "4.4.10", features = ["derive"] }
crossterm = "0.27.0"
dirs = "5.0"
indicatif = "0.17.7"
itertools = "0.11.0"
futures = "0.3.28"
hex = "0.4.3"
log = "0.4"
nix = { version = "0.27.1", features = ["user", "fs", "sched", "process", "mount", "hostname"] }
nom = "7.1.3"
nix = { version = "0.27.1", features = ["user", "fs", "sched", "process", "mount", "hostname", "signal"] }
once_cell = "1.18.0"
petgraph = "0.6.4"
rayon = "1.8"
Expand All @@ -30,6 +32,7 @@ serde = { version = "1", features = ["derive"] }
serde_yaml = "0.9"
sha2 = "0.10.8"
sqlx = { version = "0.7.3", features = ["sqlite", "chrono", "runtime-tokio"] }
strum = { version = "0.25", features = ["derive"] }
thiserror = "1"
tokio = { version = "1.34", features = ["full"] }
tokio-stream = { version = "0.1.14", features = ["time"] }
Expand Down
13 changes: 13 additions & 0 deletions crates/boulder/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,21 @@ edition.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
config = { path = "../config" }
container = { path = "../container" }
moss = { path = "../moss" }
stone_recipe = { path = "../stone_recipe" }
tui = { path = "../tui" }

clap.workspace = true
dirs.workspace = true
futures.workspace = true
hex.workspace = true
itertools.workspace = true
nix.workspace = true
serde.workspace = true
sha2.workspace = true
strum.workspace = true
thiserror.workspace = true
tokio.workspace = true
url.workspace = true
62 changes: 62 additions & 0 deletions crates/boulder/data/macros/actions/autotools.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
actions:

# Perform ./configure with the default options
- configure:
command: |
test -x ./configure || ( echo "%%configure: The ./configure script could not be found" ; exit 1 )
./configure %(options_configure)
dependencies:
- autoconf
- automake

# Perform a make
# TIP: Add V=1 VERBOSE=1 after '%make' in the recipe if you need a more verbose build
- make:
command: |
make -j "%(jobs)"
dependencies:
- make

# Install results of build to the destination directory
- make_install:
command: |
%make install DESTDIR="%(installroot)"
dependencies:
- make

# Re autotools-configure a project without an autogen.sh script
- reconfigure:
command: |
autoreconf -vfi || ( echo "%%reconfigure: Failed to run autoreconf"; exit 1 )
%configure
dependencies:
- autoconf
- automake

# Run autogen.sh script, attempting to only configure once
- autogen:
command: |
NOCONFIGURE="noconfigure"; export NOCONFIGURE
sh ./autogen.sh %(options_configure)
./configure %(options_configure)
dependencies:
- autoconf
- automake

definitions:

# Default configuration options as passed to configure
- options_configure: |
--prefix="%(prefix)" \
--bindir="%(bindir)" \
--sbindir="%(sbindir)" \
--build="%(build_platform)" \
--host="%(host_platform)" \
--libdir="%(libdir)" \
--mandir="%(mandir)" \
--infodir="%(infodir)" \
--datadir="%(datadir)" \
--sysconfdir="%(sysconfdir)" \
--localstatedir="%(localstatedir)" \
--sharedstatedir="%(sharedstatedir)" \
--libexecdir="%(libexecdir)"
44 changes: 44 additions & 0 deletions crates/boulder/data/macros/actions/cmake.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
actions:

# Perform cmake with the default options in a subdirectory
- cmake:
command: |
cmake %(options_cmake)
dependencies:
- cmake

# Perform cmake with unity build enabled
- cmake_unity:
command: |
cmake -DCMAKE_UNITY_BUILD=ON %(options_cmake)
dependencies:
- cmake

# Build the cmake project
- cmake_build:
command: |
ninja -v -j "%(jobs)" -C "%(builddir)"
dependencies:
- ninja

# Install results of the build to the destination directory
- cmake_install:
command: |
DESTDIR="%(installroot)" ninja install -v -j "%(jobs)" -C "%(builddir)"
dependencies:
- ninja

definitions:

# Default cmake options as passed to cmake
- options_cmake: |
-G Ninja -S . -B "%(builddir)" \
-DCMAKE_C_FLAGS="${CFLAGS}" \
-DCMAKE_CXX_FLAGS="${CXXFLAGS}" \
-DCMAKE_C_FLAGS_RELEASE="" \
-DCMAKE_CXX_FLAGS_RELEASE="" \
-DCMAKE_LD_FLAGS="${LDFLAGS}" \
-DCMAKE_BUILD_TYPE="Release" \
-DCMAKE_INSTALL_LIBDIR="lib" \
-DCMAKE_INSTALL_PREFIX="%(prefix)" \
-DCMAKE_LIB_SUFFIX="%(libsuffix)"
43 changes: 43 additions & 0 deletions crates/boulder/data/macros/actions/meson.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
actions:

# Run meson with the default options in a subdirectory
- meson:
command: |
test -e ./meson.build || ( echo "%%meson: The ./meson.build script could not be found" ; exit 1 )
CFLAGS="${CFLAGS}" CXXFLAGS="${CXXFLAGS}" LDFLAGS="${LDFLAGS}" meson setup %(options_meson)
dependencies:
- meson

# Run meson with unity build enabled
- meson_unity:
command: |
test -e ./meson.build || ( echo "%%meson: The ./meson.build script could not be found" ; exit 1 )
CFLAGS="${CFLAGS}" CXXFLAGS="${CXXFLAGS}" LDFLAGS="${LDFLAGS}" meson setup --unity on %(options_meson)
dependencies:
- meson

# Build the meson project
- meson_build:
command: |
meson compile -v -j "%(jobs)" -C "%(builddir)"
dependencies:
- meson

# Install results of the build to the destination directory
- meson_install:
command: |
DESTDIR="%(installroot)" meson install --no-rebuild -C "%(builddir)"
dependencies:
- meson

definitions:

# Default meson options as passed to meson
- options_meson: |
--prefix="%(prefix)" \
--buildtype="plain" \
--libdir="lib%(libsuffix)" \
--libexecdir="lib%(libsuffix)/%(name)" \
--sysconfdir="%(sysconfdir)" \
--localstatedir="%(localstatedir)" \
"%(builddir)"
Loading