Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build(nix): create si-fs-standalone Nix pkg for a Nix-free binary #5472

Merged
merged 4 commits into from
Feb 13, 2025
Merged

build(nix): create si-fs-standalone Nix pkg for a Nix-free binary #5472

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bc48c1d
style(fmt): format `flake.nix` via alejandra
fnichol Feb 12, 2025
cf90ad5
build(si-fs): remove unsed imports from `BUCK` file
fnichol Feb 12, 2025
a00c5fb
style(fmt): wrap long lines in `flake.nix`
fnichol Feb 13, 2025
74e5de8
build(nix): create `si-fs-standalone` Nix pkg for a Nix-free binary
fnichol Feb 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions bin/si-fs/BUCK
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
load(
"@prelude-si//:macros.bzl",
"docker_image",
"nix_omnibus_pkg",
"rust_binary",
)

Expand Down
80 changes: 67 additions & 13 deletions flake.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
};

outputs = {
self,
nixpkgs,
flake-utils,
rust-overlay,
Expand Down Expand Up @@ -109,18 +108,60 @@
flyctl
azure-cli
govc
(pkgs.python3.withPackages (p: with p; [
cfn-lint
]))
(pkgs.python3.withPackages (p:
with p; [
cfn-lint
]))
deno
];

standaloneBinaryDerivation = {
pkgName,
fromPkg,
bin ? pkgs.lib.strings.removeSuffix "-standalone" pkgName,
}:
pkgs.stdenv.mkDerivation {
name = pkgName;
__impure = true;
src = ./.;
buildInputs = [fromPkg];
installPhase = ''
install -Dv "${fromPkg}/bin/${bin}" "$out/bin/${bin}"
'';
postFixup =
""
+ pkgs.lib.optionalString (pkgs.stdenv.isDarwin) ''
nix_lib="$(otool -L "$out/bin/$name" \
| grep libiconv.dylib \
| awk '{print $1}'
)"
install_name_tool \
-change \
"$nix_lib" \
/usr/lib/libiconv.2.dylib \
"$out/bin/$name" \
2>/dev/null
''
+ pkgs.lib.optionalString (pkgs.stdenv.isLinux) ''
patchelf \
--set-interpreter "${systemInterpreter}" \
--remove-rpath \
"$out/bin/${bin}"
'';
dontPatchELF = true;
dontAutoPatchELF = true;
Comment on lines +131 to +152
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify my understanding, the idea here is to undo the autopatching done in the binDerivation to just use the system's libc bits, yeah? Out of curiostiy, why do we need a separate derivation for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Essentially yeah. The default in Nix is to use patchelf to hardcode the linker path, set RPATH (i.e. where to look for libs), etc., and that's what we want to un-do. At some level I need a way to specify that I only want certain package targets to mangle themselves like this, not all packages (as we are relying on properly built Nix packages in our omnibus packages). At the moment this other derivation is using the output of a prior built package, but I may change this in the future to build is entirely (i.e. perform the buck2 build) but may play with which libs are present at build time (that is, can we provide an older Glibc at build time to help with backwards compat, etc).

And how did I arrive here? I partially resurrected the tricks and strategies used for the old launcher (

si/flake.nix

Lines 241 to 286 in 5534c19

# Builds a standalone binary that is divorced from the Nix build and
# runtime environment
si-binary = buck2Derivation {
pathPrefix = "bin";
pkgName = "si";
stdBuildPhase = ''
# TODO(fnichol): we currently have a dynamically linked library of
# libc++abi being looked for in a `/nix/store` path when this
# binary is compiled on macOS, so for the meantime until this is
# fixed, we're going to perform a Cargo build of the binary (it
# does not suffer from this issue).
cargo build --bin "$name" --release
cp -pv "target/release/$name" "build/$name-$system"
'';
installPhase = ''
mkdir -pv "$out/bin"
cp -pv "build/$name-$system" "$out/bin/$name"
'';
dontPatchELF = true;
dontAutoPatchELF = true;
postFixup =
""
+ pkgs.lib.optionalString (pkgs.stdenv.isDarwin) ''
nix_lib="$(otool -L "$out/bin/$name" \
| grep libiconv.dylib \
| awk '{print $1}'
)"
install_name_tool \
-change \
"$nix_lib" \
/usr/lib/libiconv.2.dylib \
"$out/bin/$name" \
2>/dev/null
''
+ pkgs.lib.optionalString (pkgs.stdenv.isLinux) ''
patchelf \
--set-interpreter "${systemInterpreter}" \
--remove-rpath \
"$out/bin/$name"
''
+ ''
mkdir -pv "$out/tarballs"
tar cvf "$out/tarballs/$name-$system.tar" -C "$out/bin" "$name"
gzip -9 "$out/tarballs/$name-$system.tar"
'';
};
)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, okay. Totally makes sense.

};

buck2Derivation = {
pathPrefix,
pkgName,
extraBuildInputs ? [],
stdBuildPhase ? ''
buck2 build @//mode/release "$buck2_target" --verbose 8 --out "build/$name-$system"
buck2 build \
@//mode/release \
"$buck2_target" \
--verbose 8 \
--out "build/$name-$system"
'',
extraBuildPhase ? "",
installPhase,
Expand All @@ -129,7 +170,7 @@
dontAutoPatchELF ? false,
postFixup ? "",
}:
pkgs.stdenv.mkDerivation rec {
pkgs.stdenv.mkDerivation {
name = pkgName;
buck2_target = "//${pathPrefix}/${pkgName}";
__impure = true;
Expand All @@ -156,7 +197,8 @@
'';
configurePhase = ''
export LIBCLANG_PATH="${pkgs.llvmPackages.libclang.lib}/lib"
export BINDGEN_EXTRA_CLANG_ARGS="$(< ${pkgs.stdenv.cc}/nix-support/libc-crt1-cflags) \
export BINDGEN_EXTRA_CLANG_ARGS="\
$(< ${pkgs.stdenv.cc}/nix-support/libc-crt1-cflags) \
$(< ${pkgs.stdenv.cc}/nix-support/libc-cflags) \
$(< ${pkgs.stdenv.cc}/nix-support/cc-cflags) \
$(< ${pkgs.stdenv.cc}/nix-support/libcxx-cxxflags) \
Expand Down Expand Up @@ -258,14 +300,18 @@
# is where we expect it. This gets droppped into the rootfs as
# /lib64/ld-linux-x86-64.so.2 -> /nix/store/*/ld-linux-x86-64.20.2
mkdir -p $out/lib64
ln -sf ${pkgs.glibc}/lib/ld-linux-x86-64.so.2 $out/lib64/ld-linux-x86-64.so.2
ln -sf ${pkgs.glibc}/lib/ld-linux-aarch64.so.1 $out/lib64/ld-linux-aarch64.so.1
ln -sf \
"${pkgs.glibc}/lib/ld-linux-x86-64.so.2" \
"$out/lib64/ld-linux-x86-64.so.2"
ln -sf \
"${pkgs.glibc}/lib/ld-linux-aarch64.so.1" \
"$out/lib64/ld-linux-aarch64.so.1"

wrapProgram $out/bin/lang-js \
--set LD_LIBRARY_PATH "${pkgs.lib.makeLibraryPath [
pkgs.glibc
pkgs.gcc-unwrapped.lib
]}" \
pkgs.glibc
pkgs.gcc-unwrapped.lib
]}" \
--set DENO_DIR "$out/cache" \
--prefix PATH : ${pkgs.lib.makeBinPath langJsExtraPkgs} \
--run 'cd "$(dirname "$0")"'
Expand All @@ -284,6 +330,13 @@

sdf = binDerivation {pkgName = "sdf";};

si-fs = binDerivation {pkgName = "si-fs";};

si-fs-standalone = standaloneBinaryDerivation {
pkgName = "si-fs-standalone";
fromPkg = packages.si-fs;
};

veritech = binDerivation {pkgName = "veritech";};

web = appDerivation rec {
Expand Down Expand Up @@ -312,7 +365,8 @@
# Env Vars so bindgen can find libclang
shellHook = ''
export LIBCLANG_PATH="${pkgs.llvmPackages.libclang.lib}/lib"
export BINDGEN_EXTRA_CLANG_ARGS="$(< ${pkgs.stdenv.cc}/nix-support/libc-crt1-cflags) \
export BINDGEN_EXTRA_CLANG_ARGS="\
$(< ${pkgs.stdenv.cc}/nix-support/libc-crt1-cflags) \
$(< ${pkgs.stdenv.cc}/nix-support/libc-cflags) \
$(< ${pkgs.stdenv.cc}/nix-support/cc-cflags) \
$(< ${pkgs.stdenv.cc}/nix-support/libcxx-cxxflags) \
Expand Down