Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stylix: extract nix-flake-check script #898

Open
danth opened this issue Feb 23, 2025 · 1 comment
Open

stylix: extract nix-flake-check script #898

danth opened this issue Feb 23, 2025 · 1 comment
Assignees
Labels
technical debt Things which should be cleaned up or improved

Comments

@danth
Copy link
Owner

danth commented Feb 23, 2025

This issue refers to the following:

stylix/flake.nix

Lines 171 to 199 in f121a14

nix-flake-check = pkgs.writeShellApplication {
meta.description = "A parallelized alternative to 'nix flake check'";
name = "nix-flake-check";
runtimeInputs = with pkgs; [
nix
jq
parallel
];
text = ''
nix flake show --json --no-update-lock-file ${self} |
jq --raw-output '
((.checks."${system}" // {}) | keys) as $checks |
((.packages."${system}" // {}) | keys) as $packages |
(($checks - $packages)[] | "checks.${system}.\(.)"),
($packages[] | "packages.${system}.\(.)")
' |
parallel \
--bar \
--color \
--color-failed \
--halt now,fail=1 \
--tagstring '{}' \
'
nix build --no-update-lock-file --print-build-logs \
${self}#{}
'
'';

  • The script tries to improve the performance of the regular nix flake check command. I believe we shouldn't ship a replacement for a built in command just to improve its performance, as this should be left for upstream developers to improve, or for users to select their own preferred alternative.
  • Compared to the regular command, the script adds the feature of building everything under packages in addition to checks. This is unnecessary for Stylix, as we export our packages as checks too:
    checks = lib.attrsets.unionOfDisjoint {
  • If ${self} was a command line argument, it would be possible to use the script for any flake.
  • When the script is built in our CI, the name of the job implies that we're running nix flake check, not just building a script which does so.

Given the above points, I feel that the script would be better off as a separate repository (or Gist given its simplicity).

This doesn't mean it can't be a thing we suggest in comments and such, it just doesn't make sense for it to be shipped as part of Stylix per se.

CC @trueNAHO, as it looks like you're the original author

@danth danth added the technical debt Things which should be cleaned up or improved label Feb 23, 2025
@trueNAHO
Copy link
Collaborator

  • The script tries to improve the performance of the regular nix flake check command. I believe we shouldn't ship a replacement for a built in command just to improve its performance, as this should be left for upstream developers to improve, or for users to select their own preferred alternative.

The nix-flake-check script was added under the assumption that it will take upstream a long time before nix flake check is fully parallelized:

stylix: add nix-flake-check package

Add the nix-flake-check package, which is a parallelized alternative to
'nix flake check', as it is not yet natively parallel:

> In the near future, we will make more Nix subcommands multi-threaded,
> such as 'nix flake check'.
>
> -- Eelco Dolstra, https://determinate.systems/posts/parallel-nix-eval

On a 16-threaded machine, 'nix run .#nix-flake-check' runs three times
faster (74s vs. 243s) than 'nix flake check'.

Link: https://github.com/danth/stylix/pull/519

-- a083892 ("stylix: add nix-flake-check package")

  • Compared to the regular command, the script adds the feature of building everything under packages in addition to checks. This is unnecessary for Stylix, as we export our packages as checks too

Only checks that are not already in packages are added, meaning that checks derivations are not duplicated:

stylix/flake.nix

Lines 186 to 187 in f121a14

(($checks - $packages)[] | "checks.${system}.\(.)"),
($packages[] | "packages.${system}.\(.)")

IIRC, I prioritized packages over checks because their evaluation was better cached.

  • When the script is built in our CI, the name of the job implies that we're running nix flake check, not just building a script which does so.

I am open to renaming the package.

  • If ${self} was a command line argument, it would be possible to use the script for any flake.

[...]

Given the above points, I feel that the script would be better off as a separate repository (or Gist given its simplicity).

Considering how small the script is, it should not be a problem to keep the implementation in-tree.

This doesn't mean it can't be a thing we suggest in comments and such, it just doesn't make sense for it to be shipped as part of Stylix per se.

Considering how long nix flake check takes in our repository and therefore how frequently is may be used, keeping this package makes it far more convenient to use than copy-pasting the command.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
technical debt Things which should be cleaned up or improved
Projects
None yet
Development

No branches or pull requests

2 participants