just adopt a task runner

.github/workflows/format-check.yml

@@ -9,6 +9,9 @@ jobs:
   clang-format:
     runs-on: ubuntu-latest
     steps:
+      - uses: extractions/setup-just@v2
+        with:
+          just-version: 1.26.0
       - uses: actions/checkout@v4
-      - name: run format check
-        run: ./.github/format-check
+      - name: run format check on the C++
+        run: just install-denv init format-cpp --verbose -Werror --dry-run

README.md

@@ -16,50 +16,102 @@
     <img src="https://github.com/LDMX-Software/ldmx-sw/actions/workflows/basic_test.yml/badge.svg" />
 </p>
 
-## Quick Start
+## Start Up
+ldmx-sw is a large software project and so it is helpful to separate _using_ it to
+perform physics studies from _developing_ it to fix/improve/enable other studies.
+In both cases, we use containers to share a fixed software environment, so everyone
+will need a method for running these containers.
 
 - [Install the docker engine](https://docs.docker.com/engine/install/)
-- (on Linux systems) [Manage docker as non-root user](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user)
-- (on MacOS systems) Make sure `git lfs` is installed. (Test: `git lfs` prints out a help message instead of an error about `lfs` not being found.)
-  - The default installation of `git` that is included with Apple's developer tools does not include `git lfs` which is required by acts to download and unpack one of its own submodules. [GitHub has a nice tutorial](https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage?platform=mac) on how to install `git lfs` on MacOS.
-  - This is only an issue observed on MacOS systems. Linux repositories (used in WSL and Linux systems) include `git lfs` within their installed versions of `git`.
-- Clone the repo: `git clone --recursive [email protected]:LDMX-Software/ldmx-sw.git`
-  - **Note**: You need to [setup an SSH-key with your GitHub account](https://docs.github.com/en/authentication/connecting-to-github-with-ssh) on the computer you are using.
-- Setup the environment (in bash): `source ldmx-sw/scripts/ldmx-env.sh`
-  - **Note**: If you are working with ldmx-sw at SLAC's SDF, you will need to set the `TMPDIR` environment variable so that program running the container has more than ~5GB of space to write intermediate files. The default temporary space (`/tmp`) is often full of other files already. A decent replacement is `TMPDIR=${SCRATCH}` which gives the program plenty of room for the files it needs to manipulate.
-- Configure and compile: `ldmx compile`
-- Now you can run any processor in _ldmx-sw_ through `ldmx fire myconfig.py`
-- If you are developing and need to recompile and run `ldmx fire`, you can use `ldmx recompFire myconfig.py`
-
-## Documentation 
-The full documentation for **ldmx-sw** is available on [github pages](https://ldmx-software.github.io/).
-A brief description of common commands is given below.
-
-### Common Commands inside Container
-
-Command | Purpose
----|---
-`ldmx cmake ..` | Configure the ldmx-sw build
-`ldmx make` | Compile/build ldmx-sw
-`ldmx make install` | Install ldmx-sw
-`ldmx compile` | Configure and compile ldmx-sw
-`ldmx fire config.py` | Use ldmx-sw application and processors with input python configuration
-`ldmx recompFire config.py` | Recompile and run fire on a config file
-`ldmx python3 analysis.py` | Run python-based analysis
-`ldmx ./bin/mg5_aMC` | Run MadGraph5 inside (ubuntu-based) container
-
-### Other Container Configuration Commands
-
-The environment script defines several other shell commands to help configure and debug the container environment.
-
-- `ldmx list repo` : List the container tags that you could use with the input repository: `dev`, `pro`, or `local`
-- `ldmx use repo tag` : Setup the environment for the container 'ldmx/repo:tag' and pull down the newest version if the repo is remote
-- `ldmx config` : Print out how the container environment is currently configured
-- `ldmx clean all` : Reset environment to a blank state
-
-Use `ldmx help` for a full listing of these commands.
-If we don't define a command outside of the container,
-then the command is given to the container to run inside the current directory.
+  - Only necessary on personal computers. Shared computing clusters should have `apptainer` installed.
+  - (on Linux personal computers) [Manage docker as non-root user](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user)
+- [Install `denv`](https://tomeichlersmith.github.io/denv/getting_started.html#installation)
+```
+curl -s https://raw.githubusercontent.com/tomeichlersmith/denv/main/install | sh
+```
+Some folks may see an error about something not being within your `PATH`,
+you just need to update your shell's configuration to look for `denv` within
+that directory.
+A program being "in your `PATH`" can be checked by making sure your shell
+can find it.
+```
+denv help
+```
+The above should printout a help message instead of a "command not found"
+message.
+
+Additionally, many folks have gotten used to using `ldmx` as the command
+to put programs into the containerized environment in which case you can
+use the following to add this symlink to your `denv` installation.
+(Note: This requires `denv` to be in your `PATH`!).
+```
+ln -s $(which denv) $(dirname $(which denv))/ldmx
+```
+
+### Using
+In order to use ldmx-sw, no more dependencies are required!
+Simply choose the version of ldmx-sw you wish to use with your project.
+```
+mkdir my-project
+cd my-project
+denv init ldmx/pro:v4.0.1 # or some other ldmx-sw version
+```
+And then you can run ldmx-sw with a configuration script of your choice.
+```
+denv fire my-config.py
+```
+More detail on configuration scripts and analyzing the output files
+is given in the first section of the [online manual](ldmx-software.github.io).
+
+### Developing
+For development, we use a few more tools to help track our changes and share commands
+that we use regularly.
+
+> [!WARNING]
+> If you are on Windows, make sure to install these tools _inside_ WSL where `docker`
+> will be run and ldmx-sw will be developed. Since WSL is often a virtual Ubuntu machine,
+> following the instructions for Ubuntu or Linux can be appropriate.
+
+- Make sure `git` is installed.
+  - `git` is a very common tool used by software developers and so it may already be available.
+  - (on MacOS systems) Make sure `git lfs` is installed. (Test: `git lfs` prints out a help message instead of an error about `lfs` not being found.) The default installation of `git` that is included with Apple's developer tools does not include `git lfs` which is required by acts to download and unpack one of its own submodules. [GitHub has a nice tutorial](https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage?platform=mac) on how to install `git lfs` on MacOS.
+- [Install `just`](https://just.systems/man/en/chapter_5.html)
+  - This tool is not required but it is highly encouraged. The recipes we share via the [justfile](justfile) can be run without `just` but are longer to type.
+
+One can install `just` in a similar way to `denv`. Below is an example where the
+destination directory is set to the same one as the default for `denv` (`~/.local/bin`).
+```
+curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh |\
+  bash -s -- --to ~/.local/bin
+```
+Other [package manager options](https://just.systems/man/en/chapter_4.html) are available
+as well.
+You will probably want to make sure `just`'s tab complete is available.
+If you press `just -<Tab><Tab>` and nothing is listed, then the tab complete is
+not present and you must manually install it.
+This can be accomplished by including its completions within your shell's
+configuration script. For example, in `bash`, we would add the following
+to your `~/.bashrc` file.
+```
+eval "$(just --completions bash)"
+```
+If you are not in `bash`, look to your shell's documentation on where to place
+this line. `just` supports many popular shells including `bash`, `zsh`, and `fish`.
+
+With these additional tools, developers can clone the repository and start development.
+```
+git clone --recursive [email protected]:LDMX-Software/ldmx-sw.git
+```
+
+> [!NOTE]
+> You need to [setup an SSH-key with your GitHub account](https://docs.github.com/en/authentication/connecting-to-github-with-ssh) on the computer you are using.
+
+```
+cd ldmx-sw
+just # no arguments prints out the possible options
+just init # initialize a new development environment (once per clone)
+just configure build test # configure ldmx-sw, build it, then test it
+```
 
 ## Maintainer 
 

justfile

@@ -0,0 +1,176 @@
+# Developer Notes
+#   If you are looking at this file there are a few helpful things to note.
+#   - `@` is used to alter what `just` chooses to print.
+#     It can largely be ignored during development and inserted after when tuning the UI.
+#   - Double curly braces `{{...}}` are used for evaluating `just` variables and functions
+#   - By default, these recipes are run from the directory of this file.
+#     This can be changed but is helpful for us in most recipes.
+#
+# other recipe ideas:
+#   - production image building
+#   - format python
+
+help_message := "shared recipes for ldmx-sw development
+
+    Some folks use 'ldmx' as an alias for 'just' in which case you can
+    replace 'just' with 'ldmx' in the examples below.
+
+  USAGE:
+    just <cmd> [arguments...]
+
+  Multiple commands can be provided at once and they will be run in sequence.
+
+    just init configure build test
+
+  COMMANDS:
+"
+
+# inherited from ldmx-env bash functions
+# we could look into removing this and instead having the denv_workspace be
+# the justfile_directory() itself but that is a larger change than introducing just
+# the denv workspace is colloquially known as LDMX_BASE
+
+export LDMX_BASE := parent_directory(justfile_directory())
+
+# tell denv where the workspace is
+# usually, denv deduces where the workspace is by finding the .denv directory,
+# but we want to set where the denv is within the justfile so users could (for example)
+# run their ldmx-sw build from within some other denv by invoking fire from just
+#   just -f path/to/ldmx-sw/justfile fire config.py
+# would run this denv even if there is a denv in the directory where config.py is.
+
+export denv_workspace := LDMX_BASE
+
+# make sure APPTAINER_CACHEDIR is not in the home directory
+# unless the user has already defined it
+#   just 1.15
+
+export APPTAINER_CACHEDIR := env("APPTAINER_CACHEDIR", LDMX_BASE / ".apptainer")
+
+_default:
+    @just --list --justfile {{ justfile() }} --list-heading "{{ help_message }}"
+
+# this install is private since I'd prefer users knowing what tools they are installing;
+
+# however, the CI needs to install denv before it can run any testing
+[private]
+install-denv:
+    curl -s https://raw.githubusercontent.com/tomeichlersmith/denv/main/install | sh
+
+# configure how ldmx-sw will be built
+configure *CONFIG:
+    denv cmake -B build -S . {{ CONFIG }}
+
+# compile and install ldmx-sw
+build ncpu=num_cpus():
+    denv cmake --build build --target install -- -j{{ ncpu }}
+
+# run the ldmx-sw tests
+test *ARGS:
+    cd build && denv ctest {{ ARGS }}
+
+# run ldmx-sw with the input configuration script
+[no-cd]
+fire config_py *ARGS:
+    denv fire {{ config_py }} {{ ARGS }}
+
+# initialize a containerized development environment
+init:
+    #!/usr/bin/env sh
+    # while setting the denv_workspace is helpful for other
+    # commands that can assume the denv is already initialized,
+    # we need to unset this environment variable to make sure
+    # the test is done appropriately.
+    # just makes sure this recipe runs from the directory of
+    # the justfile so we know we are in the correct location.
+    unset denv_workspace
+    if denv check --workspace --quiet; then
+      echo "\033[32mWorkspace already initialized.\033[0m"
+      denv config print
+    else
+      denv init --clean-env --name ldmx ldmx/dev:latest ${LDMX_BASE}
+    fi
+
+# check that the necessary programs for running ldmx-sw are present
+check:
+    #!/usr/bin/env sh
+    if ! command -v denv 2>&1 > /dev/null; then
+      echo "\033[31mThe program 'denv' is not present.\033[0m"
+      exit 1
+    else
+      echo "\033[32m'denv' has been found.\033[0m"
+    fi
+    # denv can check for container runners it needs
+    denv check
+
+# confirm(PROMPT) just 1.23
+
+# remove the build and install directories of ldmx-sw
+[confirm("This will remove the build and install directories. Are you sure?")]
+clean:
+    rm -r build install
+
+# format the ldmx-sw source code
+format: format-cpp format-just
+
+# format the C++ source code of ldmx-sw
+format-cpp *ARGS='-i':
+    #!/usr/bin/env sh
+    set -exu
+    format_list=$(mktemp)
+    git ls-tree -r HEAD --name-only | egrep '(\.h|\.cxx)$' > ${format_list}
+    denv clang-format {{ ARGS }} $(cat ${format_list})
+    rm ${format_list}
+
+# format the justfile
+format-just:
+    @just --fmt --unstable --justfile {{ justfile() }}
+
+# shellcheck doesn't have a "apply-formatting" option
+# because it really is more of a tidier (its changes could affect code meaning)
+# so only a check is implemented here
+#  ISSUE: the filter implemented here gets all files that are either executable
+#    or have the '.sh' extension. This includes a python script in TrigScint
+#    and some bash-specific scripts as well. Not sure how to handle them.
+
+# check the scripts for common errors and bugs
+shellcheck:
+    #!/usr/bin/env sh
+    set -exu
+    format_list=$(mktemp)
+    git ls-tree -r HEAD | awk '{ if ($1 == 100755 || $4 ~ /\.sh/) print $4 }' > ${format_list}
+    shellcheck --severity style --shell sh $(cat ${format_list})
+
+# below are the mimics of ldmx <cmd>
+# we could think about removing them if folks are happy with committing to the
+# just-style commands above
+
+# open the ROOT shell within the software environment
+root *ARGS="":
+    denv root {{ ARGS }}
+
+# open a ROOT file with a graphical browser
+rootbrowse FILE:
+    denv rootbrowse {{ FILE }}
+
+# change which image is used for the denv
+use IMAGE:
+    denv config image {{ IMAGE }}
+
+# make sure the image is pulled down
+pull IMAGE:
+    denv config image {{ IMAGE }} && denv config image pull
+
+# mount a directory into the denv
+mount DIR:
+    denv config mounts {{ DIR }}
+
+# pass an environment variable into the denv
+setenv +ENVVAR:
+    denv config env copy {{ ENVVAR }}
+
+# configure and build ldmx-sw
+compile ncpu=num_cpus() *CONFIG='': (configure CONFIG) (build ncpu)
+
+# re-build ldmx-sw and then run a config
+recompFire config_py *ARGS: build (fire config_py ARGS)