move environment initialization into separate script

[tomeichlersmith]

Check List

• I successfully built the container using docker
• I was able to build ldmx-sw using this new container build
• I was able to test run a small simulation and reconstruction inside this container
• I was able to successfully use the new packages. Explain what you did to test them below:

Testing

Moving the env init into its own script opens the door to having both an entrypoint (used by the ldmx bash functions in ldmx-sw/scripts/ldmx-env.sh) and an updated .profile (copied by denv into the working directory for the user). The entrypoint is tested by the CI, so I will focus on making sure the updated .profile is useful when running the image with denv.

Set Up

Just in case anyone in the future is reproducing this, I should emphasize that we need
to start from an empty denv. denv does not copy the skeleton files if they already
exist. In addition, in order to save time in start-up, denv creates an empty file
signalling to the entrypoint that it does not need to even check for skeleton files.
In short, I just started from an emtpy envrionment to avoid this.

cd ldmx/
rm -rf .bash* .profile .denv ldmx-sw

Testing

Inside of the directory I want to be LDMX_BASE (on my computers I use ~/code/ldmx)

cd ~/code/ldmx
denv init ldmx/dev:94-copy-env-to-skel

This downloads the image if it isn't already downloaded.
Download the ldmx-sw source code.

git clone --recursive [email protected]:LDMX-Software/ldmx-sw.git

Configure the build

cd ldmx-sw
denv cmake -B build -S .

Compile and install

cd build
denv make install

Run the tests

denv ctest

Build production image

# change FROM image to be the image built here
podman build . -t ldmx/local:entry-test

Make sure local build of production image can run a config

# with denv
denv config image ldmx/local:entry-test
denv fire SimCore/test/basic.py
# with ldmx
. scripts/ldmx-env.sh
ldmx use local entry-test
ldmx fire SimCore/test/basic.py

[tomeichlersmith]

The tests fail due to the LDMX python module not being available.
This is because, following the above procedure, LDMX_BASE is not defined and thus the paths added are unhelpful.

$ denv printenv PYTHONPATH
/ldmx-sw/install/python:/ldmx-sw/install/lib:/usr/local/lib:/externals/lib:/externals/python:/externals/lib/python

This can be fixed by editing .profile with

if [ -z "${LDMX_BASE+x}" ]; then
  export LDMX_BASE="${HOME}"
fi

before source the ldmx-env-init.sh script. This makes me want to add two edits:

1. Add this code into .profile so the start up and be seamless
2. Add a check in ldmx-env-init.sh that LDMX_BASE is defined so that users will be warned if not

[tomeichlersmith]

Reset

# in ~/code/ldmx (LDMX_BASE)
denv config image pull # download new copy of same image tag
rm .profile .denv/skel-init # make sure skels are re-init

Since the libraries are the same between the old and new images, I don't need to re-compile. I just waited until after the "merge" check was done so that the tag on DockerHub points to the new set of image layers.

Resume Testing

• Re-running denv ctest from the build directory works now.
• I can see the additional code block in .profile
• Removing the code that defines LDMX_BASE produces the warning.

[tomeichlersmith]

Edit the ldmx-sw/Dockerfile to use the ldmx/dev:94-copy-env-to-skel in the FROM line rather than the latest release.

diff --git a/Dockerfile b/Dockerfile
index a20e8973..1e47622a 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -3,7 +3,7 @@
 #   for the development image, look at the LDMX-Software/docker repo
 ###############################################################################

-FROM ldmx/dev:4.2.1
+FROM ldmx/dev:94-copy-env-to-skel

 # install ldmx-sw into the container at /usr/local
 COPY . /code

I have podman on this particular computer so I'm going to test build the production image with that.¹

podman build . -t ldmx/local:entry-test

Test run with denv

denv config image ldmx/local:entry-test
denv fire SimCore/test/basic.py
# runs like normal

Test run with ldmx

. scripts/ldmx-env.sh
ldmx use local entry-test
ldmx fire SimCore/test/basic.py
# runs like normal

Footnotes

1. Both podman and docker follow the OCI specification for building and running images, so I expect the test to have the same results regardless on if I use podman or docker. ↩

[tomeichlersmith]

Alright, it appears like this update is not breaking the current setup and enables a smoother use of denv. I am now passing on review to other parties who may be interested.

Whether you are testing denv or ldmx, make sure to switch to using the development image built with this branch: ldmx/dev:94-copy-env-to-skel.

denv init ldmx/dev:94-copy-env-to-skel
# OR if denv init has already been done
denv config image ldmx/dev:94-copy-env-to-skel
# or if using ldmx
ldmx use dev 94-copy-env-to-skel

[EinarElen]

The code changes look perfectly fine to me and I think I understand what the use-case is but maybe you could write down why this change makes denv easier to use?

[tomeichlersmith]

There are two things this does that makes denv easier to use and they are both centered on the *PATH variables within the container.

1. Putting the ldmx-sw install into the *PATH variables. Without this update to the /etc/skel/.profile, a user first trying to use denv would have to edit their local copy of .profile in LDMX_BASE to include the ldmx-sw install in the container *PATH variables. An advanced user would know that the fire: command not found error is related to this, but an inexperienced user would not necessarily know this.
2. Updating PYTHONPATH. Even if the user changed the install of ldmx-sw to be a location that is already in the *PATH variables (e.g. ${LDMX_BASE}/.local instead of ${LDMX_BASE}/ldmx-sw/install), we still put our python config modules in a non-normal install location meaning the user would still need to update PYTHONPATH. Without this change, users would see the LDMX module not found error when trying to run a config which is confusingly far from the issue.

[tvami]

Looks good to me as well! Thanks!