develop: Update NRL batch install script, add configuration for Cole and Tusk #1489
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…k; add configs/sites/tier1/blueback-tds-cole
…itHub for CMakeModules and spack
…elease/neptune-1.5
…slurm before openmpi on Atlantis/GCC
…s.yaml to avoid duplicate packages
…duplicate packages
…unified-env with Intel Classic
…nfigs/sites/tier1/narwhal/compilers.gcc-direct.tmp to /site/compilers.yaml
…atus size changed, may indicate binary incompatibility.'
…kages at the end of each environment build in installer mode
…l_batch_install.sh
… mirror logic, fix permissions on directories
…ents (not sure we need it) to util/nrl/batch_install.sh
…batch_install.sh (#6) 1. Rename util/nrl_batch_install.sh to util/nrl/batch_install.sh, consistent with authoritative spack-stack code 2. Update util/nrl/batch_install.sh: a. Add capability to create and use bootstrap mirrors b. Define function fix_permissions and use it to set the correct permissions for caches (mirrors) in developer mode, and for environments in installer mode
…eature/blueback_tds_cole
…cache of you are installer
configs/sites/tier1/blueback-tds-tusk --> configs/sites/tier2/tusk
…k info, no longer valid
Add Makefile and update util/nrl/batch_install.sh to fine tune actions for different roles (ops, dev) combined with different modes (build, install).
… for air-gapped systems
…Blueback config (JCSDA#21) This PR adds tier2 site configs for Cole and Tusk and updates util/nrl/batch_install.sh accordingly. It also removes the confusing configuration for Blueback, which was based on the Tusk configuration. We will add a proper Blueback configuration once we have access to the system.
…eature/blueback_tds_cole
climbfuji
commented
Feb 4, 2025
configs/common/packages.yaml
Outdated
| py-netcdf4: | ||
| require: '@1.7.1 +mpi' |
3 tasks
areinecke
approved these changes
Feb 4, 2025
climbfuji
added a commit
that referenced
this pull request
Feb 4, 2025
…mpi` from #1489 (#1491) In PR #1489 we are changing the requirements for py-netcdf4 from [email protected]: +mpi to [email protected]: ~mpi in configs/common/packages.yaml. This change is required to fix an error with py-netcdf4 on certain systems when built with +mpi. We used to build py-netcdf4 without mpi, but for a period this wasn't possible until we added a patch to disable the py-netcdf4 auto-detect parallel feature. That patch allows us to build py-netcdf4 ~mpi even if netcdf-c was built with +mpi. --------- Co-authored-by: Alex Richert <[email protected]>
climbfuji
added a commit
that referenced
this pull request
Feb 4, 2025
…elop * Update .gitmodules and doc/source/conf.py for spack-stack release/1.9.0 * Avoid linking to libirc.so in spack (parallel-netcdf), turn off crypt variant for Python, and update Orion site config to fix tar issue (#1435) 1. Applications built with spack-stack packages esmf, parallelio, parallel-netcdf have libirc.so dynamically linked. Applications linked against libirc.so fail to start up. See Avoid linking to Intel's libirc.so library (aka bad configure script of package parallel-netcdf) #1436. The spack PR that is part of the suggested changes here fixes this by replacing libirc.so with libintlc.so in the parallel-netcdf build. See Bug fix in parallel-netcdf to avoid linking to libirc.so AND cherry-pick spack develop PR 48251 (conflict Intel Classic with [email protected]) spack#495. 2. Turn off crypt variant for Python; this variant leads to build errors with Intel in py-cryptography unless external curl and openssl are removed, which itself is problematic. 3. Add external wget on Orion, latest versions don't build with Intel on the machine. --------- Co-authored-by: Stephen Herbener <[email protected]> * Update ectrans from 1.2.0 to 1.5.0 in configs/common/packages.yaml (#1474) * Update .gitmodules and submodule pointer for spack for code review and testing * In spack-ext/lib/jcsda-emc/spack-stack/stack, update meta_modules.py and templates/{mpi,mpi.lua}: set compiler paths in MPI meta modules directly using SUBSTITUTES_SAVE, not using environment variables (#1479) * Revert .gitmodules and update submodule pointer for spack * release/1.9.0: Update instructions for setting up spack-stack with Nvidia compilers (#1462) This PR brings the Nvidia instructions a bit more up-to-date. On develop, the instructions only worked with Ubuntu 22.04 spack-stack 34bfda1 [email protected] With this PR, these constraints are updated to the slightly more recent Ubuntu 24.04 spack-stack 26901af [email protected] * For orion, intel config, pin py-numpy to version 1.26. This prevents (#1482) getting unwanted duplicate packages during concretize. * release/1.9.0: Add [email protected] to unified-dev and skylab-dev templates, bug fix in depencies for awscli-v2, bump wgrib2 to 3.5.0 and re-enable for all compilers (#1486) 1. Add [email protected] to templates skylab-dev and unified-dev (new version was added in recently merged PR Update crtm(-fix), wgrib2 spack#510) 2. Bump wgrib2 from 3.1.1 to 3.5.0 and re-enable for all compilers in spack-ext packages (new version was added in recently merged PR Update crtm(-fix), wgrib2 spack#510). Note. [email protected] doesn't compile on macOS with apple-clang (version 14.0.3 on the CI runner), see wgrib 3.5.0 does not compile with apple-clang 14.0.3 on macOS NOAA-EMC/wgrib2#312. But 3.4.0 does compile, thereforeuse this version on macOS only 3. Update spack submodule pointer for PR Update crtm(-fix), wgrib2 spack#510 and the changes in release/1.9.0: Fix bug in awcli-v2, add upper bound for py-cryptography spack#511 (fix upper bound for py-cryptography in awscli-v2) and release/1.9.0: Bug fix in wgrib2: apply '-Wno-error=implicit-function-declaration' for LLVM clang spack#513 (bug fix for wgrib2 with apple-clang) --------- Co-authored-by: Alex Richert <[email protected]> * Update .gitmodules and submodule pointer for spack for code review and testing * For release/1.9.0: cherry-pick `[email protected]: ~mpi` instead of `+mpi` from #1489 (#1491) In PR #1489 we are changing the requirements for py-netcdf4 from [email protected]: +mpi to [email protected]: ~mpi in configs/common/packages.yaml. This change is required to fix an error with py-netcdf4 on certain systems when built with +mpi. We used to build py-netcdf4 without mpi, but for a period this wasn't possible until we added a patch to disable the py-netcdf4 auto-detect parallel feature. That patch allows us to build py-netcdf4 ~mpi even if netcdf-c was built with +mpi. --------- Co-authored-by: Alex Richert <[email protected]> * Revert .gitmodules and update submodule pointer for spack --------- Co-authored-by: Stephen Herbener <[email protected]> Co-authored-by: Francois Hebert <[email protected]> Co-authored-by: Stephen Herbener <[email protected]> Co-authored-by: Alex Richert <[email protected]>
…1/heinzell/spack-stack/oneapi-2024.2.1/modulefiles' for Cole
…eature/blueback_tds_cole
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR merges selected features of the release/neptune-1.5 branch at NRL into develop: an update of the NRL
batch_install.shscript and tier2 site configurations for Cole and Tusk. The confusing Blueback site config is removed and will be added back in once Blueback is available.Testing
Used updated batch install script on all NRL systems.
Applications affected
None
Systems affected
NRL systems
Dependencies
None
Issue(s) addressed
None
Checklist
All dependency PRs/issues have been resolved and this PR can be merged.