How to build Proteus using Spack

Useful information before beginning

Warning: This remains a work in progress. This set of build scripts reliably builds Proteus on Linux systems and also works on MacOSX, though not as robustly. Shell programming is brittle at best and is like death by 1000 paper cuts. I surely expect some initial issues with this approach. Direct any questions to: greg dot burgreen at msstate dot edu, or post them on the Proteus slack channel #spack_issues.

Spack works only on Linux and MacOS platforms. Windows is not supported.

Before installing Spack, make sure your system supports Environment Modules package. It can be easily installed.

Ensure your system has the minimum Spack prerequisites.

Install Proteus in Five Easy Steps

The 5 steps involve only a total of 10 commands, 6 of which are standard Unix commands.

The beauty of this approach is that only one Proteus stack is needed to support any number of independent Proteus builds (all on one computer system, of course). Also, if one has different variants of Proteus stacks, any Proteus installation can build against any Proteus stack at any time. Like the HashDist approach, there should be consistency in Proteus builds on any system. However, it may be the case that only one process can compile against a Proteus stack at a time. If so, then perhaps each user needs to create their own personal stack.

Direct any questions to: greg dot burgreen at msstate dot edu

Step 1. Download the spack-setup.tar installation package

$ wget <url>/spack-setup.tar

where <url> = ftp://ftp.hpc.msstate.edu/outgoing/burgreen/spack-proteus/0.3/

$ tar xfv spack-setup.tar

• This will inflate to:

rw-r----- user/group 3020 2018-07-28 13:14 spack-setup-0-README.txt
rwxr-x--- user/group  810 2018-07-28 13:14 spack-setup-1-names-proteus.sh
rwxr-x--- user/group 1177 2018-07-28 13:15 spack-setup-2-transfer-archives.sh
rwxr-x--- user/group 2153 2018-07-28 13:15 spack-setup-3-install-spack.sh

Step 2. Transfer the required archives to the target system

• Optionally, modify spack-setup-1-names-proteus.sh to reflect which specific archives you want to transfer.
• Optionally, modify spack-setup-2-transfer-archives.sh to reflect how to transfer them. Inside this script, there are options that can be easily modified to wget, cp, or scp files.

$ ./spack-setup-2-transfer-archives.sh wget

• This will transfer 7 archive files from the <url> defined in Step 1:

-rw-r----- 1 user group    317440 Jul 20 14:43 packages-builtin-gcc-7.3.0.tar
-rw-r----- 1 user group   1095680 Jul 23 13:59 packages-builtin-proteus-stack.tar
-rw-r----- 1 user group 171550720 Jul 20 12:26 sources-thirdparty-gcc-7.3.0.tar
-rw-r----- 1 user group    153600 Jul 27 15:14 sources-thirdparty-libelf.tar
-rw-r----- 1 user group 364257280 Jul 23 16:13 sources-thirdparty-proteus-stack.tar
-rw-r----- 1 user group   6685201 Jul 17 20:24 spack-develop-2018.07.10.zip
-rw-r----- 1 user group     61440 Jul 27 23:45 spack-scripts.tar

Step 3. Install the Spack software

$ ./spack-setup-3-install-spack.sh <stack_dir>

where <stack_dir> is the directory where the Proteus stack software will reside.

Step 4. Install the Proteus stack software into the Spack repository

$ cd <stack_dir>

• Setup the compiler that is known to successfully install Spack packages. Instructions on how to do this are found in Appendix A.
• For example, a typical work flow on Linux for gcc-4.9.3 will look like:

$ module load gcc/4.9.3

• It is important to confirm that the compiler you wanted is actually loaded.

$ gcc --version

For reliable installation on Linux systems:

• This script will install both a custom gcc-7.3.0 compiler and the packages needed to build Proteus. See explanation in Appendix A.

$ ./scripts/install-gcc-proteus-stack.sh [email protected]

For risky installation on Linux systems using the system compiler only:

• Use a suitable native Linux compiler to install only the packages needed to build Proteus. See explanation in Appendix A. Don't be surprised if this installation fails.

$ ./scripts/install-proteus-stack.sh [email protected]

For MacOS systems:

• Use a suitable native clang compiler to install only the packages needed to build Proteus. See explanation in Appendix A. Don't be surprised if this installation fails.

$ ./scripts/install-proteus-stack.sh [email protected]

Step 5. Build the Proteus software using a Proteus stack

$ cd <proteus_dir>

• where <proteus_dir> is the root direction of existing Proteus software
• This directory can be located anywhere.
• There can be multiple of these directories.

$ <stack_dir>/scripts/build-proteus.sh

• This will build Proteus and install the following files and directories into <proteus_dir>:

rwxr-x---  1 user group   100 Jul 28 11:55 0-README.txt
rwxr-x---  1 user group  3590 Jul 28 11:55 1-setup-proteus.sh*
rw-r-----  1 user group   605 Jul 28 11:54 2-makefile
rwxr-x---  1 user group   365 Jul 28 11:56 make.sh*
drwxr-x--- 2 user group  4096 Jul 28 11:56 bin/
drwxr-x--- 4 user group  4096 Jul 28 11:55 python/

To rebuild this version of Proteus:

$ source <proteus_dir>/1-setup-proteus.sh # needed only once per session

• If you revise only python source code.

$ ./make.sh

• If you revise any non-python source code.

$ ./make.sh clean

$ <stack_dir>/scripts/build-proteus.sh

To rebuild this version of Proteus against a different Proteus stack:

$ source <proteus_dir>/1-setup-proteus.sh # needed only once per session

$ ./make.sh clean

$ <different_stack_dir>/scripts/build-proteus.sh

To execute this version of Proteus:

$ source <proteus_dir>/1-setup-proteus.sh # needed only once per session

$ cd <case_dir>

$ which parun # just for confirmation

$ parun ...

To share the Proteus stack or Proteus builds, simply change directory access privileges.

$ chmod 777 <stack_dir> <proteus_dir>

Appendix A: Finding suitable system compilers that install Spack packages

• In our limited experience, many system compilers fail for various reasons to install the full Proteus stack.

• On Linux systems, the approach that has been most consistent for us has been to build gcc-7.3.0 and binutils using a suitable system compiler. Then, use the installed Spack gcc-7.3.0 to build the Proteus stack and any Proteus source installations. The current scripts follow this approach too. We will revise the scripts soon to allow the build process to use optimized system libraries (e.g., system blas, system mpi, etc).

• Do note that one can use a system gcc-7.3.0 to build the Spack gcc-7.3.0. In fact, we had to do this very thing in order to reliably build the full Proteus stack on our systems.

• On MacOS systems, one can build Spack gcc-7.3.0 using a clang compiler, but gcc on a Mac does not play well with cmake and other packages.

Be advised: A system compiler may succeed in installing the small package (libelf) related to try-compiler.sh, but may fail for various other reasons during the build of the more complex stack packages. If this happens, setup another system compiler, and try again.

• Below is a typical work flow.

$ cd <stack_dir>

$ ./scripts/add-compiler.sh 
$ ./scripts/try-compiler.sh 
$ ./scripts/try-compiler.sh [email protected]

$ module load gcc/4.9.3
$ ./scripts/add-compiler.sh 
$ ./scripts/try-compiler.sh [email protected]

$ module load gcc/7.3.0
$ ./scripts/add-compiler.sh 
$ ./scripts/try-compiler.sh [email protected]