-
Notifications
You must be signed in to change notification settings - Fork 0
Python Workflow
Landon Clipp edited this page Nov 30, 2017
·
2 revisions
This page details the Python implementation for the Basic Fusion workflow. The master script for the workflow is basicFusion/util/TerraGen/BFprocess.py. In order to run this script, run the basicFusion/util/configureEnv.sh script and then activate the Basic Fusion Python virtualenv:
source activateBF
Run python BFprocess.py -h to see the arguments. This script performs the following steps:
- Pull the tarred input from Nearline. Verify integrity of the tar file, then untar the data.
- Run the Basic Fusion program on each orbit of the untarred input data.
- Push the Basic Fusion output data back to Nearline.
PBS scripts use #PBS -l flags=commtransparent. Need to ask NCSA to add user to list that allows use of this flag. This flag spreads the nodes across the entire system. Needed because each MPI rank has high IO output and jobs were causing network congestion.