Merge pull request PolusAI#252 from jfennick/doc_update2

Doc update2

docs/advanced.md

@@ -6,8 +6,8 @@ Here is an example that shows how to swap out constant pressure implementations.
 
 ```yaml
 wic:
-  default_backend: gromacs
-  backends:
+  default_implementation: gromacs
+  implementations:
     gromacs:
       steps:
         - npt_gromacs.wic:
@@ -31,13 +31,13 @@ wic:
   steps:
     (2, npt.wic):
       wic:
-        backend: amber
+        implementation: amber
 ```
 This will override the default implementation of `gromacs` and use `amber`. This really just means that `npt_amber.wic` is called instead of `npt_gromacs.wic` (If `--insert_steps_automatically` is enabled, the compiler will attempt to automatically insert the necessary file format conversions as determined below.)
 
 ## Subinterpreters
 
-A portion of [`examples/gromacs/nvt.wic`](https://github.com/PolusAI/mm-workflows/blob/main/examples/gromacs/nvt.wic) in `mm-workflows` is shown below. You can see that the `in:` tag of gmx_energy is identical to the `config:` tag of cwl_watcher. This currently needs to be manually copy & pasted (and indented), but it should be possible to automatically do this in the future.
+A portion of [`examples/gromacs/nvt.wic`](https://github.com/PolusAI/mm-workflows/blob/main/examples/gromacs/nvt.wic) in `mm-workflows` is shown below. You can see that the `in:` tag of gmx_energy is identical to the `config:` tag of cwl_subinterpreter. This currently needs to be manually copy & pasted (and indented), but it should be possible to automatically do this in the future.
 
 ```yaml
 ...
@@ -51,9 +51,9 @@ A portion of [`examples/gromacs/nvt.wic`](https://github.com/PolusAI/mm-workflow
         config: !ii
           terms: [Temperature]
         output_xvg_path: temperature.xvg
-# NOTE: explicit edges are not supported with cwl_watcher, and all filenames
+# NOTE: explicit edges are not supported with cwl_subinterpreter, and all filenames
 # must be globally unique!
-  - cwl_watcher:
+  - cwl_subinterpreter:
       in:
         #cachedir_path: /absolute/path/to/cachedir/ (automatically filled in by wic)
         file_pattern: '*nvt.edr'  # Any strings that start with & or * need to be escaped in quotes
@@ -68,13 +68,13 @@ A portion of [`examples/gromacs/nvt.wic`](https://github.com/PolusAI/mm-workflow
 ...
 ```
 
-Note that although gmx_energy appears before cwl_watcher in the YAML file, gmx_energy is independent of cwl_watcher in the DAG and thus not considered to be a previous step. We include gmx_energy simply to guarantee that the analysis gets run one more time in the main workflow, when all the files are known to be in their final state.
+Note that although gmx_energy appears before cwl_subinterpreter in the YAML file, gmx_energy is independent of cwl_subinterpreter in the DAG and thus not considered to be a previous step. We include gmx_energy simply to guarantee that the analysis gets run one more time in the main workflow, when all the files are known to be in their final state.
 
 ### Known Issues
 
 Since the two runtimes are not linked, there is not currently a reliable way to determine if the previous steps have finished. Thus, to guarantee termination of the second runtime, we simply execute `cwl_tool` upto `max_times`. We also waive any guarantees about the files, so the subworkflow in the second runtime may of course fail for any number of reasons. Thus, we do not propagate speculative failures up to the main workflow.
 
-The runtime system intentionally hides the working sub-directories of each step. Thus, we are forced to use a file watcher (hence the name cwl_watcher) recursively starting from `cachedir_path`. This is why all filenames used with cwl_watcher must be globally unique. (Actually, for technical reasons we cannot use a file watching library; we simply use a good old fashioned polling loop.)
+The runtime system intentionally hides the working sub-directories of each step. Thus, we are forced to use a file watcher recursively starting from `cachedir_path`. This is why all filenames used with cwl_subinterpreter must be globally unique. (Actually, for technical reasons we cannot use a file watching library; we simply use a good old fashioned polling loop.)
 
 ## Real-time plots
 

docs/overview.md

@@ -4,13 +4,13 @@
 
 Subworkflows are used extensively to create reusable building blocks.
 
-For example, a basic molecular dynamics workflow is composed of minimization, equilibration, and production steps, the equilibration step is composed of constant volume (NVT) and constant pressure (NPT) steps, and each of those are composed of primitive backend-specific steps. If we then want to do a stability analysis, we should be able to incorporate the molecular dynamics workflow as a black box, and we should only have to append the stability analysis subworkflow. See [here](https://github.com/PolusAI/mm-workflows/blob/main/examples/gromacs/tutorial.wic)!
+For example, a basic molecular dynamics workflow is composed of minimization, equilibration, and production steps, the equilibration step is composed of constant volume (NVT) and constant pressure (NPT) steps, and each of those are composed of primitive implementation-specific steps. If we then want to do a stability analysis, we should be able to incorporate the molecular dynamics workflow as a black box, and we should only have to append the stability analysis subworkflow. See [here](https://github.com/PolusAI/mm-workflows/blob/main/examples/gromacs/tutorial.wic)!
 
-* Multiple backends
+* Multiple implementations
 
-There are often several backend engines that implement the same algorithm (e.g. amber / gromacs / namd). In principle, each backend ought to be exactly interchangeable, but in practice backends may randomly crash, etc. For this reason, we want the ability to arbitrarily switch backends at any step. Moreover, different users may be familiar with different backends, but we still want to compose together their workflows.
+There are often several implementation engines that implement the same algorithm (e.g. amber / gromacs / namd). In principle, each implementation ought to be exactly interchangeable, but in practice implementations may randomly crash, etc. For this reason, we want the ability to arbitrarily switch implementations at any step. Moreover, different users may be familiar with different implementations, but we still want to compose together their workflows.
 
-For example, we should be able to compose system setup using amber/tleap, equilibration using namd, and metadynamics using gromacs. File format conversions should be automatically inserted between steps. This is not possible with other 'backend-independent' software packages which require a single backend to be fixed at the beginning and used throughout.
+For example, we should be able to compose system setup using amber/tleap, equilibration using namd, and metadynamics using gromacs. File format conversions should be automatically inserted between steps. This is not possible with other 'implementation-independent' software packages which require a single implementation to be fixed at the beginning and used throughout.
 
 See [static dispatch](advanced.md#static-dispatch)
 

examples/config_ci.json

@@ -65,10 +65,10 @@
         "cg",
         "yank",
         // These (currently) always return success, so no point in running them.
-        "cwl_watcher_analysis",
-        "cwl_watcher_complex",
-        "cwl_watcher_ligand",
-        "cwl_watcher_protein",
+        "cwl_subinterpreter_analysis",
+        "cwl_subinterpreter_complex",
+        "cwl_subinterpreter_ligand",
+        "cwl_subinterpreter_protein",
         // Skip vs_demo_4 and run_diffdock so we don't accidentally DOS pdbbind.org.cn
         "vs_demo_4",
         "run_diffdock"

examples/docking/docking_stability.wic

@@ -37,12 +37,12 @@ wic:
                 (3, prod.wic):
                   wic:
                     steps:
-                      (4, cwl_watcher_analysis.wic):
+                      (4, cwl_subinterpreter_analysis.wic):
                         wic:
-                          backend: complex
+                          implementation: complex
           (3, analysis.wic):
             wic:
               steps:
                 (1, analysis_realtime.wic):
                   wic:
-                    backend: complex
+                    implementation: complex

examples/docking/vs_demo_2.wic

@@ -178,18 +178,18 @@ wic:
                         - output_crd_path: !& prod_complex.gro  # overload prod.gro
                         - output_trr_path: !& prod.trr
                         - output_edr_path: !& prod.edr
-                      (4, cwl_watcher_analysis.wic):
+                      (4, cwl_subinterpreter_analysis.wic):
                         wic:
-                          backend: complex
+                          implementation: complex
           (3, analysis.wic):
             wic:
               steps:
                 (1, analysis_realtime.wic):
                   wic:
-                    backend: complex
+                    implementation: complex
                 (2, analysis_final.wic):
                   wic:
-                    backend: complex
+                    implementation: complex
     (11, autodock_vina_rescore.wic):
       wic:
         inlineable: False

examples/docking/vs_demo_3.wic

@@ -154,18 +154,18 @@ wic:
                         - output_crd_path: !& prod_complex.gro  # overload prod.gro
                         - output_trr_path: !& prod.trr
                         - output_edr_path: !& prod.edr
-                      (4, cwl_watcher_analysis.wic):
+                      (4, cwl_subinterpreter_analysis.wic):
                         wic:
-                          backend: complex
+                          implementation: complex
           (3, analysis.wic):
             wic:
               steps:
                 (1, analysis_realtime.wic):
                   wic:
-                    backend: complex
+                    implementation: complex
                 (2, analysis_final.wic):
                   wic:
-                    backend: complex
+                    implementation: complex
     (9, autodock_vina_rescore.wic):
       wic:
         inlineable: False

examples/docking/vs_demo_4.wic

@@ -229,18 +229,18 @@ wic:
                         - output_crd_path: !& prod_complex.gro  # overload prod.gro
                         - output_trr_path: !& prod.trr
                         - output_edr_path: !& prod.edr
-                      (4, cwl_watcher_analysis.wic):
+                      (4, cwl_subinterpreter_analysis.wic):
                         wic:
-                          backend: complex
+                          implementation: complex
           (3, analysis.wic):
             wic:
               steps:
                 (1, analysis_realtime.wic):
                   wic:
-                    backend: complex
+                    implementation: complex
                 (2, analysis_final.wic):
                   wic:
-                    backend: complex
+                    implementation: complex
     (13, autodock_vina_rescore.wic):
       wic:
         inlineable: False

examples/gromacs/analysis_final.wic

@@ -1,6 +1,6 @@
 wic:
-  default_backend: protein
-  backends:
+  default_implementation: protein
+  implementations:
     protein:
       steps:
         - analysis_final_steps.wic:

examples/gromacs/analysis_realtime.wic

@@ -1,6 +1,6 @@
 wic:
-  default_backend: protein
-  backends:
+  default_implementation: protein
+  implementations:
     protein:
       steps:
         - analysis_realtime_protein.wic:

examples/gromacs/analysis_realtime_complex.wic

@@ -3,7 +3,7 @@
 # This causes the rmsd w.r.t. the last equil timestep to start at exactly zero,
 # which distorts the comparison between the xray and equil cases.
 # NOTE: Most paths can be inferred, but explicitly specify them anyway because
-# for cwl_watcher and so we can re-order steps in analysis.
+# for cwl_subinterpreter and so we can re-order steps in analysis.
 steps:
   - gmxselect: # NOTE: make_ndx does NOT work! (It erroneously selects all of the atoms.)
       in:

examples/gromacs/analysis_realtime_ligand.wic

@@ -3,7 +3,7 @@
 # This causes the rmsd w.r.t. the last equil timestep to start at exactly zero,
 # which distorts the comparison between the xray and equil cases.
 # NOTE: Most paths can be inferred, but explicitly specify them anyway because
-# for cwl_watcher and so we can re-order steps in analysis.
+# for cwl_subinterpreter and so we can re-order steps in analysis.
 steps:
   - gmxselect: # NOTE: make_ndx does NOT work! (It erroneously selects all of the atoms.)
       in:

examples/gromacs/analysis_realtime_protein.wic

@@ -3,7 +3,7 @@
 # This causes the rmsd w.r.t. the last equil timestep to start at exactly zero,
 # which distorts the comparison between the xray and equil cases.
 # NOTE: Most paths can be inferred, but explicitly specify them anyway because
-# for cwl_watcher and so we can re-order steps in analysis.
+# for cwl_subinterpreter and so we can re-order steps in analysis.
 steps:
   - gmx_rms:
       in:

examples/gromacs/cwl_watcher_analysis.wic → examples/gromacs/cwl_subinterpreter_analysis.wic

@@ -1,15 +1,15 @@
 wic:
-  default_backend: protein
-  backends:
+  default_implementation: protein
+  implementations:
     protein:
       steps:
-        - cwl_watcher_protein.wic:
+        - cwl_subinterpreter_protein.wic:
     ligand:
       steps:
-        - cwl_watcher_ligand.wic:
+        - cwl_subinterpreter_ligand.wic:
     complex:
       steps:
-        - cwl_watcher_complex.wic:
+        - cwl_subinterpreter_complex.wic:
   graphviz:
     label: 'Real-time\nAnalysis'
     style: invis # Make this graph (and all subgraphs) invisible

examples/gromacs/cwl_watcher_complex.wic → examples/gromacs/cwl_subinterpreter_complex.wic

@@ -1,5 +1,5 @@
 steps:
-  - cwl_watcher:
+  - cwl_subinterpreter:
       in:
         #cachedir_path: /absolute/path/to/cachedir/ (automatically filled in by wic)
         file_pattern: !ii '*prod.trr'
@@ -24,7 +24,7 @@ steps:
 
 wic:
   steps:
-    (1, cwl_watcher):
+    (1, cwl_subinterpreter):
       wic:
         graphviz:
           style: invis # Make this node invisible

examples/gromacs/cwl_watcher_ligand.wic → examples/gromacs/cwl_subinterpreter_ligand.wic

@@ -1,5 +1,5 @@
 steps:
-  - cwl_watcher:
+  - cwl_subinterpreter:
       in:
         #cachedir_path: /absolute/path/to/cachedir/ (automatically filled in by wic)
         file_pattern: !ii '*prod.trr'
@@ -11,7 +11,7 @@ steps:
 
 wic:
   steps:
-    (1, cwl_watcher):
+    (1, cwl_subinterpreter):
       wic:
         graphviz:
           style: invis # Make this node invisible

examples/gromacs/cwl_watcher_protein.wic → examples/gromacs/cwl_subinterpreter_protein.wic

@@ -1,5 +1,5 @@
 steps:
-  - cwl_watcher:
+  - cwl_subinterpreter:
       in:
         #cachedir_path: /absolute/path/to/cachedir/ (automatically filled in by wic)
         file_pattern: !ii '*prod.trr'
@@ -14,7 +14,7 @@ steps:
 
 wic:
   steps:
-    (1, cwl_watcher):
+    (1, cwl_subinterpreter):
       wic:
         graphviz:
           style: invis # Make this node invisible

examples/gromacs/equil.wic

@@ -8,5 +8,5 @@ wic:
   steps:
     (2, npt.wic):
       wic:
-        backend: gromacs
-#        backend: amber
+        implementation: gromacs
+#        implementation: amber

examples/gromacs/npt.wic

@@ -1,6 +1,6 @@
 wic:
-  default_backend: gromacs
-  backends:
+  default_implementation: gromacs
+  implementations:
     gromacs:
       steps:
         - npt_gromacs.wic:

examples/gromacs/npt_amber.wic

@@ -13,7 +13,7 @@ steps:
         config: !ii
           terms: [DENSITY]
         output_dat_path: !ii density.dat
-  - cwl_watcher:
+  - cwl_subinterpreter:
       in:
         #cachedir_path: /absolute/path/to/cachedir/ (automatically filled in by wic)
         file_pattern: !ii '*amb_npt.log'
@@ -38,7 +38,7 @@ wic:
       wic:
         graphviz:
           label: 'Analyze & Plot\nDensity\nTimeseries'
-    (3, cwl_watcher):
+    (3, cwl_subinterpreter):
       wic:
         graphviz:
           label: 'Real-time\nAnalysis'

examples/gromacs/npt_gromacs.wic

@@ -33,7 +33,7 @@ steps:
         config: !ii
           terms: [Density]
         output_xvg_path: !ii density.xvg
-  # - cwl_watcher:
+  # - cwl_subinterpreter:
   #     in:
   #       #cachedir_path: /absolute/path/to/cachedir/ (automatically filled in by wic)
   #       file_pattern: !ii '*npt.edr'
@@ -64,7 +64,7 @@ wic:
       wic:
         graphviz:
           label: 'Analyze & Plot\nDensity\nTimeseries'
-    (4, cwl_watcher):
+    (4, cwl_subinterpreter):
       wic:
         graphviz:
           #label: 'Real-time\nAnalysis'

examples/gromacs/nvt.wic

@@ -28,9 +28,9 @@ steps:
         config: !ii
           terms: [Temperature]
         output_xvg_path: !ii temperature.xvg
-# NOTE: explicit edges are not supported with cwl_watcher, and all filenames
+# NOTE: explicit edges are not supported with cwl_subinterpreter, and all filenames
 # must be globally unique!
-  # - cwl_watcher:
+  # - cwl_subinterpreter:
   #     in:
   #       #cachedir_path: /absolute/path/to/cachedir/ (automatically filled in by wic)
   #       file_pattern: !ii '*nvt.edr'
@@ -60,7 +60,7 @@ wic:
       wic:
         graphviz:
           label: 'Analyze & Plot\nTemperature\nTimeseries'
-    (4, cwl_watcher):
+    (4, cwl_subinterpreter):
       wic:
         graphviz:
           label: 'Real-time\nAnalysis'

examples/gromacs/prod.wic

@@ -45,12 +45,12 @@ steps:
       - output_tpr_path: !& prod.tpr
   - mdrun:
       in:
-        output_trr_path: !ii prod.trr # Explicitly specify for cwl_watcher
+        output_trr_path: !ii prod.trr # Explicitly specify for cwl_subinterpreter
       out:
       - output_crd_path: !& prod.gro
       - output_trr_path: !& prod.trr
       - output_edr_path: !& prod.edr
-  # - cwl_watcher_analysis.wic:
+  # - cwl_subinterpreter_analysis.wic:
 
 wic:
   graphviz:
@@ -69,9 +69,9 @@ wic:
         namespace: gpu  # To run MD on GPU
         graphviz:
           label: 'Molecular\nDynamics'
-    (4, cwl_watcher_analysis.wic):
+    (4, cwl_subinterpreter_analysis.wic):
       wic:
-        backend: protein
+        implementation: protein
         graphviz:
           label: 'Real-time\nAnalysis'
           style: invis # Make this node invisible