Merge develop into dev/jk before merging it back

.github/workflows/ci_workflows.yml

@@ -23,12 +23,12 @@ jobs:
 
     steps:
     - name: Checkout code
-      uses: actions/checkout@v2
+      uses: actions/checkout@v4
       with:
         fetch-depth: 0
 
     - name: Set up Python ${{ matrix.python }}
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python }}
 
@@ -42,10 +42,15 @@ jobs:
 
     - name: Get WebbPSF Data
       run: | # Get WebbPSF data files (just a subset of the full dataset!) and set up environment variable
-           wget https://stsci.box.com/shared/static/0ojjfg3cieqdpd18vl1bjnpe63r82dx8.gz -O /tmp/minimal-webbpsf-data.tar.gz
+           wget https://stsci.box.com/shared/static/gkbnn3jwcootr9riwv0rv1t8rpvxl3vr.gz -O /tmp/minimal-webbpsf-data.tar.gz
            tar -xzvf /tmp/minimal-webbpsf-data.tar.gz
            echo "WEBBPSF_PATH=${{github.workspace}}/webbpsf-data" >> $GITHUB_ENV
 
+    - name: Set webbpsf_ext and CDBS Data Paths
+      run: | # Set up temporary data environment variables
+        echo "WEBBPSF_EXT_PATH=$(mktemp -d)" >> $GITHUB_ENV
+        echo "PYSYN_CDBS=$(mktemp -d)" >> $GITHUB_ENV
+
     - name: Get SpaceKLIP test Data
       if: ${{ contains(matrix.toxenv,'test') }}
       run: |  

.gitignore

@@ -3,6 +3,9 @@ __pycache__/
 *.py[cod]
 *$py.class
 
+# Transmission mask files
+spaceKLIP/resources/transmissions/NRC*
+
 # C extensions
 *.so
 

README.rst

@@ -1,13 +1,15 @@
-###############
-SpaceKLIP 2 🚀🪐
-###############
+##############
+SpaceKLIP 🚀🪐
+##############
 
 .. image:: logo.png
 
 SpaceKLIP is a data reduction pipeline for JWST high-contrast imaging. The current release has been tested with NIRCam and MIRI coronagraphy data.
 
 Compatible test data: `here <https://stsci.box.com/s/0oteh8smujl3pup07hyut6hr4ag1i2el>`_ 
 
+Documentation: `here <https://spaceklip.readthedocs.io/en/latest/>`_
+
 Installation
 ************
 
@@ -27,7 +29,7 @@ From here, it is **highly** recommended that you create a unique Anaconda enviro
 
 ::
 
-	conda create -n spaceklip python=3
+	conda create -n spaceklip python=3.11
 	conda activate spaceklip
 
 With the Anaconda environment created, move to the cloned directory and install the dependencies and spaceKLIP itself:
@@ -38,18 +40,6 @@ With the Anaconda environment created, move to the cloned directory and install
 	pip install -r requirements.txt
 	pip install -e .
 
-Install the develop branch of WebbPSF_ext:
-
-::
-
-	pip install git+https://github.com/JarronL/webbpsf_ext.git@develop
-
-NEW AS OF 1 MAY 2023: you also need to switch to the jwst branch of pyKLIP:
-
-::
-
-	pip install git+https://bitbucket.org/pyKLIP/pyklip.git@jwst
-
 NEW AS OF 1 JUNE 2023: you also need to make the custom PSF mask files before running spaceKLIP for the first time:
 
 ::
@@ -158,7 +148,7 @@ The current capabilities of the code are summarized below.
 - ``raw_contrast``: working with ``pyklippipeline`` products, not working with ``coron3pipeline`` products since PSF center not being tracked
 - ``extract_companions``: working with ``pyklippipeline`` products, not working with ``coron3pipeline`` products since PSF center not being tracked
 - combined coronagraphic throughput map: working
-- need to add calibrated contrast
+- `` calibrated_contrast`` working with ``pyklippipeline`` products
 
 ::
 

docs/source/Attribution.rst

@@ -1,7 +1,7 @@
 Attribution
 -----------
 
-Maintenance and current development of ``spaceKLIP`` is led by Aaryn Carter and Jens Kammerer. spaceKLIP also benefitted from contributions made by a number of collaborators.
+Maintenance and current development of ``spaceKLIP`` is led by Aarynn Carter and Jens Kammerer. spaceKLIP also benefitted from contributions made by a number of collaborators.
 More details about the respective contributions are available `here <https://github.com/kammerje/spaceKLIP/graphs/contributors>`_.
 
 

docs/source/Installation-and-dependencies.rst

@@ -1,4 +1,4 @@
-Installation and dependencies
+Installation and Dependencies
 -----------------------------
 The current release of spaceKLIP is not stable enough for a straightforward install with conda / pip. At this stage
 it is recommended that you clone the git repository directory for installation:
@@ -14,11 +14,11 @@ If you would like a specific branch:
 	git clone https://github.com/kammerje/spaceKLIP.git@branch
 
 From here, it is **highly** recommended that you create a unique anaconda environment to hold all of the spaceKLIP
-dependencies. 
+dependencies. spaceKLIP is not currently compatible with python 3.12
 
 ::
 
-	conda create -n sklip python=3
+	conda create -n sklip python=3.11
 	conda activate sklip
 
 With the anaconda environment created, move to the cloned directory and install most of the dependencies:
@@ -27,13 +27,10 @@ With the anaconda environment created, move to the cloned directory and install
 
 	cd where/you/installed/the/git/repo
 	pip install -r requirements.txt
+	pip install -e .
+	cd spaceKLIP/
+	python make_psfmasks.py
 
-You will also need to install some other specific packages, namely:
 
-::
-
-	pip install git+https://bitbucket.org/pyKLIP/pyklip.git@jwst
-	pip install git+https://github.com/JarronL/webbpsf_ext.git
 
-Finally, and very importantly, you will need to download reference files to support the functioning of 
-the :code:`webbpsf` and :code:`webbpsf_ext`. Instructions to do this can be found at the respective package websites (`WebbPSF <https://webbpsf.readthedocs.io/en/latest/installation.html#installing-the-required-data-files>`_, `webbpsf_ext <https://github.com/JarronL/webbpsf_ext>`_). Ensure that if you edit your .bashrc file that you reopen and close your terminal to fully apply the changes (:code:`source ~/.bashrc` or :code:`source ~/.zshrc` may also work)
+Finally, and very importantly, you will need to download reference files to support the functioning of the :code:`webbpsf` and :code:`webbpsf_ext`. Instructions to do this can be found at the respective package websites (`WebbPSF <https://webbpsf.readthedocs.io/en/latest/installation.html#installing-the-required-data-files>`_, `webbpsf_ext <https://github.com/JarronL/webbpsf_ext>`_). Ensure that if you edit your .bashrc file that you reopen and close your terminal to fully apply the changes (:code:`source ~/.bashrc` or :code:`source ~/.zshrc` may also work)

docs/source/TLDR.rst

@@ -1,8 +1,8 @@
-TL;DR setup guide
+TL;DR Setup Guide
 -----------------
 .. code-block:: bash
 
     $ git clone https://github.com/kammerje/spaceKLIP.git
     $ pip install -r requirements.txt
-    $ pip install git+https://bitbucket.org/pyKLIP/pyklip.git@jwst
-    $ pip install git+https://github.com/JarronL/webbpsf_ext.git
+    $ pip install -e .
+    $ python make_psfmasks.py

docs/source/_static/wider_docs.css

@@ -0,0 +1,3 @@
+.wy-nav-content {
+max-width: 1200px !important;
+}

docs/source/analysistools.rst

@@ -1,6 +1,6 @@
 .. _analysistools:
 
-spaceklip.analysistools 
---------------------
+Analysis Tools
+--------------
 
-Documentation of the ``spaceklip.analysistools`` module will go here.
+TBD: Detailed documentation of the ``spaceklip.analysistools`` module will go here.

docs/source/conf.py

@@ -18,14 +18,32 @@
 #
 import os
 import sys
+from pathlib import Path
+
+try:
+    import tomllib
+except ImportError:
+    import tomli as tomllib
 
 # fix for `ImportError: No module named _tkinter`:
 import matplotlib
 matplotlib.use("agg")
 
-import spaceKLIP
 
-version = spaceKLIP.__version__
+#### Get version 
+with open(Path(__file__).parent.parent.parent / "pyproject.toml", "rb") as metadata_file:
+    configuration = tomllib.load(metadata_file)
+    metadata = configuration["project"]
+    project = metadata["name"]
+
+    # The short X.Y version.
+    try:
+        version = project.__version__.split("-", 1)[0]
+        # The full version, including alpha/beta/rc tags.
+        release = project.__version__
+    except AttributeError:
+        version = "dev"
+        release = "dev"
 
 # -- General configuration ------------------------------------------------
 # Add any Sphinx extension module names here, as strings. They can be
@@ -36,14 +54,25 @@
     'sphinx.ext.todo',
     'sphinx.ext.mathjax',
     'sphinx.ext.viewcode',
-    'sphinx.ext.napoleon',
+    'sphinx_automodapi.automodapi',
     'nbsphinx'
     # 'sphinx.ext.githubpages',
 ]
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = []#['./_templates']
 
+# mock imports for autodoc
+autodoc_mock_imports = ["webbpsf"]
+
+# nbsphinx settings
+nbsphinx_allow_errors = True
+nbsphinx_execute = 'never'
+nbsphinx_prolog = """
+{% set docname = env.doc2path(env.docname, base=None) %}
+.. note::  `Download the full notebook for this tutorial here <https://github.com/kammerje/spaceKLIP/tree/develop/docs/source/{{ docname }}>`_
+"""
+
 # The suffix(es) of source filenames.
 # You can specify multiple suffix as a list of string:
 #
@@ -204,6 +233,8 @@
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
+def setup(app):
+    app.add_css_file('wider_docs.css')
 html_static_path = ['./_static']
 
 # Add any extra paths that contain custom files (such as robots.txt or

docs/source/imagetools.rst

@@ -1,6 +1,6 @@
 .. _imagetools:
 
-spaceklip.imagetools 
---------------------
+Image Tools
+-----------
 
-Documentation of the ``spaceklip.imagetools`` module will go here.
+TBD: Detailed documentation of the ``spaceklip.imagetools`` module will go here.

docs/source/index.rst

@@ -31,7 +31,7 @@ On a high level, spaceKLIP consists of three different reduction steps:
 SpaceKLIP also provides additional functions for post-pipeline scientific analyses of the PSF subtracted images. These functions can be found in the :ref:`analysistools` module. There are functions to:
 
  * retrieve the properties of detected companions 
- * to compute basic contrast curves
+ * to compute basic and calibrated contrast curves
  * to inject one or more companions into a given data set. 
 
 There are several tutorial Jupyter notebooks which show how to use these
@@ -47,41 +47,53 @@ adapted by the user if necessary.
 Compatible Simulated Data: `Here <https://stsci.box.com/s/cktghuyrwrallb401rw5y5da2g5ije6t>`_
 
 .. toctree::
-   :maxdepth: 2
-   :caption: Getting started
+   :maxdepth: 1
+   :caption: Getting Started
    :hidden:
 
    TLDR
    Installation-and-dependencies
-   imagetools
-   analysistools
 
 .. toctree::
-   :maxdepth: 3
+   :maxdepth: 1
    :caption: Tutorials
    :hidden:
 
-   tutos
-   tutorials/Introduction.ipynb
+   tutorials
    tutorials/tutorial_NIRCam_reductions.ipynb
    tutorials/tutorial_MIRI_reductions.ipynb
    tutorials/tutorial_NIRCam_contrast_analyses.ipynb
    tutorials/tutorial_MIRI_contrast_analyses.ipynb
    tutorials/MAST query tools for coronagraphic datasets.ipynb
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
+   :caption: spaceKLIP Functionality (Unfinished)
+   :hidden:
+
+   stage1
+   stage2
+   imagetools
+   subtraction
+   analysistools
+
+.. toctree::
+   :maxdepth: 1
    :caption: About
    :hidden:
 
    Contact
    Attribution
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
    :caption: Package content
    :hidden:
 
    spaceKLIP
    gen_index
 
+
+
+
+

docs/source/spaceKLIP.rst

@@ -1,4 +1,4 @@
-spaceKLIP detailed API
+Detailed spaceKLIP API
 ======================
 
 spaceKLIP.analysistools module
@@ -57,14 +57,6 @@ spaceKLIP.imagetools module
    :undoc-members:
    :show-inheritance:
 
-spaceKLIP.make\_psfmasks module
--------------------------------
-
-.. automodule:: spaceKLIP.make_psfmasks
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
 spaceKLIP.mast module
 ---------------------
 

docs/source/stage1.rst

@@ -0,0 +1,6 @@
+.. _stage1:
+
+Stage 1
+-------
+
+TBD: Detailed documentation of spaceKLIP Stage 1 processing will go here.

docs/source/stage2.rst

@@ -0,0 +1,6 @@
+.. _stage2:
+
+Stage 2
+-------
+
+TBD: Detailed documentation of spaceKLIP Stage 2 processing will go here.