Merge branch 'develop' into badpixels

.github/ISSUE_TEMPLATE/bug-template.yml

@@ -0,0 +1,72 @@
+name: Bug Report
+description: File a bug report
+title: "[Bug]: "
+labels: ["bug"]
+body:
+  - type: checkboxes
+    id: verify
+    attributes:
+      label: Check Existing Issues
+      description: |
+        Before submitting this issue, please confirm the following:
+      options:
+        - label: Yes, I have checked existing issues to ensure this problem hasn't already been reported.
+          required: true
+  - type: dropdown
+    id: instrument
+    attributes:
+      label: Instrument or Category
+      description: Which instrument or piece of code were you running?
+      multiple: true
+      options:
+        - NIRCam Stage 1/2 Pipeline
+        - MIRI Stage 1/2 Pipeline
+        - Image Tools
+        - PSF Subtraction
+        - Analysis Tools
+        - Other
+    validations:
+      required: true
+  - type: textarea
+    id: description
+    attributes:
+      label: Description
+      description: Please describe what happened and/or what you expected to happen.
+      placeholder: Describe the issue and/or expected outcome.
+    validations:
+      required: true
+  - type: textarea
+    id: logs
+    attributes:
+      label: Error traceback output
+      description: Please copy and paste the full traceback of the error you encountered.
+      placeholder: Paste the error traceback here.
+    validations:
+      required: false
+  - type: input
+    id: OS
+    attributes:
+      label: What operating system are you using?
+      placeholder: E.g., Windows 11; Mac OS 10.10 El Capitan
+  - type: input
+    id: python_version
+    attributes:
+      label: What version of Python are you running?
+      description: If you're not sure, open a terminal with the environment you're running spaceKLIP in and type "python --version"
+      placeholder: E.g., Python 3.7
+  - type: textarea
+    id: package_list
+    attributes:
+      label: What Python packages do you have installed?
+      description: To get a full list, open a terminal with the environment you're running spaceKLIP in and type "conda list". Paste the full output here.
+  - type: textarea
+    id: additional-context
+    attributes:
+      label: Additional context or information
+      description: |
+        Provide any additional context or information that might be relevant to this issue.
+      placeholder: Add any additional context or information here.
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to fill out this bug report! After you submit this issue, check this GitHub thread for any updates/responses from the spaceKLIP team.

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1 @@
+blank_issues_enabled: false

.github/ISSUE_TEMPLATE/documentation-template.yml

@@ -0,0 +1,43 @@
+name: Documentation Issue or Request
+description: Report issues or suggest improvements to the spaceKLIP documentation.
+title: "[Documentation]: "
+labels: ["documentation"]
+body:
+  - type: checkboxes
+    id: verify
+    attributes:
+      label: Check Existing Issues
+      description: |
+        Before submitting this issue, please confirm the following:
+      options:
+        - label: Yes, I have checked existing issues to ensure this problem hasn't already been reported.
+          required: true
+  - type: textarea
+    id: problem-description
+    attributes:
+      label: Describe the issue or suggest an improvement
+      description: |
+        Please describe any issues with our documentation or suggest improvements. Your feedback is appreciated!
+      placeholder: Describe the issue or suggest improvements here.
+    validations:
+      required: true
+  - type: textarea
+    id: page-urls
+    attributes:
+      label: URLs of the documentation pages (if applicable)
+      description: |
+        If the issue is specific to particular pages, provide the URLs here.
+        This helps us locate and address the issue more efficiently. You can list multiple URLs if needed.
+      placeholder: Enter the URLs.
+  - type: textarea
+    id: additional-context
+    attributes:
+      label: Additional context or information
+      description: |
+        Provide any additional context or information that might be relevant to this issue or suggestion.
+      placeholder: Add any additional context or information here.
+  - type: markdown
+    attributes:
+      value: |
+        Thank you for helping us improve our documentation! After you submit this issue, check this GitHub thread for any updates/responses from the spaceKLIP team.
+

.github/ISSUE_TEMPLATE/improvement-template.yml

@@ -0,0 +1,53 @@
+name: Enhancement Suggestion
+description: Suggest an enhancement to spaceKLIP
+title: "[Enhancement]: "
+labels: ["enhancement"]
+body:
+  - type: dropdown
+    id: instrument
+    attributes:
+      label: Instrument or Category
+      description: Which instrument or piece of code would you like to see improved?
+      multiple: true
+      options:
+        - NIRCam Stage 1/2 Pipeline
+        - MIRI Stage 1/2 Pipeline
+        - Image Tools
+        - PSF Subtraction
+        - Analysis Tools
+        - Other
+    validations:
+      required: true
+  - type: textarea
+    id: suggestion
+    attributes:
+      label: What is your suggestion?
+      description: What feature would you like to see in spaceKLIP?
+      placeholder: It would be super helpful if spaceKLIP did...
+    validations:
+      required: true
+  - type: textarea
+    id: logs
+    attributes:
+      label: Error traceback output
+      description: If relevant, please copy and paste the full traceback of any errors you encountered.
+  - type: input
+    id: OS
+    attributes:
+      label: What operating system are you using?
+      placeholder: E.g., Windows 11; Mac OS 10.10 El Capitan
+  - type: input
+    id: python_version
+    attributes:
+      label: What version of Python are you running?
+      description: If you're not sure, open a terminal with the environment you're running spaceKLIP in and type "python --version"
+      placeholder: E.g., Python 3.7
+  - type: textarea
+    id: package_list
+    attributes:
+      label: What Python packages do you have installed?
+      description: To get a full list, open a terminal with the environment you're running spaceKLIP in and type "conda list". Paste the full output here.
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to suggest an enhancement! After you submit this issue, check this GitHub thread for any updates/responses from the spaceKLIP team.

.github/workflows/autolabel.yml

@@ -0,0 +1,14 @@
+name: "Set Issue Label and Assignee"
+on:
+  issues:
+    types: [opened]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: Naturalclar/[email protected]
+        with:
+          title-or-body: "both"
+          parameters: '[ {"keywords": ["NIRCam Stage 1/2 Pipeline"], "labels": ["NIRCam"], "assignees": ["AarynnCarter","mperrin","JarronL"]}, {"keywords": ["MIRI Stage 1/2 Pipeline"], "labels": ["MIRI"], "assignees": ["AarynnCarter","mperrin","JarronL"]}, {"keywords": ["Image Tools"], "labels": ["image tools"], "assignees": ["kglidic","kammerje","AarynnCarter","wbalmer","kdlawson","JarronL"]}, {"keywords": ["PSF Subtraction"], "labels": ["PSF subtraction"], "assignees": ["kdlawson","wbalmer","AarynnCarter"]}, {"keywords": ["Analysis Tools"], "labels": ["analysis tools"], "assignees": ["kdlawson","wbalmer","AarynnCarter","juliengirard","kammerje","kglidic"]}, {"keywords": ["Other"], "labels": ["other"], "assignees": ["kdlawson","AarynnCarter","juliengirard","JarronL","kglidic"]}, {"keywords": ["Documentation"], "labels": ["documentation"], "assignees": ["AarynnCarter","mperrin","kglidic"]}]'
+          github-token: "${{ secrets.GITHUB_TOKEN }}"

.gitignore

@@ -213,4 +213,9 @@ tests/ultranest
 tests/species_*
 
 # setuptools-scm
+
+# PyCharm files
+.idea/
+
+
 spaceKLIP/_version.py

README.rst

@@ -6,152 +6,10 @@ SpaceKLIP 🚀🪐
 
 SpaceKLIP is a data reduction pipeline for JWST high-contrast imaging. The current release has been tested with NIRCam and MIRI coronagraphy data.
 
-Compatible test data: `here <https://stsci.box.com/s/0oteh8smujl3pup07hyut6hr4ag1i2el>`_ 
+Installation Instructions: `here <https://spaceklip.readthedocs.io/en/latest/Installation-and-dependencies.html>`_
 
 Documentation: `here <https://spaceklip.readthedocs.io/en/latest/>`_
 
-Installation
-************
+Compatible test data: `here <https://stsci.box.com/s/0oteh8smujl3pup07hyut6hr4ag1i2el>`_
 
-At this stage it is recommended that you clone the Git repository directory for installation:
 
-::
-
-	git clone https://github.com/kammerje/spaceKLIP.git
-
-If you would like to install a specific branch:
-
-::
-
-	git clone https://github.com/kammerje/spaceKLIP.git@branch
-
-From here, it is **highly** recommended that you create a unique Anaconda environment to hold all of the spaceKLIP dependencies:
-
-::
-
-	conda create -n spaceklip python=3.11
-	conda activate spaceklip
-
-With the Anaconda environment created, move to the cloned directory and install the dependencies and spaceKLIP itself:
-
-::
-
-	cd where/you/saved/the/git/repo
-	pip install -r requirements.txt
-	pip install -e .
-
-NEW AS OF 1 JUNE 2023: you also need to make the custom PSF mask files before running spaceKLIP for the first time:
-
-::
-
-	cd spaceKLIP/
-        python make_psfmasks.py
-
-Finally, and very importantly, you will need to download the reference files and set the environment variables supporting the functioning of :code:`webbpsf` and :code:`webbpsf_ext`. Instructions to do this can be found at the respective package websites (`WebbPSF <https://webbpsf.readthedocs.io/en/latest/installation.html#installing-the-required-data-files>`_, `WebbPSF_ext <https://github.com/JarronL/webbpsf_ext>`_). Ensure that if you edit your .bashrc file, close and reopen your terminal to fully apply the changes (:code:`source ~/.bashrc` or :code:`source ~/.zshrc` may also work).
-
-SpaceKLIP also makes use of the JWST Calibration Reference Data System (CRDS) and you will need to set the corresponding environment variables. Follow the instructions here for bash or zsh: https://jwst-crds.stsci.edu/docs/cmdline_bestrefs/. Note that you do not have to install AstroConda, just set the environment variables (making sure that the CRDS path you set actually exists, i.e., you may need to create the directory).
-
-Code status
-***********
-
-The current capabilities of the code are summarized below.
-
-::
-
-	database.py
-
-- ``read_jwst_s012_data``: working
-- ``read_jwst_s3_data``: working
-- ``read_jwst_s4_data``: working
-- ``print_obs``: working
-- ``print_red``: working
-- ``print_src``: working
-- ``update_obs``: working
-- ``update_src``: working
-- ``summarize``: working
-
-::
-
-	plotting.py
-
-- ``display_coron_image``: working
-- ``display_coron_dataset``: working
-
-::
-
-	utils.py
-
-- ``read_obs``: working
-- ``write_obs``: working
-- ``read_red``: working
-
-::
-
-	coron1pipeline.py
-
-- ``run_obs``: working
-- custom saturation step: working
-- custom reference pixel step: needs updating (side reference pixels not working for subarrays)
-
-::
-
-	coron2pipeline.py
-
-- ``run_obs``: working
-- additional outlier detection step: working
-
-::
-
-	coron3pipeline.py
-
-- ``make_asn_file``: working
-- ``run_obs``: working
-
-::
-
-	pyklippipeline.py
-
-- ``run_obs``: working
-- no absolute PSF alignment, this is now done using the ``ImageTools`` library
-- no relative frame alignment, this is now done using the ``ImageTools`` library
-- the old ``JWST.py`` is now implemented here
-- save individual rolls with ``save_rolls = True`` keyword
-
-::
-
-	classpsfsubpipeline.py
-
-- ``run_obs``: working, under development
-- save individual rolls with ``save_rolls = True`` keyword
-
-::
-
-	imagetools.py
-
-- ``remove_frames``: working
-- ``crop_frames``: working
-- ``pad_frames``: working
-- ``coadd_frames``: working
-- ``subtract_median``: working
-- ``subtract_background``: working
-- ``fix_bad_pixels``: working
-- ``replace_nans``: working
-- ``blur_frames``: working
-- ``update_nircam_centers``: working with filter-dependent shifts
-- ``recenter_frames``: working (absolute PSF alignment)
-- ``align_frames``: working (relative frame alignment)
-
-::
-
-	analysistools.py
-
-- ``raw_contrast``: working with ``pyklippipeline`` products, not working with ``coron3pipeline`` products since PSF center not being tracked
-- ``extract_companions``: working with ``pyklippipeline`` products, not working with ``coron3pipeline`` products since PSF center not being tracked
-- combined coronagraphic throughput map: working
-- `` calibrated_contrast`` working with ``pyklippipeline`` products
-
-::
-
-	mast.py
-
-- ``query_coron_datasets``: working

docs/source/Installation-and-dependencies.rst

@@ -18,8 +18,8 @@ dependencies. spaceKLIP is not currently compatible with python 3.12
 
 ::
 
-	conda create -n sklip python=3.11
-	conda activate sklip
+	conda create -n spaceklip python=3.11
+	conda activate spaceklip
 
 With the anaconda environment created, move to the cloned directory and install most of the dependencies:
 
@@ -28,8 +28,7 @@ With the anaconda environment created, move to the cloned directory and install
 	cd where/you/installed/the/git/repo
 	pip install -r requirements.txt
 	pip install -e .
-	cd spaceKLIP/
-	python make_psfmasks.py
 
+Very importantly, you will need to download reference files to support the functioning of the :code:`webbpsf` and :code:`webbpsf_ext`. Instructions to do this can be found at the respective package websites (`WebbPSF <https://webbpsf.readthedocs.io/en/latest/installation.html#installing-the-required-data-files>`_, `webbpsf_ext <https://github.com/JarronL/webbpsf_ext>`_). Ensure that if you edit your .bashrc file that you reopen and close your terminal to fully apply the changes (:code:`source ~/.bashrc` or :code:`source ~/.zshrc` may also work)
 
-Finally, and very importantly, you will need to download reference files to support the functioning of the :code:`webbpsf` and :code:`webbpsf_ext`. Instructions to do this can be found at the respective package websites (`WebbPSF <https://webbpsf.readthedocs.io/en/latest/installation.html#installing-the-required-data-files>`_, `webbpsf_ext <https://github.com/JarronL/webbpsf_ext>`_). Ensure that if you edit your .bashrc file that you reopen and close your terminal to fully apply the changes (:code:`source ~/.bashrc` or :code:`source ~/.zshrc` may also work)
+Finally, spaceKLIP makes use of the JWST data reduction pipeline (:code:`jwst` package) and its dependencies, including the JWST Calibration Reference Data System (CRDS).You will need to set environment variables for CRDS. Instructions are available here in the JWST pipeline docs: https://jwst-pipeline.readthedocs.io/en/latest/getting_started/quickstart.html (Section 3). If you already have a working copy of the JWST data pipeline on your computer, then this is probably already taken care of.

requirements-dev.txt

@@ -1,6 +1,6 @@
 -r requirements.txt
-sphinx
-pandoc
-nbsphinx
-sphinx-rtd-theme
-sphinx_automodapi
+sphinx>=7.2.6
+pandoc>=2.3
+nbsphinx>=0.9.3
+sphinx-rtd-theme>=2.0.0
+sphinx_automodapi>=0.17.0