Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optional Dependencies #967

Merged
merged 13 commits into from
Nov 9, 2023
Merged

Optional Dependencies #967

merged 13 commits into from
Nov 9, 2023

Conversation

swainn
Copy link
Member

@swainn swainn commented Jun 16, 2023 •
edited by sdc50
Loading

This PR makes most Python dependencies optional.

TODO:

  • Tests and Coverage
  • Linting and Black
  • Documentation include instructions for dependency installation
  • Test manual production deployment
  • Update tethys gen metayaml to generate minimal and full versions of the environment
  • Update CI to build micro-tethys-platform package in addition to the "normal/full" build

@swainn swainn requested review from romer8 and removed request for romer8 June 16, 2023 19:30
@coveralls
Copy link

coveralls commented Jun 16, 2023 •
edited
Loading

Coverage Status

coverage: 100.0%. remained the same
when pulling 98b0ca3 on optional_dependencies
into f62cebb on main.

@swainn
Copy link
Member Author

swainn commented Jun 16, 2023 •
edited by sdc50
Loading

Add instructions to the documentation for installing dependencies for the following features:

  • Security Plugins

    • django-cors-headers # enable cors?
    • django-session-security # session timeouts
    • django-axes # tracked failed login attempts / prevent brute-force logins

  • Login/Account Plugins

    • django-gravatar2 # user profile avatars
    • django-simple-captcha
    • django-mfa2 # multifactor support
    • django-recaptcha2
    • social-auth-app-django # authenticate with 3rd-party oauth providers
    • hs_restclient # Used with HydroShare
    • python-jose # required by django-mfa2 - used for onelogin backend
    • django-oauth-toolkit

  • Misc Plugins

    • django-termsandconditions # require users to accept terms and conditions
    • django-analytical # track usage analytics
    • django-json-widget # enable json widget for app settings
    • djangorestframework # enable REST API framework

  • Map Layout Feature

    • PyShp (used by the convert_geojson_to_shapefile() method, but this method is undocumented at the moment)

  • Docker CLI

    • docker-py

  • Database dependencies

    • postgresql
    • psycopg2 # required by tethys_dataset_services
    • sqlalchemy=1.* # TODO: what will it take to support sqlalchemy 2.0?
    • geoalchemy2 # requires sqlalchemy

  • Plotting Gizmo dependencies

    • plotly
    • bokeh<3

  • TethysJob Types

    • dask
    • condorpy
    • tethys_dask_scheduler>=1.0.2

  • External services dependencies

    • tethys_dataset_services>=2.0.0 # used with all data services
    • owslib # used for creating WPS services
    • siphon # used with Threads

  • Datetime dependencies for "humanize" template filter

    • arrow
    • isodate

  • Conda to allow Python API access to Conda Install

    • conda
    • conda-libmamba-solver

  • Add language/tip boxes to tutorials that use features with optional dependencies (e.g.: gee tutorial uses plotly)

    • Bokeh Integration
    • Advanced Concepts (Persistent Stores)
    • Apps Showcase?
    • Dask

  • Mention how to managed optional dependencies in the production deployment

@swainn swainn linked an issue Jun 16, 2023 that may be closed by this pull request
[FEATURE] Make Most Dependencies Optional #911
Closed
@swainn
Copy link
Member Author

swainn commented Aug 21, 2023

Tutorial language: "...this package may not be included in some Tethys installations, so you should include it as a dependency of your app to ensure it is installed."

@sdc50
Copy link
Member

sdc50 commented Sep 22, 2023

@swainn We need to rebase this branch on master. I'm inclined to squash the commits and then rebase and force push. That would require you (and anyone else that has this branch checked out) to re-set it (or delete and re-checkout). Do you have any changes to push before I do that?

@swainn
Copy link
Member Author

swainn commented Sep 22, 2023

@swainn We need to rebase this branch on master. I'm inclined to squash the commits and then rebase and force push. That would require you (and anyone else that has this branch checked out) to re-set it (or delete and re-checkout). Do you have any changes to push before I do that?

Sounds like a good plan to me. I don't have any changes that I need to merge.

@sdc50 sdc50 marked this pull request as ready for review September 26, 2023 00:07
swainn
swainn commented Oct 2, 2023
View reviewed changes
Copy link
Member Author

@swainn swainn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside from my other comments I have one other concern: how are we testing the micro version / optional dependencies. I think we decided that it was too complex to test both ways, but I wonder if there is a straightforward way to modify tests that are affected by an optional dependency. Something like a try: except: that catches the exception raised? Maybe as a decorator so we can easily add it to the tests that need it. Have you tried anything like that?

min_env.yml Outdated Show resolved Hide resolved
environment.yml Outdated Show resolved Hide resolved
.gitignore Outdated Show resolved Hide resolved
docs/installation.rst Show resolved Hide resolved
docs/installation/production/manual/installation.rst Outdated Show resolved Hide resolved
docs/tethys_cli/db.rst Outdated Show resolved Hide resolved
docs/supplementary/optional_features.rst Outdated Show resolved Hide resolved
docs/supplementary/optional_features.rst Outdated Show resolved Hide resolved
docs/supplementary/optional_features.rst Outdated Show resolved Hide resolved
docs/supplementary/optional_features.rst Outdated Show resolved Hide resolved
@sdc50
explore optional dependency implementations
462c641 
make all possible dependencies optional

tests passing

add min env file

add additional_url_patterns

add oauth2_provider

remove warning about user keys

Fix pygments lexer warnings in docs build

Add install dependency tip box to plot gizmos.

black formatting

fix lint

Removed lingering references to the old developer tools page

Add "Starting with Tethys 5.0..."

Update tutorial to require installing plotly

optional dep notes for Tethys Portal configuration
Fixes typo in GRAVATAR_DEFAULT_SECURE setting

Replace deprecated Axes setting in docs.
Note: settings.py is updated in master

docs for social_auth, mf2, and simple_captcha

Reviewed manual production configuration docs for optional args

REST API  and Terms and conditions optional deps

JSON Custom setting optional dependency docs

MapLayout and pyshp optional dependency.

document template tags

most optional dependencies documented
@sdc50
all optional dependencies documented
6711276
@sdc50
black
eb7ae15
@sdc50
tests
8ec683d 
add micro metayaml
update ci
@sdc50
fix tests
8058cb1
sdc50 and others added 8 commits October 14, 2023 20:17
@sdc50
Copy link
Member

sdc50 commented Nov 8, 2023

fixes #980 and fixes #985

@sdc50
Copy link
Member

sdc50 commented Nov 8, 2023

@swainn I've finished the docs for this PR, so it should be ready to go.

swainn
swainn commented Nov 9, 2023
View reviewed changes
Copy link
Member Author

@swainn swainn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I Approve

@swainn swainn merged commit f863b48 into main Nov 9, 2023
10 checks passed
@swainn swainn deleted the optional_dependencies branch November 9, 2023 06:51
This was referenced Nov 9, 2023
Closed
Closed
@mattw-nws mattw-nws mentioned this pull request Jul 2, 2024
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sdc50 sdc50 sdc50 left review comments

Assignees

@swainn swainn

@sdc50 sdc50

Labels
maintain dependencies
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEATURE] Make Most Dependencies Optional
3 participants
@swainn @coveralls @sdc50