⚡️ v2.2.2 Faster dependencies, add @web_page, better REPL, and more. (

#158) # v2.2.2 - **Speed up package installation in virtual environments.** Dynamic dependencies will now be installed via `uv`, which dramatically speeds up installation times. - **Add sub-cards for children pipes.** Pipes with `children` defined now include cards for these pipes under the Parameters menu item. This is especially useful when working managing pipeline hierarchies. - **Add "Open in Python" to pipe cards.** Clicking "Open in Python" on a pipe's card will now launch `ptpython` with the pipe object already created. ```python # Clicking "Open in Python" executes the following: # $ mrsm python "pipe = mrsm.Pipe('plugin:noaa', 'weather', 'gvl', instance='sql:main')" >>> import meerschaum as mrsm >>> pipe = mrsm.Pipe('plugin:noaa', 'weather', 'gvl', instance='sql:main') ``` - **Add the decorators `@web_page` and `@dash_plugin`.** You may now quickly add your own pages to the web console by decorating your layout functions with `@web_page`: ```python # example.py from meerschaum.plugins import dash_plugin, web_page @dash_plugin def init_dash(dash_app): import dash.html as html import dash_bootstrap_components as dbc from dash import Input, Output, no_update @web_page('/my-page', login_required=False) def my_page(): return dbc.Container([ html.H1("Hello, World!"), dbc.Button("Click me", id='my-button'), html.Div(id="my-output-div"), ]) @dash_app.callback( Output('my-output-div', 'children'), Input('my-button', 'n_clicks'), ) def my_button_click(n_clicks): if not n_clicks: return no_update return html.P(f'You clicked {n_clicks} times!') ``` - **Use `ptpython` for the `python` action.** Rather than opening a classic REPL, the `python` action will now open a `ptpython` shell. - **Add `--venv` to the `python` action.** Launching a Python REPL with `mrsm python` will now default to `--venv mrsm`. Run `mrsm install package` to make packages importable. ```python # $ mrsm python >>> import requests Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'requests' # $ mrsm install package requests >>> import requests >>> requests.__file__ '/meerschaum/venvs/mrsm/lib/python3.12/site-packages/requests/__init__.py' # $ mrsm install plugin noaa # $ mrsm python --venv noaa >>> import requests >>> requests.__file__ '/meerschaum/venvs/noaa/lib/python3.12/site-packages/requests/__init__.py' ``` - **Allow passing flags to venv `ptpython` binaries.** You may now pass flags directly to the `ptpython` binary of a virtual environment (by escaping with `[]`): ```bash mrsm python [--help] ``` - **Allow for custom connectors to implement a `sync()` method.** Like module-level `sync()` functions for `plugin` connectors, any custom connector may implement `sync()` instead of `fetch()`. ```python # example.py from typing import Any import meerschaum as mrsm from meerschaum.connectors import Connector, make_connector @make_connector class ExampleConnector(Connector): def register(self, pipe: mrsm.Pipe) -> dict[str, Any]: return { 'columns': { 'datetime': 'ts', 'id': 'example_id', }, } def sync(self, pipe: mrsm.Pipe, **kwargs) -> mrsm.SuccessTuple: ### Implement a custom sync. return True, f"Successfully synced {pipe}!" ``` - **Install `uvicorn` and `gunicorn` in virtual environments.** The packages `uvicorn` and `gunicorn` are now installed into the default virtual environment.
bmeares · Jun 21, 2024 · 4d2f9b4 · 4d2f9b4
1 parent 9a62f98
commit 4d2f9b4
Show file tree

Hide file tree

Showing 42 changed files with 1,123 additions and 299 deletions.
diff --git a/.github/workflows/pytest.yaml b/.github/workflows/pytest.yaml
@@ -18,7 +18,7 @@ jobs:
   build_and_test:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Setup dev tools
         env:
           MRSM_SKIP_DOCKER_EXPERIMENTAL: 1

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,116 @@
 
 This is the current release cycle, so stay tuned for future releases!
 
+### v2.2.2
+
+- **Speed up package installation in virtual environments.**  
+  Dynamic dependencies will now be installed via `uv`, which dramatically speeds up installation times.
+
+- **Add sub-cards for children pipes.**  
+  Pipes with `children` defined now include cards for these pipes under the Parameters menu item. This is especially useful when working managing pipeline hierarchies.
+
+- **Add "Open in Python" to pipe cards.**  
+  Clicking "Open in Python" on a pipe's card will now launch `ptpython` with the pipe object already created.
+
+  ```python
+  # Clicking "Open in Python" executes the following:
+  # $ mrsm python "pipe = mrsm.Pipe('plugin:noaa', 'weather', 'gvl', instance='sql:main')"
+  >>> import meerschaum as mrsm
+  >>> pipe = mrsm.Pipe('plugin:noaa', 'weather', 'gvl', instance='sql:main')
+  ```
+
+- **Add the decorators `@web_page` and `@dash_plugin`.**  
+  You may now quickly add your own pages to the web console by decorating your layout functions with `@web_page`:
+
+  ```python
+  # example.py
+  from meerschaum.plugins import dash_plugin, web_page
+
+  @dash_plugin
+  def init_dash(dash_app):
+
+      import dash.html as html
+      import dash_bootstrap_components as dbc
+      from dash import Input, Output, no_update
+
+      @web_page('/my-page', login_required=False)
+      def my_page():
+          return dbc.Container([
+              html.H1("Hello, World!"),
+              dbc.Button("Click me", id='my-button'),
+              html.Div(id="my-output-div"),
+          ])
+
+      @dash_app.callback(
+          Output('my-output-div', 'children'),
+          Input('my-button', 'n_clicks'),
+      )
+      def my_button_click(n_clicks):
+          if not n_clicks:
+              return no_update
+          return html.P(f'You clicked {n_clicks} times!')
+  ```
+
+- **Use `ptpython` for the `python` action.**  
+  Rather than opening a classic REPL, the `python` action will now open a `ptpython` shell.
+
+- **Add `--venv` to the `python` action.**  
+  Launching a Python REPL with `mrsm python` will now default to `--venv mrsm`. Run `mrsm install package` to make packages importable.
+
+  ```python
+  # $ mrsm python
+  >>> import requests
+  Traceback (most recent call last):
+    File "<stdin>", line 1, in <module>
+  ModuleNotFoundError: No module named 'requests'
+
+  # $ mrsm install package requests
+  >>> import requests
+  >>> requests.__file__
+  '/meerschaum/venvs/mrsm/lib/python3.12/site-packages/requests/__init__.py'
+
+  # $ mrsm install plugin noaa
+  # $ mrsm python --venv noaa
+  >>> import requests
+  >>> requests.__file__
+  '/meerschaum/venvs/noaa/lib/python3.12/site-packages/requests/__init__.py'
+  ``` 
+
+- **Allow passing flags to venv `ptpython` binaries.**  
+  You may now pass flags directly to the `ptpython` binary of a virtual environment (by escaping with `[]`):
+
+  ```bash
+  mrsm python [--help]
+  ```
+
+- **Allow for custom connectors to implement a `sync()` method.**  
+  Like module-level `sync()` functions for `plugin` connectors, any custom connector may implement `sync()` instead of `fetch()`.
+
+  ```python
+  # example.py
+  from typing import Any
+  import meerschaum as mrsm
+  from meerschaum.connectors import Connector, make_connector
+
+  @make_connector
+  class ExampleConnector(Connector):
+
+      def register(self, pipe: mrsm.Pipe) -> dict[str, Any]:
+          return {
+              'columns': {
+                  'datetime': 'ts',
+                  'id': 'example_id',
+              },
+          }
+
+      def sync(self, pipe: mrsm.Pipe, **kwargs) -> mrsm.SuccessTuple:
+          ### Implement a custom sync.
+          return True, f"Successfully synced {pipe}!"
+  ```
+
+- **Install `uvicorn` and `gunicorn` in virtual environments.**  
+  The packages `uvicorn` and `gunicorn` are now installed into the default virtual environment.
+
 ### v2.2.1
 
 - **Fix `--schedule` in the interactive shell.**  

diff --git a/Dockerfile b/Dockerfile
@@ -30,7 +30,8 @@ WORKDIR $MRSM_WORK_DIR
 ### Layer 2: Install Python packages.
 ### Only rebuilds cache if dependencies have changed.
 COPY --chown=$MRSM_USER:$MRSM_USER requirements $MRSM_HOME/requirements
-RUN python -m pip install --user --no-cache-dir -r $MRSM_HOME/requirements/$MRSM_DEP_GROUP.txt && \
+RUN python -m pip install --user --no-cache-dir \
+  -r $MRSM_HOME/requirements/$MRSM_DEP_GROUP.txt && \
   rm -rf $MRSM_HOME/requirements
 
 ### Layer 3: Install Meerschaum.

diff --git a/docker-compose.yaml b/docker-compose.yaml
@@ -1,6 +1,8 @@
 services:
   mrsm-dev:
-    image: "bmeares/meerschaum"
+    image: "bmeares/meerschaum:minimal"
+    # image: ubuntu
+    # image: python
     entrypoint: ["/scripts/dev.sh"]
     network_mode: "host"
     init: true

diff --git a/docs/mkdocs/news/changelog.md b/docs/mkdocs/news/changelog.md
@@ -4,6 +4,116 @@
 
 This is the current release cycle, so stay tuned for future releases!
 
+### v2.2.2
+
+- **Speed up package installation in virtual environments.**  
+  Dynamic dependencies will now be installed via `uv`, which dramatically speeds up installation times.
+
+- **Add sub-cards for children pipes.**  
+  Pipes with `children` defined now include cards for these pipes under the Parameters menu item. This is especially useful when working managing pipeline hierarchies.
+
+- **Add "Open in Python" to pipe cards.**  
+  Clicking "Open in Python" on a pipe's card will now launch `ptpython` with the pipe object already created.
+
+  ```python
+  # Clicking "Open in Python" executes the following:
+  # $ mrsm python "pipe = mrsm.Pipe('plugin:noaa', 'weather', 'gvl', instance='sql:main')"
+  >>> import meerschaum as mrsm
+  >>> pipe = mrsm.Pipe('plugin:noaa', 'weather', 'gvl', instance='sql:main')
+  ```
+
+- **Add the decorators `@web_page` and `@dash_plugin`.**  
+  You may now quickly add your own pages to the web console by decorating your layout functions with `@web_page`:
+
+  ```python
+  # example.py
+  from meerschaum.plugins import dash_plugin, web_page
+
+  @dash_plugin
+  def init_dash(dash_app):
+
+      import dash.html as html
+      import dash_bootstrap_components as dbc
+      from dash import Input, Output, no_update
+
+      @web_page('/my-page', login_required=False)
+      def my_page():
+          return dbc.Container([
+              html.H1("Hello, World!"),
+              dbc.Button("Click me", id='my-button'),
+              html.Div(id="my-output-div"),
+          ])
+
+      @dash_app.callback(
+          Output('my-output-div', 'children'),
+          Input('my-button', 'n_clicks'),
+      )
+      def my_button_click(n_clicks):
+          if not n_clicks:
+              return no_update
+          return html.P(f'You clicked {n_clicks} times!')
+  ```
+
+- **Use `ptpython` for the `python` action.**  
+  Rather than opening a classic REPL, the `python` action will now open a `ptpython` shell.
+
+- **Add `--venv` to the `python` action.**  
+  Launching a Python REPL with `mrsm python` will now default to `--venv mrsm`. Run `mrsm install package` to make packages importable.
+
+  ```python
+  # $ mrsm python
+  >>> import requests
+  Traceback (most recent call last):
+    File "<stdin>", line 1, in <module>
+  ModuleNotFoundError: No module named 'requests'
+
+  # $ mrsm install package requests
+  >>> import requests
+  >>> requests.__file__
+  '/meerschaum/venvs/mrsm/lib/python3.12/site-packages/requests/__init__.py'
+
+  # $ mrsm install plugin noaa
+  # $ mrsm python --venv noaa
+  >>> import requests
+  >>> requests.__file__
+  '/meerschaum/venvs/noaa/lib/python3.12/site-packages/requests/__init__.py'
+  ``` 
+
+- **Allow passing flags to venv `ptpython` binaries.**  
+  You may now pass flags directly to the `ptpython` binary of a virtual environment (by escaping with `[]`):
+
+  ```bash
+  mrsm python [--help]
+  ```
+
+- **Allow for custom connectors to implement a `sync()` method.**  
+  Like module-level `sync()` functions for `plugin` connectors, any custom connector may implement `sync()` instead of `fetch()`.
+
+  ```python
+  # example.py
+  from typing import Any
+  import meerschaum as mrsm
+  from meerschaum.connectors import Connector, make_connector
+
+  @make_connector
+  class ExampleConnector(Connector):
+
+      def register(self, pipe: mrsm.Pipe) -> dict[str, Any]:
+          return {
+              'columns': {
+                  'datetime': 'ts',
+                  'id': 'example_id',
+              },
+          }
+
+      def sync(self, pipe: mrsm.Pipe, **kwargs) -> mrsm.SuccessTuple:
+          ### Implement a custom sync.
+          return True, f"Successfully synced {pipe}!"
+  ```
+
+- **Install `uvicorn` and `gunicorn` in virtual environments.**  
+  The packages `uvicorn` and `gunicorn` are now installed into the default virtual environment.
+
 ### v2.2.1
 
 - **Fix `--schedule` in the interactive shell.**  

diff --git a/docs/mkdocs/reference/plugins/writing-plugins.md b/docs/mkdocs/reference/plugins/writing-plugins.md
@@ -94,6 +94,8 @@ Plugins are just modules with functions. This section explains the roles of the
   Create new commands.
 - **`#!python @api_plugin`**  
   Create new FastAPI endpoints.
+- **`#!python @dash_plugin` and `#!python @web_page`**  
+  Create new Dash pages on the Web Console.
 - **`#!python @pre_sync_hook` and `#!python @post_sync_hook`**  
   Inject callbacks when pipes are synced by the `sync pipes` action.
 - **`#!python setup(**kwargs)`**  
@@ -445,6 +447,89 @@ For your endpoints, arguments will be used as HTTP parameters, and to require th
 
     ![Custom Meerschaum API endpoint which requires a login.](/assets/screenshots/api-plugin-endpoint-login-required.png)
 
+### **The `#!python @dash_plugin` and `#!python @web_page` Decorators**
+
+Add new [Plotly Dash](https://dash.plotly.com/) pages to the Web Console with `#!python @dash_plugin` and `#!python @web_page`.
+
+Like `#!python @api_plugin`, `#!python @dash_app` designates an initialization function which accepts the Dash app. Within this function, you can create callbacks with `#!python @dash_app.callback()`.
+
+!!! note inline end ""
+
+    `#!python @web_page` may be used with or without arguments. When invoked without arguments, it will use the name of the function as the path string.
+
+Functions decorated with `#!python @web_page` return a [Dash layout](https://dash.plotly.com/layout). These don't need to reside within the `#!python @dash_app`-decorated function, but this is recommended to improve lazy-loading performance.
+
+??? example "Dash Plugin Example"
+
+    ```python
+    ### ~/.config/meerschaum/plugins/example.py
+
+    from meerschaum.plugins import dash_plugin, web_page
+
+    @dash_plugin
+    def init_dash(dash_app):
+
+        import dash.html as html
+        import dash_bootstrap_components as dbc
+        from dash import Input, Output, no_update
+
+        ### Routes to '/dash/my-page'
+        @web_page('/my-page', login_required=False)
+        def my_page():
+            return dbc.Container([
+                html.H1("Hello, World!"),
+                dbc.Button("Click me", id='my-button'),
+                html.Div(id="my-output-div"),
+            ])
+
+        @dash_app.callback(
+            Output('my-output-div', 'children'),
+            Input('my-button', 'n_clicks'),
+        )
+        def my_button_click(n_clicks):
+            if not n_clicks:
+                return no_update
+            return html.P(f'You clicked {n_clicks} times!')
+    ```
+
+??? tip "Dynamic Routing"
+
+    Sub-paths will be routed to your base callback, e.g. `/items/apple` will be accepted by `#!python @web_page('/items')`. You can then parse the path string with [`dcc.Location`](https://dash.plotly.com/dash-core-components/location).
+
+    ```python
+    from meerschaum.plugins import web_page, dash_plugin
+
+    @dash_plugin
+    def init_dash(dash_app):
+
+        import dash.html as html
+        import dash.dcc as dcc
+        from dash import Input, Output, State, no_update
+        import dash_bootstrap_components as dbc
+
+        ### Omitting arguments will default to the path '/dash/items'.
+        ### `login_required` is `True` by default.
+        @web_page
+        def items():
+            return dbc.Container([
+                dcc.Location(id='my-location'),
+                html.Div(id='my-output-div'),
+            ])
+
+        @dash_app.callback(
+            Output('my-output-div', 'children'),
+            Input('my-location', 'pathname'),
+        )
+        def my_button_click(pathname):
+            if not str(pathname).startswith('/dash/items'):
+                return no_update
+
+            item_id = pathname.replace('/dash/items', '').lstrip('/').rstrip('/').split('/')[0]
+            if not item_id:
+                return html.P("You're looking at the base /items page.")
+
+            return html.P(f"You're looking at item '{item_id}'.")
+    ```
 
 ### **The `#!python @pre_sync_hook` and `#!python @post_sync_hook` Decorators**