DSE-38028 - Rebase JupyterAI to v2.19.1 (#6)

* Fix plugin ID format (#872) * Fix incorrect plugin ID * Update plugin ID for inline completions * Remove unused toolbars/menus from schema (#873) * Do not redefine `refreshCompleterState` on each render (#875) * Limit chat input height to 20 lines (#877) * Limit the number of displayed rows in chat input to 20 * affix send button to bottom of chat input --------- Co-authored-by: David L. Qiu <[email protected]> * Address error on `/learn` after change of embedding model (#870) * Address error on `/learn` after change of embedding model. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Set `all_files` option to default to False * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix prefix removal when streaming inline completions (#879) * Fix removal of prefix when streaming * Fix typo Co-authored-by: david qiu <[email protected]> --------- Co-authored-by: david qiu <[email protected]> * Add keyboard shortcut command to focus chat input (#876) * Focus chat input with keyboard * Focus chat input with keyboard * Rename `inputFocusRequested` to `focusInputSignal` * Make sure that the shortcut does not interrupt default action * Make preferred-dir the default read/write directory for slash commands (#881) * add preferred dir functionality to BaseChatHandler * remove print * cleanup * Change /learn to use _output_dir instead of root dir * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add preferred dir functionality to BaseChatHandler * remove print * cleanup * Change /learn to use _output_dir instead of root dir * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change _output_dir to output_dir * Update packages/jupyter-ai/jupyter_ai/chat_handlers/base.py Co-authored-by: david qiu <[email protected]> * add type hint --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: david qiu <[email protected]> * Ensure all slash commands support `-h/--help` (#878) * Fix help for slash commands * Do not add `-h` to the help command itself Co-authored-by: david qiu <[email protected]> * Fix typo Co-authored-by: david qiu <[email protected]> * Centralize handling of `-h`/`--help`, with opt-out via class attribute by default the `/help` and "default" handlers opt-out from `-h` handling * Add colon delimiter and wrap the arguments in backticks * Use Title case for help messages * Remove final leftover * Rename `display_arguments_help` to `supports_help` --------- Co-authored-by: david qiu <[email protected]> * Ensuring restricted access to the `/learn` index directory (#887) * Add a test ensuring correct mode of index dir * Ensure dir exists with correct mode * Add Ollama (#646) * Add Ollama * mistral:text for embeddings * Add gemma * Update the list of models Co-authored-by: Bc <[email protected]> * Mention Ollama in the docs * Apply suggestions from code review Co-authored-by: Piyush Jain <[email protected]> Co-authored-by: david qiu <[email protected]> --------- Co-authored-by: Bc <[email protected]> Co-authored-by: Piyush Jain <[email protected]> Co-authored-by: david qiu <[email protected]> * Updated integration tests workflow (#890) * Update to `actions/checkout@v4` (#893) * Upload `jupyter-releaser` built distributions (#892) * Upload `jupyter-releaser` built distributions * Try fix paths * fix path * Fix removal of pending messages on error (#888) * Add a test case for closing pending messages * Remove pending message on error * Add a test for not sending closing message twice * Review: make `close_pending` idempotent * Fix streaming in `HuggingFaceHub` provider (#894) the overridden method is no longer overriding allowed task list since #784 but is only breaking streaming - it can be removed * Rework selection replacement (#895) * remove replace selection checkbox * add useCopy() hook * fixup * add useReplace() hook * add menu for replacement & insertion options on agent messages * show 'replace 1 line' instead of 'replace 1 lines' * Upgrade to `langchain~=0.2.0` and `langchain_community~=0.2.0` (#897) * upgrade to langchain~=0.2.0 and langchain_community~=0.2.0 * lower version floor to allow for langchain v0.1 Co-authored-by: Michał Krassowski <[email protected]> --------- Co-authored-by: Michał Krassowski <[email protected]> * Publish 2.19.0 SHA256 hashes: jupyter-ai-core-2.19.0.tgz: f299f1a94c386b88359e7a240450e5e2ea74c51ebe82346c2566def204e7c5a0 jupyter_ai-2.19.0-py3-none-any.whl: d03540ea42b181946ccad7287b4306bd8f37af09d9f7b3e45b0adc0eb0449b34 jupyter_ai-2.19.0.tar.gz: 746220ac7205f57047cb8c7bb1c94c7a02cdbad9af0c534762cd06168cd66b8e jupyter_ai_magics-2.19.0-py3-none-any.whl: b9ac48e70d7c452f739289e588d89ebbc1abbfd37b7e6f8ea085da7169052293 jupyter_ai_magics-2.19.0.tar.gz: a0a078ee9c533e576769ce1330f2571eda5c3e5d3f3ce74ee249126d6c3c9eb5 * [pre-commit.ci] pre-commit autoupdate (#416) * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/pre-commit/pre-commit-hooks: v4.5.0 → v4.6.0](pre-commit/pre-commit-hooks@v4.5.0...v4.6.0) - [github.com/psf/black: 23.9.1 → 24.4.2](psf/black@23.9.1...24.4.2) - [github.com/PyCQA/isort: 5.12.0 → 5.13.2](PyCQA/isort@5.12.0...5.13.2) - [github.com/asottile/pyupgrade: v3.15.0 → v3.16.0](asottile/pyupgrade@v3.15.0...v3.16.0) - [github.com/pycqa/flake8: 6.1.0 → 7.1.0](PyCQA/flake8@6.1.0...7.1.0) - [github.com/sirosen/check-jsonschema: 0.27.0 → 0.29.0](python-jsonschema/check-jsonschema@0.27.0...0.29.0) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * update docs to reflect Python 3.12 support (#898) * Fix python<3.10 compatibility (#899) * Trim leading whitespace when processing (#900) * Make magic aliases user-customizable (#901) * Allow overriding the Ollama base URL (#904) * Add notebooks to the documentation (#906) * Add notebooks to the documentation * pin docutils * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: David L. Qiu <[email protected]> * Publish 2.19.1 SHA256 hashes: jupyter-ai-core-2.19.1.tgz: a2ca5466579fce1a63806f440ec40095ba096ccea8531195b93eeac2a361ad9d jupyter_ai-2.19.1-py3-none-any.whl: 9521cdfaada7c2e9c660cdbd7084235ebfaa7dfbf4f734f639f88fa892661e74 jupyter_ai-2.19.1.tar.gz: 4a4f61411d41ac16bc5838970ed4c54c2dd64c9b56d9277430c6a83ebd792f3d jupyter_ai_magics-2.19.1-py3-none-any.whl: bb6ab5287378869dd5fa5f9ef5e90aa40c6300a628fb2147b9e6da1c7972681c jupyter_ai_magics-2.19.1.tar.gz: bf2452b4a40b13be5e325b0abc3ac8a7575d1c8377e7311571c7d05028c3305f * Added new models (#908) Added the latest llama-3.1 models available on Amazon Bedrock, and gpt-4o mini. All these models were recently released and have become popular. * Fix merge issue. --------- Co-authored-by: Michał Krassowski <[email protected]> Co-authored-by: David L. Qiu <[email protected]> Co-authored-by: Sanjiv Das <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Andrew Fulton <[email protected]> Co-authored-by: Jeremy Tuloup <[email protected]> Co-authored-by: Bc <[email protected]> Co-authored-by: Piyush Jain <[email protected]> Co-authored-by: dlqqq <[email protected]> Co-authored-by: michaelchia <[email protected]>
cloudera · Jul 26, 2024 · 0de60bd · 0de60bd
1 parent fdf642f
commit 0de60bd
Show file tree

Hide file tree

Showing 68 changed files with 1,003 additions and 288 deletions.
diff --git a/.github/workflows/check-release.yml b/.github/workflows/check-release.yml
@@ -13,7 +13,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
 
       - name: Setup environment
         uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
@@ -26,6 +26,11 @@ jobs:
           token: ${{ secrets.GITHUB_TOKEN }}
           version_spec: minor
 
+      - name: Upload Distributions
+        uses: actions/upload-artifact@v4
+        with:
+          name: jupyter-ai-jupyter-releaser-dist-${{ github.run_number }}
+          path: .jupyter_releaser_checkout/dist
 
       - name: Runner debug info
         if: always()

diff --git a/.github/workflows/e2e-tests.yml b/.github/workflows/e2e-tests.yml
@@ -20,7 +20,7 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
 
       - name: Base Setup
         uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
@@ -7,7 +7,7 @@ jobs:
     name: Lint TypeScript source
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
         with:
           python_version: "3.11.x"
@@ -22,7 +22,7 @@ jobs:
     name: Lint Python imports
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Echo environment details
         run: |
           which python

diff --git a/.github/workflows/unit-tests.yml b/.github/workflows/unit-tests.yml
@@ -16,7 +16,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
 
       - name: Base Setup
         uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

diff --git a/.github/workflows/update-snapshots.yml b/.github/workflows/update-snapshots.yml
@@ -3,34 +3,67 @@ name: Update Playwright Snapshots
 on:
   issue_comment:
     types: [created, edited]
-  workflow_dispatch:
-    inputs:
-      number:
-        description: 'PR number'
-        required: true
 
 permissions:
   contents: write
   pull-requests: write
 
 jobs:
   update-snapshots:
-    if: ${{ github.event.inputs || (github.event.issue.pull_request && contains(github.event.comment.body, 'please update playwright snapshots')) }}
+    if: >
+      (
+        github.event.issue.author_association == 'OWNER' ||
+        github.event.issue.author_association == 'COLLABORATOR' ||
+        github.event.issue.author_association == 'MEMBER'
+      ) && github.event.issue.pull_request && contains(github.event.comment.body, 'please update snapshots')
     runs-on: ubuntu-latest
 
     steps:
+      - name: React to the triggering comment
+        run: |
+          gh api repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions --raw-field 'content=+1'
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           token: ${{ secrets.GITHUB_TOKEN }}
 
-      - name: Configure git to use https
-        run: git config --global hub.protocol https
+      - name: Get PR Info
+        id: pr
+        env:
+          PR_NUMBER: ${{ github.event.issue.number }}
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GH_REPO: ${{ github.repository }}
+          COMMENT_AT: ${{ github.event.comment.created_at }}
+        run: |
+          pr="$(gh api /repos/${GH_REPO}/pulls/${PR_NUMBER})"
+          head_sha="$(echo "$pr" | jq -r .head.sha)"
+          pushed_at="$(echo "$pr" | jq -r .pushed_at)"
+
+          if [[ $(date -d "$pushed_at" +%s) -gt $(date -d "$COMMENT_AT" +%s) ]]; then
+              echo "Updating is not allowed because the PR was pushed to (at $pushed_at) after the triggering comment was issued (at $COMMENT_AT)"
+              exit 1
+          fi
+
+          echo "head_sha=$head_sha" >> $GITHUB_OUTPUT
 
       - name: Checkout the branch from the PR that triggered the job
-        run: hub pr checkout ${{ github.event.inputs.number || github.event.issue.number }}
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: gh pr checkout ${{ github.event.issue.number }}
+
+      - name: Validate the fetched branch HEAD revision
+        env:
+          EXPECTED_SHA: ${{ steps.pr.outputs.head_sha }}
+        run: |
+          actual_sha="$(git rev-parse HEAD)"
+
+          if [[ "$actual_sha" != "$EXPECTED_SHA" ]]; then
+              echo "The HEAD of the checked out branch ($actual_sha) differs from the HEAD commit available at the time when trigger comment was submitted ($EXPECTED_SHA)"
+              exit 1
+          fi
 
       - name: Base Setup
         uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.5.0
+    rev: v4.6.0
     hooks:
       - id: end-of-file-fixer
       - id: check-case-conflict
@@ -18,25 +18,25 @@ repos:
       - id: trailing-whitespace
 
   - repo: https://github.com/psf/black
-    rev: 23.9.1
+    rev: 24.4.2
     hooks:
       - id: black
 
   - repo: https://github.com/PyCQA/isort
-    rev: 5.12.0
+    rev: 5.13.2
     hooks:
       - id: isort
         args: ["--profile", "black"]
         files: \.py$
 
   - repo: https://github.com/asottile/pyupgrade
-    rev: v3.15.0
+    rev: v3.16.0
     hooks:
       - id: pyupgrade
         args: [--py37-plus]
 
   - repo: https://github.com/pycqa/flake8
-    rev: 6.1.0
+    rev: 7.1.0
     hooks:
       - id: flake8
         additional_dependencies:
@@ -48,7 +48,7 @@ repos:
         stages: [manual]
 
   - repo: https://github.com/sirosen/check-jsonschema
-    rev: 0.27.0
+    rev: 0.29.0
     hooks:
       - id: check-jsonschema
         name: "Check GitHub Workflows"

diff --git a/.readthedocs.yaml b/.readthedocs.yaml
@@ -8,6 +8,8 @@ build:
   os: ubuntu-22.04
   tools:
     python: "3.11"
+  apt_packages:
+    - pandoc
 
 sphinx:
   configuration: docs/source/conf.py

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,78 @@
 
 <!-- <START NEW CHANGELOG ENTRY> -->
 
+## 2.19.1
+
+([Full Changelog](https://github.com/jupyterlab/jupyter-ai/compare/@jupyter-ai/[email protected]))
+
+### Enhancements made
+
+- Allow overriding the Ollama base URL [#904](https://github.com/jupyterlab/jupyter-ai/pull/904) ([@jtpio](https://github.com/jtpio))
+- Make magic aliases user-customizable [#901](https://github.com/jupyterlab/jupyter-ai/pull/901) ([@krassowski](https://github.com/krassowski))
+
+### Bugs fixed
+
+- Trim leading whitespace when processing [#900](https://github.com/jupyterlab/jupyter-ai/pull/900) ([@krassowski](https://github.com/krassowski))
+- Fix python\<3.10 compatibility [#899](https://github.com/jupyterlab/jupyter-ai/pull/899) ([@michaelchia](https://github.com/michaelchia))
+
+### Maintenance and upkeep improvements
+
+### Documentation improvements
+
+- Add notebooks to the documentation [#906](https://github.com/jupyterlab/jupyter-ai/pull/906) ([@andrewfulton9](https://github.com/andrewfulton9))
+- Update docs to reflect Python 3.12 support [#898](https://github.com/jupyterlab/jupyter-ai/pull/898) ([@dlqqq](https://github.com/dlqqq))
+
+### Contributors to this release
+
+([GitHub contributors page for this release](https://github.com/jupyterlab/jupyter-ai/graphs/contributors?from=2024-07-15&to=2024-07-22&type=c))
+
+[@andrewfulton9](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Aandrewfulton9+updated%3A2024-07-15..2024-07-22&type=Issues) | [@dlqqq](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Adlqqq+updated%3A2024-07-15..2024-07-22&type=Issues) | [@jtpio](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Ajtpio+updated%3A2024-07-15..2024-07-22&type=Issues) | [@krassowski](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Akrassowski+updated%3A2024-07-15..2024-07-22&type=Issues) | [@michaelchia](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Amichaelchia+updated%3A2024-07-15..2024-07-22&type=Issues) | [@pre-commit-ci](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Apre-commit-ci+updated%3A2024-07-15..2024-07-22&type=Issues)
+
+<!-- <END NEW CHANGELOG ENTRY> -->
+
+## 2.19.0
+
+This is a significant release that implements LLM response streaming in Jupyter AI along with several other enhancements & fixes listed below. Special thanks to @krassowski for his generous contributions this release!
+
+([Full Changelog](https://github.com/jupyterlab/jupyter-ai/compare/@jupyter-ai/[email protected]))
+
+### Enhancements made
+
+- Upgrade to `langchain~=0.2.0` and `langchain_community~=0.2.0` [#897](https://github.com/jupyterlab/jupyter-ai/pull/897) ([@dlqqq](https://github.com/dlqqq))
+- Rework selection replacement [#895](https://github.com/jupyterlab/jupyter-ai/pull/895) ([@dlqqq](https://github.com/dlqqq))
+- Ensure all slash commands support `-h/--help` [#878](https://github.com/jupyterlab/jupyter-ai/pull/878) ([@krassowski](https://github.com/krassowski))
+- Add keyboard shortcut command to focus chat input [#876](https://github.com/jupyterlab/jupyter-ai/pull/876) ([@krassowski](https://github.com/krassowski))
+- Implement LLM response streaming [#859](https://github.com/jupyterlab/jupyter-ai/pull/859) ([@dlqqq](https://github.com/dlqqq))
+- Add Ollama [#646](https://github.com/jupyterlab/jupyter-ai/pull/646) ([@jtpio](https://github.com/jtpio))
+
+### Bugs fixed
+
+- Fix streaming in `HuggingFaceHub` provider [#894](https://github.com/jupyterlab/jupyter-ai/pull/894) ([@krassowski](https://github.com/krassowski))
+- Fix removal of pending messages on error [#888](https://github.com/jupyterlab/jupyter-ai/pull/888) ([@krassowski](https://github.com/krassowski))
+- Ensuring restricted access to the `/learn` index directory [#887](https://github.com/jupyterlab/jupyter-ai/pull/887) ([@krassowski](https://github.com/krassowski))
+- Make preferred-dir the default read/write directory for slash commands [#881](https://github.com/jupyterlab/jupyter-ai/pull/881) ([@andrewfulton9](https://github.com/andrewfulton9))
+- Fix prefix removal when streaming inline completions [#879](https://github.com/jupyterlab/jupyter-ai/pull/879) ([@krassowski](https://github.com/krassowski))
+- Limit chat input height to 20 lines [#877](https://github.com/jupyterlab/jupyter-ai/pull/877) ([@krassowski](https://github.com/krassowski))
+- Do not redefine `refreshCompleterState` on each render [#875](https://github.com/jupyterlab/jupyter-ai/pull/875) ([@krassowski](https://github.com/krassowski))
+- Remove unused toolbars/menus from schema [#873](https://github.com/jupyterlab/jupyter-ai/pull/873) ([@krassowski](https://github.com/krassowski))
+- Fix plugin ID format [#872](https://github.com/jupyterlab/jupyter-ai/pull/872) ([@krassowski](https://github.com/krassowski))
+- Address error on `/learn` after change of embedding model [#870](https://github.com/jupyterlab/jupyter-ai/pull/870) ([@srdas](https://github.com/srdas))
+- Fix pending message overlapping text [#857](https://github.com/jupyterlab/jupyter-ai/pull/857) ([@michaelchia](https://github.com/michaelchia))
+- Fixes error when allowed or blocked model list is passed in config [#855](https://github.com/jupyterlab/jupyter-ai/pull/855) ([@3coins](https://github.com/3coins))
+- Fixed `/export` for timestamp, agent name [#854](https://github.com/jupyterlab/jupyter-ai/pull/854) ([@srdas](https://github.com/srdas))
+
+### Maintenance and upkeep improvements
+
+- Update to `actions/checkout@v4` [#893](https://github.com/jupyterlab/jupyter-ai/pull/893) ([@jtpio](https://github.com/jtpio))
+- Upload `jupyter-releaser` built distributions [#892](https://github.com/jupyterlab/jupyter-ai/pull/892) ([@jtpio](https://github.com/jtpio))
+- Updated integration tests workflow [#890](https://github.com/jupyterlab/jupyter-ai/pull/890) ([@krassowski](https://github.com/krassowski))
+
+### Contributors to this release
+
+([GitHub contributors page for this release](https://github.com/jupyterlab/jupyter-ai/graphs/contributors?from=2024-06-21&to=2024-07-15&type=c))
+
+[@3coins](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3A3coins+updated%3A2024-06-21..2024-07-15&type=Issues) | [@andrewfulton9](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Aandrewfulton9+updated%3A2024-06-21..2024-07-15&type=Issues) | [@brichet](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Abrichet+updated%3A2024-06-21..2024-07-15&type=Issues) | [@dannongruver](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Adannongruver+updated%3A2024-06-21..2024-07-15&type=Issues) | [@dlqqq](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Adlqqq+updated%3A2024-06-21..2024-07-15&type=Issues) | [@JasonWeill](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3AJasonWeill+updated%3A2024-06-21..2024-07-15&type=Issues) | [@jtpio](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Ajtpio+updated%3A2024-06-21..2024-07-15&type=Issues) | [@krassowski](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Akrassowski+updated%3A2024-06-21..2024-07-15&type=Issues) | [@lalanikarim](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Alalanikarim+updated%3A2024-06-21..2024-07-15&type=Issues) | [@michaelchia](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Amichaelchia+updated%3A2024-06-21..2024-07-15&type=Issues) | [@pedrogutobjj](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Apedrogutobjj+updated%3A2024-06-21..2024-07-15&type=Issues) | [@srdas](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Asrdas+updated%3A2024-06-21..2024-07-15&type=Issues)
+
 ## 2.18.1
 
 ([Full Changelog](https://github.com/jupyterlab/jupyter-ai/compare/@jupyter-ai/[email protected]))
@@ -22,8 +94,6 @@
 
 [@dlqqq](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Adlqqq+updated%3A2024-06-20..2024-06-21&type=Issues) | [@srdas](https://github.com/search?q=repo%3Ajupyterlab%2Fjupyter-ai+involves%3Asrdas+updated%3A2024-06-20..2024-06-21&type=Issues)
 
-<!-- <END NEW CHANGELOG ENTRY> -->
-
 ## 2.18.0
 
 ([Full Changelog](https://github.com/jupyterlab/jupyter-ai/compare/@jupyter-ai/[email protected]))

diff --git a/README.md b/README.md
@@ -25,7 +25,7 @@ in JupyterLab and the Jupyter Notebook. More specifically, Jupyter AI offers:
 * A native chat UI in JupyterLab that enables you to work with generative AI as a conversational assistant.
 * Support for a wide range of generative model providers, including AI21, Anthropic, AWS, Cohere,
   Gemini, Hugging Face, MistralAI, NVIDIA, and OpenAI.
-* Local model support through GPT4All, enabling use of generative AI models on consumer grade machines
+* Local model support through GPT4All and Ollama, enabling use of generative AI models on consumer grade machines
   with ease and privacy.
 
 Documentation is available on [ReadTheDocs](https://jupyter-ai.readthedocs.io/en/latest/).
@@ -36,7 +36,7 @@ Documentation is available on [ReadTheDocs](https://jupyter-ai.readthedocs.io/en
 
 You will need to have installed the following software to use Jupyter AI:
 
-- Python 3.8 - 3.11
+- Python 3.8 - 3.12
 - JupyterLab 4 or Notebook 7
 
 In addition, you will need access to at least one model provider.

diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -1,2 +1,6 @@
+# nbsphinx-link requires older version of docutils
+docutils==0.20
 myst_parser
+nbsphinx
+nbsphinx-link
 pydata_sphinx_theme
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -13,7 +13,7 @@
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
 
-extensions = ["myst_parser"]
+extensions = ["myst_parser", "nbsphinx", "nbsphinx_link"]
 myst_enable_extensions = ["colon_fence"]
 
 templates_path = ["_templates"]

diff --git a/docs/source/contributors/index.md b/docs/source/contributors/index.md
@@ -18,7 +18,7 @@ Issues and pull requests that violate the above principles may be declined. If y
 
 ## Prerequisites
 
-You can develop Jupyter AI on any system that can run a supported Python version up to and including 3.11, including recent Windows, macOS, and Linux versions.
+You can develop Jupyter AI on any system that can run a supported Python version up to and including 3.12, including recent Windows, macOS, and Linux versions.
 
 Each Jupyter AI major version works with only one major version of JupyterLab. Jupyter AI 1.x supports JupyterLab 3.x, and Jupyter AI 2.x supports JupyterLab 4.x.
 
@@ -35,7 +35,7 @@ Due to a compatibility issue with Webpack, Node.js 18.15.0 does not work with Ju
 After you have installed the prerequisites, create a new conda environment and activate it.
 
 ```
-conda create -n jupyter-ai -c conda-forge python=3.11 nodejs=20
+conda create -n jupyter-ai -c conda-forge python=3.12 nodejs=20
 conda activate jupyter-ai
 ```
 
@@ -95,7 +95,16 @@ refresh your browser tab.
 ## Building documentation
 
 The `./scripts/install.sh` should automatically install the documentation
-dependencies. To build the documentation locally, run
+dependencies. You will need to install [pandoc](https://pandoc.org/) as well. You can install [pandoc from the conda-forge channel](https://anaconda.org/conda-forge/pandoc):
+
+```
+conda install -c conda-forge pandoc
+```
+
+Otherwise have a look at pandoc's [installation instructions](https://pandoc.org/installing.html).
+
+
+To build the documentation locally, run
 
 ```
 cd docs/

diff --git a/docs/source/examples/code.nblink b/docs/source/examples/code.nblink
@@ -0,0 +1,3 @@
+{
+    "path": "../../../examples/code.ipynb"
+}
diff --git a/docs/source/examples/commands.nblink b/docs/source/examples/commands.nblink
@@ -0,0 +1,3 @@
+{
+    "path": "../../../examples/commands.ipynb"
+}
diff --git a/docs/source/examples/errors.nblink b/docs/source/examples/errors.nblink
@@ -0,0 +1,3 @@
+{
+    "path": "../../../examples/errors.ipynb"
+}
diff --git a/docs/source/examples/generate/Creating Random Arrays with Numpy.nblink b/docs/source/examples/generate/Creating Random Arrays with Numpy.nblink
@@ -0,0 +1,3 @@
+{
+    "path": "../../../../examples/generate/Creating Random Arrays with Numpy.ipynb"
+}
diff --git a/docs/source/examples/generate/Introduction to Sets.nblink b/docs/source/examples/generate/Introduction to Sets.nblink
@@ -0,0 +1,3 @@
+{
+    "path": "../../../../examples/generate/Introduction to Sets.ipynb"
+}
diff --git a/docs/source/examples/generate/index.md b/docs/source/examples/generate/index.md
@@ -0,0 +1,9 @@
+# Generate
+
+```{toctree}
+---
+maxdepth: 1
+---
+Creating Random Arrays with Numpy
+Introduction to Sets
+```
diff --git a/docs/source/examples/images.nblink b/docs/source/examples/images.nblink
@@ -0,0 +1,3 @@
+{
+    "path": "../../../examples/images.ipynb"
+}
diff --git a/docs/source/examples/index.md b/docs/source/examples/index.md
@@ -0,0 +1,14 @@
+# Examples
+
+```{toctree}
+---
+maxdepth: 1
+---
+code
+commands
+errors
+images
+magics
+sagemaker
+generate/index
+```