Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(optim) | add bootstrapfewshot #7

Merged
merged 30 commits into from
Jul 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
318941b
add: managers.py
ammirsm Jun 26, 2024
1dd4c5d
merge deflm and managers.py
ammirsm Jun 27, 2024
1d8f5d2
add: new managers.py
ammirsm Jun 28, 2024
333f6d1
add: refactor refine --> clean and duplicate.
ammirsm Jun 28, 2024
73c49f1
add: refactor abc --> base
ammirsm Jun 28, 2024
b389d88
fix: langchain.py to have IndividualEvalValue.
ammirsm Jun 29, 2024
fbbebd5
fix: langchain moved to multiple classes.
ammirsm Jul 2, 2024
85429d1
fix: tests update.
ammirsm Jul 2, 2024
ec58475
fix: types.
ammirsm Jul 2, 2024
94ae8c1
add: scripts.
ammirsm Jul 2, 2024
c90e3bd
fix: bootstrap_few_shot.py
ammirsm Jul 2, 2024
5f945c6
fix: helpers --> adaptors
ammirsm Jul 2, 2024
fcd36fb
update: test langchain.
ammirsm Jul 2, 2024
0ab0f37
fix: updte git ignore.
ammirsm Jul 2, 2024
5779956
add: test for bootstrap_few_shot.py
ammirsm Jul 2, 2024
9730ed8
fix: tests.
ammirsm Jul 2, 2024
95b63ca
fix: tests.
ammirsm Jul 2, 2024
87e8696
fix: change helpers to adaptors.
ammirsm Jul 2, 2024
68277e9
add: save optimized_args.
ammirsm Jul 2, 2024
c8bf0da
add: save optimized_args.
ammirsm Jul 3, 2024
7377c7c
add: save optimized_args.
ammirsm Jul 3, 2024
e5e0efa
fix: test and clean up.
ammirsm Jul 3, 2024
fde303a
fix: update cookbooks.
ammirsm Jul 3, 2024
84a18c9
fix: update names.
ammirsm Jul 3, 2024
6829b5c
fix: update notebook.
ammirsm Jul 3, 2024
c5ffe51
Merge branch 'main' into feat/trace-manager-merge-deflm
ammirsm Jul 5, 2024
0d32aac
feat(bootstrapfewshot) | parea integration. (#8)
ammirsm Jul 5, 2024
94579b0
feat(bootstrapfewshot) | add lunary (#10)
ammirsm Jul 5, 2024
0b0bad6
feat(bootstrapfewshot) | add langfuse (#11)
ammirsm Jul 8, 2024
64fb48b
feat(bootstrapfewshot) | cleanup notebook and readme (#12)
ammirsm Jul 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,10 @@ jobs:
run: ls -la py
- name: Run Tests
working-directory: py
run: rye run pytest -v
run: rye run pytest -v -n auto
- name: Run Integration Tests
working-directory: py
run: rye run pytest -v -m helpers
run: rye run pytest -v -m helpers -n auto
env:
LANGCHAIN_API_KEY: ${{ secrets.LANGCHAIN_API_KEY }}
LANGCHAIN_TRACING_V2: ${{ secrets.LANGCHAIN_TRACING_V2 }}
Expand Down
34 changes: 26 additions & 8 deletions py/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,32 @@ rye add zenbase

Zenbase is designed to require minimal changes to your existing codebase and integrate seamlessly with your existing eval/observability platforms. It works with any AI SDK (OpenAI, Anthropic, Cohere, Langchain, etc.).

| Cookbook | Run in Colab |
| ---------------------------------------------- |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [langsmith.ipynb](./cookbooks/langsmith.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/drive/14Lh8Azk_Ocnm2GvGvFHFz_hJ1tFNvOJW?usp=sharing) |
| [langfuse.ipynb](./cookbooks/langfuse.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/drive/1EMR_PQfsfawTvTjZSxcS_FQUr8s_gVwR?usp=sharing) |
| [parea.ipynb](./cookbooks/parea.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/drive/1oBFD8MhHjHxCltuosFeXHkqOsdeXwJ_N?usp=sharing) |
| [lunary.ipynb](./cookbooks/lunary.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/drive/1HXpW6E7AgoBbVWiiUxxtzztU6Gxy6iEA?usp=sharing) |

## Repo setup

### Labeled Few-Shot Learning Cookbooks:

LabeledFewShot will be useful for tasks that are just one layer of prompts.

| Cookbook | Run in Colab |
| ---------------------------------------------- |-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [langsmith.ipynb](cookbooks/labeled_few_shot/langsmith.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/zenbase-ai/lib/blob/main/py/cookbooks/labeled_few_shot/langsmith.ipynb) |
| [langfuse.ipynb](cookbooks/labeled_few_shot/langfuse.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/zenbase-ai/lib/blob/main/py/cookbooks/labeled_few_shot/langfuse.ipynb) |
| [parea.ipynb](cookbooks/labeled_few_shot/parea.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/zenbase-ai/lib/blob/main/py/cookbooks/labeled_few_shot/parea.ipynb) |
| [lunary.ipynb](cookbooks/labeled_few_shot/lunary.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/zenbase-ai/lib/blob/main/py/cookbooks/labeled_few_shot/lunary.ipynb) |

### Bootstrap Few-Shot Learning Cookbooks:

BootstrapFewShot will be useful for tasks that are multiple layers of prompts.

| Cookbook | Run in Colab |
| ---------------------------------------------- |-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [langsmith.ipynb](cookbooks/bootstrap_few_shot/langsmith.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/zenbase-ai/lib/blob/main/py/cookbooks/bootstrap_few_shot/langsmith.ipynb) |
| [langfuse.ipynb](cookbooks/bootstrap_few_shot/langfuse.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/zenbase-ai/lib/blob/main/py/cookbooks/bootstrap_few_shot/langfuse.ipynb) |
| [parea.ipynb](cookbooks/bootstrap_few_shot/parea.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/zenbase-ai/lib/blob/main/py/cookbooks/bootstrap_few_shot/parea.ipynb) |
| [lunary.ipynb](cookbooks/bootstrap_few_shot/lunary.ipynb) | [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/zenbase-ai/lib/blob/main/py/cookbooks/bootstrap_few_shot/lunary.ipynb) |

https://colab.research.google.com/github/zenbase-ai/lib/blob/amir/feat/bootstrapfewshot/eng-22-featbootstrapfewshot-cleanup-notebook-and-readme/py/cookbooks/bootstrap_few_shot/langfuse.ipynb

## Development setup

This repo uses Python 3.10 and [rye](https://rye.astral.sh/) to manage dependencies. Once you've gotten rye installed, you can install dependencies by running:

Expand Down
Loading