Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: remove special handling of builtin::rag tool #1015

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

ehhuang
Copy link
Contributor

@ehhuang ehhuang commented Feb 8, 2025

Summary:

Lets the model decide which tool it needs to call to respond to a query.

Test Plan:

LLAMA_STACK_CONFIG=fireworks pytest -s -v tests/client-sdk/ --safety-shield meta-llama/Llama-Guard-3-8B

Also evaluated on a small benchmark with 20 questions from HotpotQA. With this PR and some prompting, the performance is 77% recall compared to 50% currently.


Stack created with Sapling. Best reviewed with ReviewStack.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Feb 8, 2025
@ehhuang ehhuang changed the title memory feat: remove special handling of builtin::rag tool Feb 8, 2025
@ehhuang ehhuang changed the title feat: remove special handling of builtin::rag tool [RFC] feat: remove special handling of builtin::rag tool Feb 8, 2025
@ehhuang ehhuang changed the title [RFC] feat: remove special handling of builtin::rag tool memory Feb 8, 2025
@ehhuang ehhuang changed the title memory [RFC] feat: remove special handling of builtin::rag tool Feb 8, 2025
@ehhuang ehhuang force-pushed the pr1014 branch 3 times, most recently from 17c3d05 to 01cc4c0 Compare February 11, 2025 07:16
@ehhuang ehhuang changed the title [RFC] feat: remove special handling of builtin::rag tool feat: remove special handling of builtin::rag tool Feb 11, 2025
@ehhuang ehhuang marked this pull request as ready for review February 11, 2025 07:25
@hardikjshah
Copy link
Contributor

Also reminder - we will need to update getting_started and other places where we have RAG agent.

Copy link
Contributor

@hardikjshah hardikjshah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we update the tests to not hard code but manage other models properly please ?

Summary:

Test Plan:
Summary:

Test Plan:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants