Can agent use local model?

[dongdongzhaoUP]

In addition to the API usage, can agents built on local models?

[josh-ashkinaze]

Hi, thanks for the question!

This is something we may do more on in future versions. Officially no (since we didn't write tests/docs for this) but technically yes ---you can do this very easily using Ollama.

Here's a quick example for you.

1. Download Ollama if you have't download it
2. ollama start # starts ollama
3. ollama pull gemma:2b # gets gemma2b
4. ollama run gemma:2b # running locally

Now in Python

from plurals.agent Import Agent

a = Agent(task='Say hello', model="ollama/gemma:2b", 
          kwargs={'api_base':"http://localhost:11434"})
a.process()

# 'I am a large language model, trained by Google. I am a conversational AI that can engage in natural language conversations and provide information and insights.'

[dongdongzhaoUP]

Thanks for your reply. It is useful, and wish the project gets better~