[Question]: Is it possible to use plugins with 'custom endpoints'??

[bpawnzZ]

What is your question?

Specifically I am trying to get search functionality/plugins in general would be nice out of custom endpoints. Specifically I am using openrouter.

Thank you

More Details

Is this functionality not possible?. If it is can someone please point me in the right direction.

Thank you

What is the main subject of your question?

LibreChat Plugins

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[bpawnzZ]

If anyone has any ideas I am all ears?? Like for instance is there a way I can use meilisearch with my custom endpoint? Just seems like this might be a maybe since it is running in its own container. Idk.. Awesome project though! This is by far the best chat-ui type application I have thus found.

[danny-avila]

@bpawnzZ maybe you can do a blog post and we’ll host it on the LibreChat site?

[danny-avila]

https://www.librechat.ai/blog/2024-04-17_blog_guide

[bpawnzZ]

@bpawnzZ maybe you can do a blog post and we’ll host it on the LibreChat site?

I don't mind that man. No problem.
There are a couple networking things that may not be obvious to people not very into docker. But I'll explain it all in a blog post and try to have it up in next 48 hrs

[fuegovic]

@bpawnzZ here's a couple of links to help get started with the blog:
https://www.librechat.ai/blog/2024-04-17_blog_guide
https://github.com/LibreChat-AI/librechat.ai

[Phlasse]

FYI guys. The workaround for getting plugins with 3rd party models working is getting LiteLLM setup correctly and setting the OpenAI Proxy/environmental variable for LiteLLM's URL. I have mistralai/mixtral-8x22b-instruct via openrouter working perfectly. Like no issues I have spotted this far testing. Getting it to this point took some patience and trial and error but it's working flawlessly.

Just understand this is not officially supported and Devs will not give support on this method. Which is understandable from their perspective and I know they are officially rolling out 3rd party support in the next few months. But for the impatient like myself this does work

For those interested docker run command

docker run -d \
    -v $(pwd)/litellm_config.yaml:/app/config.yaml \
    -e LLM_API_KEY=sk-or-v1- \
    -e LLM_API_BASE=https://openrouter.ai/api/v1 \
    -p 4000:4000 \
    -h litellmSRV \
    ghcr.io/berriai/litellm:main-latest \
    --config /app/config.yaml --detailed_debug --model openrouter/mistralai/mixtral-8x22b-instruct

litellm_config.yaml

model_list:
  - model_name: mixtral-8x22b-instruct
    litellm_params:
      model: openrouter/mistralai/mixtral-8x22b-instruct
      api_base: https://openrouter.ai/api/v1
      api_key: sk-or-
      rpm: 30

@bpawnzZ I am working on the same thing. I get the approach you take, but I am having issues with the proper configuration of the librechat. How did you configure it in the .env file or the librechat.yaml?

[danny-avila]

“Agents” are the answer to this

#3607