Enhancement: Model Context Protocol (MCP) support #4876
Comments
|
since this is basically introducing agents, you might consider implementing the agent orchestration layer, as well. this would make for a compelling langchain alternative. https://awslabs.github.io/multi-agent-orchestrator/general/how-it-works/ |
Not necessarily, the categories that most interest me are: resources, tools, prompts. I believe the multi-agent orchestrator you're referring to is limited to AWS agents/models and does not refer to MCP (but could be a "sampling" MCP), whereas LibreChat already has an agents component and is fully capable of multi-agent orchestration (not yet implemented, but have developed some prototypes), which expands the access to models to virtually everything supported by LibreChat. |
|
Agree with @danny-avila, this is not about agents but more about data in/data out. LLMs without data are just a brain in a tank, and MCP-like protocol changes everything in this department. The first-class support of MCP servers by a good MCP client can teleport to the Moon a product in the point of view of available functionality. Some issues I see along the road: servers management is kind of like a "Store" for MCP servers. Ability to connect (enable) and configure (?) servers from the list of available servers. First-class UI to look up resources and prompts. The last one is obvious after using the Claude Desktop, as their list of resources does not have even a simple filter. Also, the ability of models to look up for tools. More tools mean more issues to resolve a proper tool by LLM, even for the most powerful models, so the tools exposed by a MCP servers should be toggleable inside UI somewhere close to the Chat window. |
|
Hi! So, how do we customize actions for agents. The custom action works fine in assistants, but seems the agent cannot see any of the endpoints there. Same with tools, I can add them to the plugins or assistants to choose, but no custom tools show up in agents either. Thanks! |
@grexzen This is unrelated to this post, create a "troubleshooting" discussion post describing the action and provide the openapi schema if possible. If you have a fork with tools added in the code, it may not be outright supported, sharing any details about that in a dedicated post would help, too. |
|
I'm working on this and starting to make significant progress. My one grievance with this is that the majority of MCP servers out are all STDIO with no other options. This is great if you are running LibreChat locally and you are the only user, but it's not viable for remote/production. There is SSE support and currently testing how well this works at scale, also found this comment by the devs on this issue:
source: https://news.ycombinator.com/item?id=42237424 One potential cool possibility for this is allowing users to bring-their-own-mcp servers, that could be running locally but sends requests to the remote deployment, though the MCP framework doesn't offer an out-of-the-box solution and would require something robust to be made that is compatible with the web. Could be a lot of work and may consider it after MCP is integrated to begin with. |
|
Is there a simple strategy that we could set up for single users. I'm an artist and digital tinkerer using MCP claude desktop and I love how fast I can build. I have a bunch of ideas, but I can't get MCP to work on LibreChat, even though Claude and I are attempting in all sorts of ways. Can I help? Testing or ... |
|
I agree with the request to add support for MCP servers, and as soon as possible... Right now, no matter how much I searched, I couldn't find a decent open source MCP client. If LibreChat is the first open source client to support MCP servers, it would be a huge advantage over all the rest. Even if you only implement StdIO, and not SSE, it would be a game changer for me and others. Because I want to use MCP servers not through a claude desktop that is limited in the number of requests. |
|
I think STDIO MCP servers are extremely limiting and damn near impossible to troubleshoot. I think SSE is the way to go. I've created a gradio (python) chat interface (MCP client) that can spin up STDIO MCP server, like Claude desktop, or can connect to a SSE server. The POC I built can support a mix of types as well. I only have tools up and running to prove the basic concept. Aside from security, SSE will be easier to maintain and scale. |
|
I should have initial support pushed to main soon. For now, only tools are implemented. I will then move on to Resources, then Prompts. Testing puppeteer:
Any connected servers would then populate the tools dialog for Agents. I'm just finishing integration for Assistants API, though Agents is recommended over using it, since there isn't an easy way to support images in tool results for Assistants among other reasons. |
|
This is great. Since the mcp server config is in the librechat.yaml, do we need to restart the docker container when changing the configuration? This question is directed towards the docker based install, since that is what I am using. Is there an option to store the config in the db and refresh from the UI? Thanks again for everyone working on this. Its exactly what I have been looking for. |
|
@cbusillo yes it requires a container restart following changes, similar to Claude desktop. Glad this can serve you, more work to be done for Resources and Prompts. |
|
Thanks for the MCP option. This is fantastic! That said, I'm not quite able to get the model to connect to an mcp server. The model keeps responding to my prompt as if it doesn't know about the external tool. I've configured my environment like this: librechat.yaml # For more information, see the Configuration Guide:
# https://www.librechat.ai/docs/configuration/librechat_yaml
# Configuration version (required)
version: 1.1.5
mcpServers:
sqlite:
# type: stdio
command: /Users/johnny/.pyenv/shims/uvx
args:
- "@modelcontextprotocol/mcp-server-sqlite"
- "--db-path"
- "/Users/johnny/path/to/database.db"docker-compose.override.yml version: '3.4'
services:
api:
volumes:
- type: bind
source: ./librechat.yaml
target: /app/librechat.yamlStartup: docker compose up -dPrompt: model: claude-3-5-sonnet-latest
Response:
|
|
@johnnyoshika https://www.librechat.ai/docs/configuration/logging |
|
@johnnyoshika did you make an agent with your tool added? |
|
Thanks @harrylaou for pointing me to the logs. Note: it still doesn't work - npx mcp servers might be easier to get up and running, LOL. Created #!/bin/sh
# This script requires executable permissions (chmod +x) to run in container
# chmod +x ./docker-entrypoint.sh
set -e
# Install UV only if not already installed
if [ ! -f "/root/.local/bin/uvx" ]; then
echo "Installing UV..."
curl -LsSf https://github.com/astral-sh/uv/releases/latest/download/uv-installer.sh | sh
fi
# Add UVX to PATH
export PATH="/root/.local/bin:$PATH"
# Debug info
echo "UVX location: $(which uvx)"
echo "UVX version: $(uvx --version)"
echo "Starting command: $@"
# Execute the original command
if [ $# -eq 0 ]; then
exec npm run backend
else
exec "$@"
fiAdded the following to version: '3.4'
services:
api:
volumes:
- type: bind
source: ./docker-entrypoint.sh
target: /docker-entrypoint.sh
- type: bind
source: ./librechat.yaml
target: /app/librechat.yaml
- type: bind
source: /path/to/database/file.db
target: /data/file.db
entrypoint: ["/bin/sh", "/docker-entrypoint.sh"]Made the database very permissible on the host: chmod 777 /path/to/database/file.dblibrechat.yaml: # For more information, see the Configuration Guide:
# https://www.librechat.ai/docs/configuration/librechat_yaml
# Configuration version (required)
version: 1.1.5
mcpServers:
sqlite:
# type: stdio
command: /root/.local/bin/uvx
args:
- "@modelcontextprotocol/mcp-server-sqlite"
- "--db-path"
- "/data/file.db"With this, I got uvx installed inside the container, but running the MCP server is still problematic. Excerpt from the debug log file: |
|
I got mcp working with an npx server. I tested this with brave-search. librechat.yaml mcpServers:
brave-search:
type: stdio
command: npx
args:
- "-y"
- "@modelcontextprotocol/server-brave-search"
env:
BRAVE_API_KEY: "[Redacted]"
PATH: "/usr/local/bin:/usr/bin:/bin"
NODE_PATH: "/usr/local/lib/node_modules"Note: I got the same I also had to create an agent to test this as @cbusillo suggested. It would still be nice to get this working with a uvx mcp server (e.g. @modelcontextprotocol/mcp-server-sqlite). Any suggestions? |
|
Some discussions in the MCP repo might prove useful to people: Note, the docker builds of LibreChat don't have the dependencies that uv/uvx require, I would need to test what we can do there |
|
I have a dumb issue: I got MCP servers working, and LibreChat is connecting to them. However, I don't see any way to use them in chat, and when I try to reference them, Sonnet doesn't seem to be aware of their existence. I'm probably missing a flag I need to turn on somewhere. 
|
|
@basilesportif it’s under Agents in the provider selector. You’ll need to activate the agents in the librechat.yaml file too if you haven’t already. |
|
Got it. I had to use the |
|
@basilesportif Can you elaborate on how you built a tool that gives access to all tools? |
|
@basilesportif I seem to have a similar problem as you, after the previous configuration is completed, I can't call the tool when I start the conversation, and then I just end the answer. When I use the disableBuilder: false option under agents, there is still no change, can you give me some suggestions |
|
Running into similar issue as @che-kola mentioned. @basilesportif would you please be able to elaborate more on your approach and how you got this to work? |
|
@che-kola you have the agent builder available to you, have you added the tools to your current agent? It seems to be aware of it, so I'm guessing you do. In that case, it would be helpful to know what provider/model you are using, it will need to support tools/function calls.
|
|
Is there a way to add all tools for an MCP? I'm using intellij MCP server and it has a few dozen tools It's quite cumbersome to have to manually add them to the agent. |
|
@maxhillaert |
|
Hi. I'm having trouble getting LibreChat container to discover my running Windows 11 instance of intellij with MCP server plugin. Instead of unpicking docker networking , I decided to run LibreChat with npm directly, but getting spawn npx ENOENT issues e.g. |
Discussed in #4815
Originally posted by karstennilsen November 30, 2024
What features would you like to see added?
Anthropic released Model Context Protocol (MCP)
Implement support for Anthropic’s newly announced Model Context Protocol (MCP) in LibreChat. MCP aims to standardize interactions with AI models, offering consistent and predictable behavior across different implementations. Integrating this protocol will enhance LibreChat’s interoperability, streamline interactions, and potentially allow broader compatibility with a increasing number of MCP servers.
More details
There are existing MCP-compliant servers available that can be self-hosted. These can be used when LibreChat is a MCP client. The example servers, hosted on GitHub, provide ready-to-use implementations of the Model Context Protocol for various use cases.
Typescript SDK
https://github.com/modelcontextprotocol/typescript-sdk
Which components are impacted by your request?
Endpoints
Pictures
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: