DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to give LLM calls access to MCP server tools with Litellm

Load tools from an MCP server and pass them to any LiteLLM model in OpenAI format.

Credentials. MCP servers that use OAuth may need a pre-registered client_id and client_secret, so store those secrets outside the code and out of version control.

Who it is forA developer building an agent or chat app who already runs an MCP server and wants a model to call its tools.

The problemEach model expects tools in its own format, so wiring MCP server tools into LLM calls takes custom glue code.

The outcomeA model call receives the MCP server's tools in OpenAI format and can request them during a chat.

What you need first

  • Python and the uv package manager installed
  • API keys for the LLM providers you plan to call, such as OpenAI or Anthropic
  • For the gateway use cases: a machine to run the proxy server on and a LiteLLM master key or virtual key for clients

Steps

  1. Install the SDK with uv add litellm
  2. Start a stdio MCP server, such as mcp_server.py, and open a ClientSession to it
  3. Call experimental_mcp_client.load_mcp_tools with format="openai"
  4. Pass the returned tools to Litellm.acompletion with a model such as gpt-4o

Prompt for your agent

Using the litellm repo at <path>, connect to the MCP server at <path>/mcp_server.py, load its tools in OpenAI format, and ask gpt-4o 'What is 3 + 5?' with those tools. Expected output: a model response that uses the MCP tool. Verify the tool list is not empty before the call.

Caveat. For MCP OAuth, the upstream provider must approve the app, and reaching its authorization page does not guarantee that tool calls will succeed.

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer