# AI use cases for Litellm (BerriAI/litellm) Repo: [BerriAI/litellm](https://github.com/BerriAI/litellm) | Use cases: 5 | Page: https://dedupedai.com/use-cases/berriai-litellm/ LiteLLM is an open source AI Gateway and Python SDK that lets you call 100+ LLM providers through one OpenAI-format interface, with cost tracking, guardrails, and load balancing. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Call 100+ LLM providers through one OpenAI-format API: https://dedupedai.com/use-cases/berriai-litellm/call-100-llms-with-one-openai-format-api/ - Run a shared self-hosted AI Gateway for your team: https://dedupedai.com/use-cases/berriai-litellm/run-shared-ai-gateway-with-virtual-keys/ - Give LLM calls access to MCP server tools: https://dedupedai.com/use-cases/berriai-litellm/give-llm-calls-access-to-mcp-tools/ - Run Claude Code or Codex on any model through the gateway: https://dedupedai.com/use-cases/berriai-litellm/run-coding-agents-on-any-model-via-gateway/ - Call A2A agents through the AI Gateway: https://dedupedai.com/use-cases/berriai-litellm/invoke-a2a-agents-through-ai-gateway/ ## How to call 100+ LLM providers through one OpenAI-format API with Litellm Repo: [BerriAI/litellm](https://github.com/BerriAI/litellm) | Difficulty: Intermediate | Time: 30 minutes | Basis: From the README Switch between OpenAI, Anthropic and other providers in Python without rewriting request code. **Who it is for:** A Python developer who calls more than one LLM provider and is tired of learning a different SDK and request format for each one. **The problem:** Each provider has its own SDK, auth pattern, request format and error types, so every new model adds integration work. **The outcome:** One completion call in OpenAI format returns responses from different providers by changing only the model string. ### Steps 1. Install the SDK with `uv add litellm` 2. Set the provider keys as environment variables, such as OPENAI_API_KEY and ANTHROPIC_API_KEY 3. Call `completion` from Litellm with a model such as openai/gpt-4o 4. Change the model string to anthropic/claude-sonnet-4-20250514 and run the same call again ### Prompt to paste into your agent ```text Using the litellm repo at , write a short Python script that sends the message 'Hello!' to openai/gpt-4o and to anthropic/claude-sonnet-4-20250514 through litellm.completion. Expected output: two printed responses. Verify that both API keys are set in the environment before running. ``` ### Caveat You still need a valid account and API key for each provider you call, and the provider's own pricing and limits apply. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/berriai-litellm/call-100-llms-with-one-openai-format-api/ ## How to run a shared self-hosted AI Gateway for your team with Litellm Repo: [BerriAI/litellm](https://github.com/BerriAI/litellm) | Difficulty: Advanced | Time: 45 minutes | Basis: From the README Deploy one proxy that gives your team OpenAI-style access to many models with virtual keys. **Who it is for:** A platform or IT admin who needs to give several developers access to LLM providers without sharing raw provider keys. **The problem:** Sharing provider keys across a team makes spend and access hard to control, and each app needs its own provider setup. **The outcome:** Team members send requests to one local endpoint in OpenAI format, using keys the gateway issues. ### Steps 1. Install the proxy with `uv tool install 'litellm[proxy]'` 2. Start the gateway with `litellm --model gpt-4o` 3. Follow the E2E tutorial at https://docs.litellm.ai/docs/proxy/docker_quick_start to create virtual keys 4. Point a client at http://0.0.0.0:4000 using the OpenAI client and a virtual key as api_key ### Prompt to paste into your agent ```text In the litellm repo at , start the AI Gateway proxy on port 4000 and send a test chat completion for model gpt-4o from a Python OpenAI client pointed at http://0.0.0.0:4000. Expected output: a valid chat response. Verify the proxy log shows the request was received. ``` ### Caveat The README's quick start runs a local endpoint on 0.0.0.0:4000, so you must add your own network, TLS and access controls before exposing it to a team. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/berriai-litellm/run-shared-ai-gateway-with-virtual-keys/ ## How to give LLM calls access to MCP server tools with Litellm Repo: [BerriAI/litellm](https://github.com/BerriAI/litellm) | Difficulty: Advanced | Time: 45 minutes | Basis: From the README Load tools from an MCP server and pass them to any LiteLLM model in OpenAI format. **Who it is for:** A developer building an agent or chat app who already runs an MCP server and wants a model to call its tools. **The problem:** Each model expects tools in its own format, so wiring MCP server tools into LLM calls takes custom glue code. **The outcome:** A model call receives the MCP server's tools in OpenAI format and can request them during a chat. ### Steps 1. Install the SDK with `uv add litellm` 2. Start a stdio MCP server, such as mcp_server.py, and open a ClientSession to it 3. Call experimental_mcp_client.load_mcp_tools with format="openai" 4. Pass the returned tools to Litellm.acompletion with a model such as gpt-4o ### Prompt to paste into your agent ```text Using the litellm repo at , connect to the MCP server at /mcp_server.py, load its tools in OpenAI format, and ask gpt-4o 'What is 3 + 5?' with those tools. Expected output: a model response that uses the MCP tool. Verify the tool list is not empty before the call. ``` ### Caveat For MCP OAuth, the upstream provider must approve the app, and reaching its authorization page does not guarantee that tool calls will succeed. Risk, credentials: MCP servers that use OAuth may need a pre-registered client_id and client_secret, so store those secrets outside the code and out of version control. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/berriai-litellm/give-llm-calls-access-to-mcp-tools/ ## How to run Claude Code or Codex on any model through the gateway with Litellm Repo: [BerriAI/litellm](https://github.com/BerriAI/litellm) | Difficulty: Advanced | Time: 60 minutes | Basis: From the README Start a coding agent in a local repo and route all its model calls through your AI Gateway. **Who it is for:** A developer who wants a coding agent such as Claude Code or Codex to work on a repo while every model call is tracked by the team's gateway. **The problem:** Coding agents usually talk to one provider directly, which makes model choice, cost and logging hard to manage centrally. **The outcome:** An agent edits files in a local repo and returns a result, with each model call sent through the AI Gateway. ### Steps 1. Install the SDK and the proxy with `uv add litellm` 2. Install the agent's CLI, such as claude or codex, plus starlette and uvicorn 3. Set LITELLM_PROXY_API_BASE and LITELLM_PROXY_API_KEY to your gateway address and key 4. Call Litellm.agent with a Harness value, a sandbox.local path to the repo, and a litellm_proxy/ model name ### Prompt to paste into your agent ```text In the litellm repo at , run litellm.agent with Harness.CLAUDE_CODE on the local repo at with the task 'Find why tests/test_router.py is flaky'. Expected output: the agent's text result, its cost, and a list of changed file paths. Verify that LITELLM_PROXY_API_BASE and LITELLM_PROXY_API_KEY are set before running. ``` ### Caveat The README describes this as a Python SDK feature that needs the agent's own CLI installed, so the agent's file changes happen in the repo you pass in. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/berriai-litellm/run-coding-agents-on-any-model-via-gateway/ ## How to call A2A agents through the AI Gateway with Litellm Repo: [BerriAI/litellm](https://github.com/BerriAI/litellm) | Difficulty: Advanced | Time: 60 minutes | Basis: From the README Register an A2A agent on the gateway and send it messages from any A2A client. **Who it is for:** A developer or platform team that runs agents from frameworks such as LangGraph or Pydantic AI and wants one gateway entry point for them. **The problem:** Each agent framework exposes its own endpoint and auth, so clients need separate setup for every agent. **The outcome:** A client sends a message to a named agent through the gateway and prints the text reply. ### Steps 1. Add your agent to the AI Gateway by following https://docs.litellm.ai/docs/a2a#adding-your-agent and set protocolVersion to 1.0 or 0.3 2. Install the A2A client with `pip install a2a-sdk>=1.1.0` (inferred from the README requirement) (inferred, check the README) 3. Set the Authorization header to a LiteLLM master key or virtual key 4. Create the client from the agent card at http://localhost:4000/a2a/ and call send_message ### Prompt to paste into your agent ```text Using the litellm repo at , register the agent on the gateway and send it the message 'Hello!' through an A2A client pointed at http://localhost:4000/a2a/. Expected output: the agent's text reply printed to the console. Verify that the agent card loads before sending the message. ``` ### Caveat The README requires a2a-sdk version 1.1.0 or newer, and the agent must already be running on its own framework before the gateway can call it. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/berriai-litellm/invoke-a2a-agents-through-ai-gateway/