DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to run Claude Code or Codex on any model through the gateway with Litellm

Start a coding agent in a local repo and route all its model calls through your AI Gateway.

Who it is forA developer who wants a coding agent such as Claude Code or Codex to work on a repo while every model call is tracked by the team's gateway.

The problemCoding agents usually talk to one provider directly, which makes model choice, cost and logging hard to manage centrally.

The outcomeAn agent edits files in a local repo and returns a result, with each model call sent through the AI Gateway.

What you need first

  • Python and the uv package manager installed
  • API keys for the LLM providers you plan to call, such as OpenAI or Anthropic
  • For the gateway use cases: a machine to run the proxy server on and a LiteLLM master key or virtual key for clients

Steps

  1. Install the SDK and the proxy with uv add litellm
  2. Install the agent's CLI, such as claude or codex, plus starlette and uvicorn
  3. Set LITELLM_PROXY_API_BASE and LITELLM_PROXY_API_KEY to your gateway address and key
  4. Call Litellm.agent with a Harness value, a sandbox.local path to the repo, and a litellm_proxy/ model name

Prompt for your agent

In the litellm repo at <path>, run litellm.agent with Harness.CLAUDE_CODE on the local repo at <repo_path> with the task 'Find why tests/test_router.py is flaky'. Expected output: the agent's text result, its cost, and a list of changed file paths. Verify that LITELLM_PROXY_API_BASE and LITELLM_PROXY_API_KEY are set before running.

Caveat. The README describes this as a Python SDK feature that needs the agent's own CLI installed, so the agent's file changes happen in the repo you pass in.

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer