Who it is forA developer whose tools already speak the OpenAI API.
The problemPointing existing tools at local models usually needs adapters.
The outcomeA local API at 127.0.0.1:8000 with chat, embeddings and audio endpoints.
What you need first
- Python with pip, or npm for React Native and Web
- A supported device; models download on first use
Steps
- Install with
pip install "runanywhere[server]" - Start it with
runanywhere serve - Point an OpenAI client at http://localhost:8000/v1
- Send a chat completion with a local model name
Prompt for your agent
Using RunanywhereAI/runanywhere-sdks, run `pip install "runanywhere[server]"` and `runanywhere serve`, then call it with the OpenAI Python client using model qwen2.5-0.5b and show the reply.
Caveat. The server is for local use; do not expose it to untrusted networks.
Use this in your AI agent
Ask your agent
Using the Deduped MCP, get the setup steps for RunanywhereAI/runanywhere-sdks and "Expose on device models as an OpenAI compatible API"
Add Deduped to Claude Code once
claude mcp add --transport http deduped https://dedupedai.com/mcp
Every Monday: the 5 hottest repos and exactly how to use them. Free, one email a week, unsubscribe anytime.
Get them freeShare: X LinkedIn Hacker News Email Copy link
Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer