DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to use your local model as an OpenAI-compatible API with Textgen

Start TextGen with --api and point existing OpenAI or Anthropic client code at it.

Short answer: Install TextGen with the portable venv steps; then launch with python server.py --portable --api --auto-launch; point your OpenAI-compatible client at the local server. All steps.

Who it is forA developer who wants to test apps against a local model for free.

The problemAPI bills and rate limits slow down local development.

The outcomeA local endpoint that speaks the OpenAI and Anthropic API formats, with tool calling.

What you need first

  • Windows, Linux or macOS
  • Python 3.9+ for the manual install, or a portable build
  • A GGUF model file from Hugging Face and enough RAM or VRAM for it

Steps

  1. Install TextGen with the portable venv steps
  2. Launch with python server.py --portable --api --auto-launch
  3. Point your OpenAI-compatible client at the local server
  4. Try tool-calling with a custom .py tool or an MCP server

Prompt for your agent

Start oobabooga/textgen with `python server.py --portable --api --auto-launch`, then change <my app> to use the local OpenAI-compatible API and run its tests.

Caveat. This card is written from the oobabooga/textgen README, not from a test run by Deduped. Before you start, make sure you have Windows, Linux or macOS. Check the repo for newer instructions before you rely on these steps.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for oobabooga/textgen and "Use your local model as an OpenAI-compatible API"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer