DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to serve an open model on your own GPUs with SGLang

Install SGLang, pick a ready-to-run launch command for your model and hardware, and send a first request.

Short answer: Pull the image: docker pull lmsysorg/sglang:latest, or install with uv pip install --prerelease=allow sglang; then open the Cookbook and choose your model and hardware to get a launch command; launch the model and follow the Quickstart to send your first request. All steps.

Who it is forA team moving from a paid API to self-hosted open models.

The problemAPI bills grow with usage, and some data cannot leave your servers.

The outcomeAn SGLang server running your chosen model, answering requests on your hardware.

What you need first

  • A supported GPU or accelerator (NVIDIA, AMD, TPU and others)
  • Docker, or a Python environment with uv

Steps

  1. Pull the image: docker pull lmsysorg/sglang:latest, or install with uv pip install --prerelease=allow sglang
  2. Open the Cookbook and choose your model and hardware to get a launch command
  3. Launch the model and follow the Quickstart to send your first request
  4. Check the hardware platform guide for your GPU before tuning

Prompt for your agent

Set up sgl-project/sglang with Docker on this <GPU> machine, use the Cookbook launch command for <model>, send one test request, and report tokens per second.

Caveat. Large models need large GPUs; check the Cookbook's hardware notes for your model first.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for sgl-project/sglang and "Serve an open model on your own GPUs"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer