Who it is forAnyone choosing which open model to use for a project.
The problemComparing models usually means setting up each one separately.
The outcomeA quick hands on comparison of a few models from one tool.
What you need first
- Python with pip
- A GPU sized for the model you pick
- A Hugging Face token for gated models
Steps
- Install with
pip install openllm - Run
openllm hellofor the guided start - List models with
openllm model list - Chat with one model using
openllm run llama3:8b - Note which model answers your sample prompts best
Prompt for your agent
With bentoml/OpenLLM, run `openllm model list`, pick two small models, chat with each using `openllm run`, and compare their answers to <my three test prompts>.
Caveat. Larger models need a GPU with enough memory; start with a small one.
Use this in your AI agent
Ask your agent
Using the Deduped MCP, get the setup steps for bentoml/OpenLLM and "Try open models interactively before you pick one"
Add Deduped to Claude Code once
claude mcp add --transport http deduped https://dedupedai.com/mcp
Every Monday: the 5 hottest repos and exactly how to use them. Free, one email a week, unsubscribe anytime.
Get them freeShare: X LinkedIn Hacker News Email Copy link
Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer