DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to run the 27B AutoThink decision model on one GPU from a GGUF with Start Lux Decision

Use the StartLux-Prism server to serve the 27B AutoThink Q8 GGUF on a single CUDA GPU.

Short answer: Run git clone https://github.com/StartLuxLabs/StartLux-Prism.git and cd StartLux-Prism; then create the environment with python3.12 -m venv .venv && source .venv/bin/activate and install requirements.txt; download the Q8_0 GGUF and the model config files with the two hf download commands from the README. All steps.

Who it is forA developer with one large GPU who wants the most accurate StartLux model without a multi-GPU vLLM setup.

The problemThe full 27B model with vLLM expects several GPUs, which most teams do not have.

The outcomeA local /v1/systemone server on port 8090 answering decisions with the 27B AutoThink model.

What you need first

  • Python environment that can install the package with vLLM support, or a Bamboo API key for the hosted model
  • A GPU for the local vLLM setup; the README numbers were measured on one H200
  • Disk space and network access to download a model from Hugging Face

Steps

  1. Run git clone https://github.com/StartLuxLabs/StartLux-Prism.git and cd StartLux-Prism
  2. Create the environment with python3.12 -m venv .venv && source .venv/bin/activate and install requirements.txt
  3. Download the Q8_0 GGUF and the model config files with the two hf download commands from the README
  4. Start the server: python -m startlux_prism.server --gguf $M/StartLux-Decision-v1.1-27B-AutoThink-Q8_0.gguf --model-dir $M --port 8090
  5. Send a test request to localhost:8090/v1/systemone

Prompt for your agent

Set up StartLux-Prism for StartLuxLabs/StartLux-Decision on this Linux GPU box, download the 27B AutoThink Q8_0 GGUF, start the server on port 8090 and send the README's ticket example to confirm it answers.

Caveat. Prism needs Linux, Python 3.12 and a CUDA 13 driver, and v1.1 Instant models are not supported by it yet.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for StartLuxLabs/StartLux-Decision and "Run the 27B AutoThink decision model on one GPU from a GGUF"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer