DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to serve NeoHorse-1 locally for agent tool calls with Neo Horse

Host NeoHorse-1-4B on your own GPU with vLLM and send it chat requests from a local script.

Who it is forA developer building a text-based agent harness who needs a self-hosted model that supports tool use and coding tasks.

The problemHosted model APIs may not fit a project that needs local control over the model and its serving settings.

The outcomeA running local endpoint at http://127.0.0.1:8000 that answers chat requests under the served name neohorse-1-4B.

What you need first

  • A machine with enough GPU memory for the 4B or 9B BF16 weights, or a quantized GGUF or MLX build for smaller hardware
  • Python with pip available
  • Disk space for the downloaded checkpoint from Hugging Face or ModelScope

Steps

  1. Download NeoHorse-1-4B from Hugging Face or ModelScope to a local directory that contains config.json, tokenizer files, and weights.
  2. Run pip install -U vllm
  3. Set MODEL_PATH to that directory, then run vllm serve "$MODEL_PATH" --served-model-name neohorse-1-4B --host 0.0.0.0 --port 8000 --max-model-len 262144 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder (inferred, check the README)
  4. Run pip install requests
  5. Run the chat example in the README's Send a Request section against http://127.0.0.1:8000 and check that it returns a reply.

Prompt for your agent

In the TokenRhythm/NeoHorse repo, follow the vLLM steps in the Deployment section to serve the NeoHorse-1-4B checkpoint at <model-directory> on port 8000 with served name neohorse-1-4B. Return the chat reply text for the prompt <test-prompt>. Verify that the server log shows the model loaded without errors before sending the request.

Caveat. The README says the 262,144-token setting is a configured limit and actual capacity depends on GPU memory, so you may need to lower --max-model-len.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page