# AI use cases for Spark X2 6 (XHToken/Spark-X2.6) Repo: [XHToken/Spark-X2.6](https://github.com/XHToken/Spark-X2.6) | Use cases: 3 | Page: https://dedupedai.com/use-cases/xhtoken-spark-x2-6/ Spark-X2.6-4B is a compact, general-purpose language model for on-device agents, released with Transformers, SGLang, vLLM, and Ollama deployment paths. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Run Spark-X2.6-4B locally in Python with Transformers: https://dedupedai.com/use-cases/xhtoken-spark-x2-6/run-spark-x26-locally-with-transformers/ - Serve Spark-X2.6-4B as an API with SGLang on NVIDIA GPUs: https://dedupedai.com/use-cases/xhtoken-spark-x2-6/serve-spark-x26-api-with-sglang-on-nvidia/ - Serve Spark-X2.6-4B with vLLM for an OpenAI-style API: https://dedupedai.com/use-cases/xhtoken-spark-x2-6/serve-spark-x26-api-with-vllm/ ## How to run Spark-X2.6-4B locally in Python with Transformers with Spark X2 6 Repo: [XHToken/Spark-X2.6](https://github.com/XHToken/Spark-X2.6) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README Load the 4B model in a Python script and get a chat reply on your own machine. **Who it is for:** A Python developer who wants to test a small on-device model in a script before wiring it into a larger app. **The problem:** Setting up a model server just to try one prompt is slow when the goal is a quick local check of answer quality. **The outcome:** A short Python script that loads XHToken/Spark-X2.6-4B and prints a generated answer to a chat message. ### Steps 1. Run `pip install transformers==4.57.1` to install the version the README specifies. 2. Copy the Transformers code block from the README Quickstart into a new file. 3. Change the content in the messages list to your own question. 4. Run the script and read the decoded output printed at the end. ### Prompt to paste into your agent ```text Using the XHToken/Spark-X2.6-4B Transformers Quickstart in the Spark-X2.6 GitHub README, install transformers==4.57.1, replace the user message with , and run the script. Expected output: a text answer printed to the console. Verify that the printed answer addresses and that the script ran without a trust_remote_code error. ``` ### Caveat The README example sets max_new_tokens=131072 and needs enough device memory, so reduce that value if generation runs out of memory. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/xhtoken-spark-x2-6/run-spark-x26-locally-with-transformers/ ## How to serve Spark-X2.6-4B as an API with SGLang on NVIDIA GPUs with Spark X2 6 Repo: [XHToken/Spark-X2.6](https://github.com/XHToken/Spark-X2.6) | Difficulty: Advanced | Time: 2 hours | Basis: From the README Run an OpenAI-compatible endpoint with tool-call and reasoning parsers enabled. **Who it is for:** A backend engineer on a GPU server who needs a local model endpoint that agent code can call over HTTP. **The problem:** Agent workflows need a stable chat endpoint with tool calling, and calling a hosted model may not fit the data or cost needs of the project. **The outcome:** A running SGLang server on port 30000 that answers chat completion requests under the model name spark2.6. ### Steps 1. Run `export MODEL_PATH=/absolute/path/to/Spark-X2.6-4B` with the absolute path to your local checkpoint. 2. Run `docker pull lmsysorg/sglang:nightly-dev-cu13-20260827-20621aa1` to get the image the README names. 3. Start the NVIDIA GPU `docker run` block from the README SGLang Server section, which mounts $MODEL_PATH and serves on port 30000. 4. Send a chat request to http://localhost:30000/v1/chat/completions using the model name spark2.6, following the README Client example. 5. To skip reasoning for one request, add "chat_template_kwargs": {"enable_thinking": false} to the request body. ### Prompt to paste into your agent ```text Using the Spark-X2.6 README SGLang NVIDIA GPU server block, start an OpenAI-compatible server for the checkpoint at on port 30000 with served model name spark2.6. Send the chat request to http://localhost:30000/v1/chat/completions and report the reply text. Verify that the response JSON has a choices array with non-empty content. ``` ### Caveat The README's 1,048,576-token context setting needs sufficient device memory, so lower --context-length on smaller GPUs. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/xhtoken-spark-x2-6/serve-spark-x26-api-with-sglang-on-nvidia/ ## How to serve Spark-X2.6-4B with vLLM for an OpenAI-style API with Spark X2 6 Repo: [XHToken/Spark-X2.6](https://github.com/XHToken/Spark-X2.6) | Difficulty: Advanced | Time: 2 hours | Basis: From the README Deploy the model with the official vLLM Docker image and prefix caching turned on. **Who it is for:** An ML platform engineer who already runs vLLM and wants to add Spark-X2.6-4B as another served model. **The problem:** Teams that standardize on vLLM need a documented way to load a new model into the same serving stack. **The outcome:** A vLLM container serving the model on port 30000 with the served name spark25, ready for chat requests. ### Steps 1. Run `export MODEL_PATH=/absolute/path/to/Spark-X2.6-4B` to point at your local checkpoint. 2. Run `docker run --rm --gpus all --ipc=host -p 30000:30000 -v "$MODEL_PATH:/models/Spark-X2.6-4B:ro" vllm/vllm-openai:latest --model /models/Spark-X2.6-4B --port 30000 --trust-remote-code --served-model-name spark25 --tensor-parallel-size 1 --gpu-memory-utilization 0.7 --enable-prefix-caching --chat-template /models/Spark-X2.6-4B/chat_template.jinja`. (inferred, check the README) 3. Wait for the server to load, then send a chat request to port 30000 using the served model name spark25. 4. Check the reply text in the JSON response before connecting the endpoint to other tools. ### Prompt to paste into your agent ```text Using the Spark-X2.6 README vLLM NVIDIA deployment block, start vllm/vllm-openai:latest with the model at mounted and served as spark25 on port 30000. Send to the chat completions endpoint and return the reply. Verify that the response contains a non-empty message content field. ``` ### Caveat The vLLM example serves the model as spark25 while the SGLang example uses spark2.6, so use the name your own server reports in requests. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/xhtoken-spark-x2-6/serve-spark-x26-api-with-vllm/