# AI use cases for vLLM (vllm-project/vllm) Repo: [vllm-project/vllm](https://github.com/vllm-project/vllm) | Use cases: 4 | Page: https://dedupedai.com/use-cases/vllm-project-vllm/ vLLM is a fast, easy-to-use library for LLM inference and serving, with an OpenAI-compatible API server. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Serve an open LLM behind an OpenAI-compatible API: https://dedupedai.com/use-cases/vllm-project-vllm/serve-open-llm-openai-compatible-api/ - Get structured JSON output from a self-hosted model: https://dedupedai.com/use-cases/vllm-project-vllm/generate-structured-json-outputs/ - Serve several LoRA fine-tunes from one base model: https://dedupedai.com/use-cases/vllm-project-vllm/serve-lora-adapters-from-one-base-model/ - Create text embeddings from a served retrieval model: https://dedupedai.com/use-cases/vllm-project-vllm/create-text-embeddings-with-served-models/ ## How to serve an open LLM behind an OpenAI-compatible API with vLLM Repo: [vllm-project/vllm](https://github.com/vllm-project/vllm) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README Turn a Hugging Face model into a local chat endpoint that existing OpenAI client code can call. **Who it is for:** A developer or platform engineer who wants apps to call a self-hosted open-weight model instead of a hosted API. **The problem:** Existing app code is written against an OpenAI-style API, but the team wants the model to run on its own hardware. **The outcome:** A running server at a local URL that answers chat requests from standard OpenAI-style clients. ### Steps 1. Install vLLM with `uv pip install vllm`. 2. Run `vllm download-kernels` to fetch precompiled kernels for faster startup. 3. Start the server with `vllm serve `; the README mentions an OpenAI-compatible API server but does not show this exact command, so check the Quickstart. (inferred, check the README) 4. Send one chat request to the server from your app using an OpenAI-style client. 5. Confirm the reply is returned and note the server's throughput and latency for your model. ### Prompt to paste into your agent ```text Using the vllm-project/vllm repo, install vLLM, start an OpenAI-compatible server for , and send one chat request. Report the server URL and the model's reply. Verify that the model name appears in the server's models list before you finish. ``` ### Caveat The README does not show the serve command, so the exact invocation must be confirmed in the official Quickstart before relying on it. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/vllm-project-vllm/serve-open-llm-openai-compatible-api/ ## How to get structured JSON output from a self-hosted model with vLLM Repo: [vllm-project/vllm](https://github.com/vllm-project/vllm) | Difficulty: Advanced | Time: 1 hour 30 minutes | Basis: From the README Constrain model replies to a schema so downstream code can parse them reliably. **Who it is for:** A data or backend engineer whose pipeline needs machine-readable fields from an LLM, not free text. **The problem:** Free-form model replies break parsers, so each extraction job needs fragile post-processing. **The outcome:** Model responses that follow a requested structure, ready for direct parsing. ### Steps 1. Install vLLM with `uv pip install vllm`. 2. Start a served model with `vllm serve `, as described in the Quickstart. (inferred, check the README) 3. Enable structured output using the xgrammar or guidance backend as described in the official documentation. (inferred, check the README) 4. Send a request that includes your JSON schema and a sample input. 5. Parse the reply with a JSON parser and check that every required field is present. ### Prompt to paste into your agent ```text Using the vllm-project/vllm repo, set up a served model with structured output enabled. Send the sample input with a JSON schema and return the parsed object. Verify that the output parses as JSON and contains every required field. ``` ### Caveat The README names xgrammar and guidance but does not show how to enable them, so the exact option must come from the official docs. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/vllm-project-vllm/generate-structured-json-outputs/ ## How to serve several LoRA fine-tunes from one base model with vLLM Repo: [vllm-project/vllm](https://github.com/vllm-project/vllm) | Difficulty: Advanced | Time: 1 hour 30 minutes | Basis: From the README Run multiple adapters on a single loaded model instead of loading a full copy for each. **Who it is for:** An ML engineer who maintains several fine-tuned variants of the same base model for different customers or tasks. **The problem:** Loading a full model per fine-tune wastes GPU memory and makes each variant expensive to host. **The outcome:** One server that answers requests for each adapter by name, sharing the base model weights. ### Steps 1. Install vLLM with `uv pip install vllm`. 2. Start the server for your base model with LoRA support enabled, following the Quickstart and supported models docs. (inferred, check the README) 3. Register each LoRA adapter path with the server as described in the official documentation. (inferred, check the README) 4. Send one request per adapter and compare the replies to the base model's reply. 5. Record which adapter names the server accepts. ### Prompt to paste into your agent ```text Using the vllm-project/vllm repo, serve the base model with the LoRA adapters at . Send the same prompt to each adapter and the base model, then report the differences. Verify that each adapter name returns a reply. ``` ### Caveat The README confirms multi-LoRA support but not the flags or adapter registration steps, so verify those in the official docs. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/vllm-project-vllm/serve-lora-adapters-from-one-base-model/ ## How to create text embeddings from a served retrieval model with vLLM Repo: [vllm-project/vllm](https://github.com/vllm-project/vllm) | Difficulty: Advanced | Time: 1 hour 30 minutes | Basis: From the README Use an embedding model served by vLLM to turn documents and queries into vectors. **Who it is for:** A researcher or developer building a search or retrieval feature who needs embeddings from an open model. **The problem:** Building a search index needs vectors from a model, and a generic LLM endpoint does not return them. **The outcome:** Embedding vectors for your documents and queries, ready to store in a vector index. ### Steps 1. Install vLLM with `uv pip install vllm`. 2. Serve an embedding model such as E5-Mistral or GTE, following the supported models list in the docs. (inferred, check the README) 3. Send your documents and queries to the embedding endpoint as described in the official documentation. (inferred, check the README) 4. Save the returned vectors with the document IDs they belong to. 5. Run one query and check that the closest document is a relevant match. ### Prompt to paste into your agent ```text Using the vllm-project/vllm repo, serve the embedding model and create vectors for the documents in . Run the query and return the top three documents by similarity. Verify that the top result is relevant to the query. ``` ### Caveat The README lists E5-Mistral, GTE and ColBERT as examples, but the embedding request format is not shown and must be checked in the docs. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/vllm-project-vllm/create-text-embeddings-with-served-models/