DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to create text embeddings from a served retrieval model with vLLM

Use an embedding model served by vLLM to turn documents and queries into vectors.

Who it is forA researcher or developer building a search or retrieval feature who needs embeddings from an open model.

The problemBuilding a search index needs vectors from a model, and a generic LLM endpoint does not return them.

The outcomeEmbedding vectors for your documents and queries, ready to store in a vector index.

What you need first

  • Python with uv or pip installed
  • A supported NVIDIA, AMD or Intel GPU, or an x86, ARM or PowerPC CPU
  • A Hugging Face model you are allowed to download and use

Steps

  1. Install vLLM with uv pip install vllm.
  2. Serve an embedding model such as E5-Mistral or GTE, following the supported models list in the docs. (inferred, check the README)
  3. Send your documents and queries to the embedding endpoint as described in the official documentation. (inferred, check the README)
  4. Save the returned vectors with the document IDs they belong to.
  5. Run one query and check that the closest document is a relevant match.

Prompt for your agent

Using the vllm-project/vllm repo, serve the embedding model <embedding-model-id> and create vectors for the documents in <documents-file>. Run the query <query-text> and return the top three documents by similarity. Verify that the top result is relevant to the query.

Caveat. The README lists E5-Mistral, GTE and ColBERT as examples, but the embedding request format is not shown and must be checked in the docs.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for vllm-project/vllm and "Create text embeddings from a served retrieval model"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer