DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to run offline inference on DeepSeek-V4.1-Flash locally with Deep Infer

Build Deep Infer from source and generate text from the default DeepSeek checkpoint.

Who it is forAn ML engineer with an SM100 GPU or Ascend 950 NPU who needs to test DeepSeek-V4.1-Flash outputs on their own hardware.

The problemGetting a DeepSeek model to run on a specific accelerator needs a build, the right environment variables, and a checkpoint in the right place.

The outcomeA working Deep Infer build that loads DeepSeek-V4.1-Flash and produces generated text from the offline example script.

What you need first

  • Linux with GCC 13.0 or newer and Python 3.12
  • NVIDIA SM100 GPU with CUDA 13, or Huawei Ascend 950 NPU with CANN 9.2
  • Access to the deepseek-ai/DeepSeek-V4.1-Flash-1017 checkpoint on Hugging Face, or a local copy
  • Enough disk space and memory for the checkpoint and the build

Steps

  1. Create and activate an empty Python 3.12 environment.
  2. Run export DEEPINFER_ACCELERATOR=nvidia (use ascend instead on the Ascend platform), and keep it exported in the shell where you run Deep Infer.
  3. Run ./build.sh from the repo root.
  4. (Optional) Run DEEPINFER_HF_MODEL_DIR=/path/to/DeepSeek-V4.1-Flash-1017 to use a local checkpoint instead of the Hugging Face cache.
  5. Run python examples/offline_inference.py and read the output.

Prompt for your agent

Clone https://github.com/deepseek-ai/DeepInfer, set up a Python 3.12 environment, export DEEPINFER_ACCELERATOR=nvidia, run ./build.sh, then run python examples/offline_inference.py using the checkpoint at <checkpoint path>. Report the generated text and the build result, and verify that the script finishes without an import or device error.

Caveat. The README says the open-source release covers offline inference only, and KV cache, prefill and decode disaggregation, and data-parallel load balancing are not included.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for deepseek-ai/DeepInfer and "Run offline inference on DeepSeek-V4.1-Flash locally"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer