DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to track an object through a video with Sam2

Prompt an object on one frame and propagate its mask through the whole video.

Short answer: Install SAM 2 and download checkpoints as in the README; then create a predictor with build_sam2_video_predictor and call predictor.init_state on your video; add a point or box with predictor.add_new_points_or_box on one frame. All steps.

  • Research
  • Advanced
  • 1 to 2 hours
  • From the README
  • Claude Code
  • Codex
  • Cursor
  • Gemini CLI

Who it is forA video editor or analyst who needs a mask on a moving object.

The problemRotoscoping frame by frame takes hours.

The outcomeMasklets for the object across every frame.

What you need first

  • Python 3.10 or newer with torch 2.5.1 and torchvision 0.20.1 or newer
  • An NVIDIA GPU with CUDA is recommended; Windows users should use WSL with Ubuntu
  • A model checkpoint from the download script or Hugging Face

Steps

  1. Install SAM 2 and download checkpoints as in the README
  2. Create a predictor with build_sam2_video_predictor and call predictor.init_state on your video
  3. Add a point or box with predictor.add_new_points_or_box on one frame
  4. Loop over predictor.propagate_in_video(state) to get masks for every frame

Prompt for your agent

With facebookresearch/sam2, write a script that uses build_sam2_video_predictor, adds a box prompt on frame 0 of <my video frames folder>, propagates with propagate_in_video, and writes an overlay video.

Caveat. Setting vos_optimized=True enables full model compilation for a major speedup, per the latest updates.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for facebookresearch/sam2 and "Track an object through a video"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer