DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to turn your own video of an object into a 3D capture with Agentic STAR

Extract frames, add object masks, run Pi3X cameras, then let the agent rebuild the object.

  • Research
  • Advanced
  • 6 hours
  • From the README
  • Claude Code

Who it is forA robotics or vision researcher with a phone video of a hinged or articulated object who needs a joint-level model.

The problemMonocular video gives camera motion and appearance but not a posed, jointed 3D model that can be inspected or reused.

The outcomeA capture directory and an agent run that produces mesh/object.glb with named parts and mesh/pose.json with per-frame joint states.

What you need first

  • Linux x86_64 host with an NVIDIA GPU
  • micromamba, git, curl, tar, xz, python3, tmux, bubblewrap, iproute2 and ffmpeg installed on the host
  • Root or CAP_NET_ADMIN for the sandbox network namespace
  • An Anthropic API key for Claude Code or an OpenAI API key for Codex
  • claude and/or codex available on PATH

Steps

  1. Run tools/video_to_frames.sh clip.mov my_input --every 5.
  2. Add binary object masks as my_input/mask_object/ .png for those frames, and optionally mask_hand/ .png.
  3. Run micromamba run -n pi3x python tools/make_capture.py --src my_input --out captures/my_object.
  4. Run tools/run_kf.sh --agent claude captures/my_object:0.
  5. Check the output under runs/ for mesh/object.glb and mesh/pose.json.

Prompt for your agent

Using the agenticSTAR repo, convert <video-file> into a capture with tools/video_to_frames.sh and tools/make_capture.py, then run tools/run_kf.sh with --agent claude on the resulting capture. Expect mesh/object.glb and mesh/pose.json under runs/. Verify that the mask folder has one PNG per extracted frame before starting the run.

Caveat. Masks can come from any segmenter, but you must supply them yourself, and the README does not describe how to produce them.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page