{
 "schema_version": 1,
 "generated": "2026-10-20",
 "id": "mirros-lab-agentgarten--train-renderer-on-own-clip-dataset",
 "repo": "MirroS-Lab/AgentGarten",
 "repo_url": "https://github.com/MirroS-Lab/AgentGarten",
 "repo_slug": "mirros-lab-agentgarten",
 "title": "Train the renderer on your own depth and normal clips",
 "page_title": "How to train the renderer on your own depth and normal clips with Agent Garten",
 "summary": "Fine-tune the neural renderer on your own clips listed in a JSONL manifest.",
 "who": "A research team with its own video clips and matching depth and normal data who wants a renderer adapted to that domain.",
 "problem": "A general renderer may not match the visual distribution of your own simulated or captured worlds.",
 "outcome": "A renderer training run on your manifest, with output written under the configured output root.",
 "steps": [
  "`pip install -e '.[data,media,wandb,te]'`",
  "Prepare a JSONL manifest with one clip per line, using paths relative to the manifest, for example `{\"video\": \"clips/0001.mp4\", \"depth\": \"clips/0001_depth.npy\", \"normal\": \"clips/0001_normal.mp4\", \"caption\": \"A car drives along a coastal road.\", \"depth_scale\": 1.0}`. (inferred, check the README)",
  "Set the manifest path and output root in .env, along with the weight paths from the Weights section.",
  "`bash scripts/run.sh experiment=cosmos3/bidirectional`",
  "For a multi-node run, set NNODES, NODE_RANK, and MASTER_ADDR, then run `bash scripts/run.sh experiment=cosmos3/bidirectional` on every node. (inferred, check the README)"
 ],
 "agent_prompt": "Using the AgentGarten repo at <repo path>, create a JSONL manifest from the clips in <clips folder> with depth and normal files, then launch bidirectional training with scripts/run.sh. Expected output: training logs and checkpoints written under the output root in .env. Verify that the manifest has one valid line per clip and that the first training iteration starts before reporting success.",
 "agents": [
  "any"
 ],
 "difficulty": "Advanced",
 "time": "4 hours",
 "basis": "readme",
 "caveat": "The README says the code worlds and agent practice loop will be released later, so this use case covers only renderer training on your own clips.",
 "category": "Research",
 "tags": [
  "Research"
 ],
 "lang": "python",
 "requirements": [
  "A CUDA host with Python 3.11+ and PyTorch 2.9+",
  "Access to the Hugging Face weights for Cosmos3-Nano and Wan2.2-TI2V-5B, plus disk space for them",
  "Git and the hf command-line tool",
  "For training: a JSONL manifest of your own clips with depth and normal data"
 ],
 "risk": "none",
 "seen_in_the_wild": false,
 "mentioned_publicly": false,
 "risks": [],
 "best": false,
 "score": 0.8,
 "evidence": [],
 "url": "https://dedupedai.com/use-cases/mirros-lab-agentgarten/train-renderer-on-own-clip-dataset/",
 "md": "https://dedupedai.com/use-cases/mirros-lab-agentgarten/train-renderer-on-own-clip-dataset/index.md",
 "disclaimer": "Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked \"Seen in the wild\", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email hello@dedupedai.com and we will look at it promptly.",
 "api": "https://dedupedai.com/api/v1/usecases/mirros-lab-agentgarten--train-renderer-on-own-clip-dataset.json",
 "api_version": "v1"
}