DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to generate renderer video block by block with streaming with Agent Garten

Run the renderer one block at a time with a bounded KV cache for step-by-step generation.

  • Research
  • Advanced
  • 2 hours 30 minutes
  • From the README
  • Any agent

Who it is forA developer building an interactive pipeline who needs the renderer's next block to depend on what was just generated.

The problemRendering a full clip in one pass does not let later conditions react to earlier output or keep memory bounded.

The outcomeA rendered sequence produced block by block, with each block committed to a bounded KV cache before the next one starts.

What you need first

  • A CUDA host with Python 3.11+ and PyTorch 2.9+
  • Access to the Hugging Face weights for Cosmos3-Nano and Wan2.2-TI2V-5B, plus disk space for them
  • Git and the hf command-line tool
  • For training: a JSONL manifest of your own clips with depth and normal data

Steps

  1. pip install -e '.[data,media,wandb,te]'
  2. Download the weights as described in the Weights section, then point WM_COSMOS3_NANO and WM_WAN22_VAE in .env at them.
  3. Load the artifact with load_artifact(inspect_artifact("weights/AgentGarten-renderer")) and call prepare_serving(artifact.student).
  4. Create Cosmos3Stream(artifact.student, StreamPolicy()), call stream.start(anchor, first_condition), then loop over blocks as shown in the Inference section.
  5. Warm the input shapes once before serving, since kernel JIT and graph capture happen on first use.

Prompt for your agent

Using the AgentGarten repo at <repo path>, write and run a script that streams the renderer block by block with Cosmos3Stream for the first-frame anchor at <anchor latent path>. Expected output: a sequence of committed blocks for the conditions in <conditions path>. Verify that the number of committed blocks equals the number of condition blocks before reporting success.

Caveat. The README gives the streaming loop as a code sample with placeholder anchor and condition inputs, so you must build those latents yourself, and the real-time rendering engine and frontend are not released yet.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page