DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to adapt the Evo VLM evolution loop to a new VLM task with Evo VLM

Reuse the typed mutation catalog, evaluation protocols, and Pareto selection for a different vision-language task.

  • Research
  • Advanced
  • 1 hour 30 minutes
  • From the README
  • Any agent

Who it is forA researcher who wants to search inference programs for another vision-language task using the same mutate, evaluate, and select pattern.

The problemHand-tuning prompts, image preprocessing, and generation budgets for each task is slow, and the researcher needs a constrained search instead of unrestricted code generation.

The outcomeThe researcher has a working evolution setup for a new task, with mutations, evaluation subsets, and Pareto-selected programs tied to recorded evidence.

What you need first

  • Git and Python 3 installed locally
  • A clone of https://github.com/zzzz7788990213-ops/EvoVLM
  • For GPU inference only: a CUDA-capable GPU, PyTorch, Transformers, and the external Qwen model snapshots described in the runtime guide

Steps

  1. Clone the repo and read the evolution/ folder to see the typed mutations, local provider, search, and selection code.
  2. Review evaluation/ to see how protocols, identity checks, and semantic verification are defined.
  3. Replace the CharXiv-specific data manifests in data/ with your own task subsets.
  4. Adjust the mutation catalog to cover prompts, image preprocessing, generation budgets, and answer extraction for your task.
  5. Run the search on small search and selection subsets, then keep the Pareto-selected programs and their evidence records.

Prompt for your agent

Clone https://github.com/zzzz7788990213-ops/EvoVLM and read the evolution/ and evaluation/ folders in research/evolving-vlm-inference-main. Propose a plan to swap in <path to your task manifest> while keeping the typed mutation catalog and Pareto selection. Verify that every proposed mutation is listed in the existing catalog and does not add new code generation.

Caveat. The README describes a deliberately small recorded search of two generations with at most six children per population, so a new task will need its own evaluation budget and cannot assume the published results transfer.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page