DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to evolve skills on a custom task set you write with Wikiskill

Write tasks.json with auto-graded checks and run the loop on your own agent work.

Who it is forA developer with a repeatable agent job, such as formatting data to a spec, who needs the loop to learn from their own examples.

The problemThe bundled demo bench does not cover your team's tasks, so skill evolution has nothing relevant to learn from.

The outcomeA custom bench where each task has an automatic grader, and an evolve run that gates skill changes against your validation tasks.

What you need first

  • Python 3.10 or newer
  • Hermes Agent installed as the reference backend, or another supported agent CLI (Claude Code, Codex or GitHub Copilot CLI)
  • A configured default model for the agent, for example through OpenRouter using --model and --provider
  • Enough API budget for several iterations (the README cites about $0.09 per iteration on one free-tier run)

Steps

  1. Write a tasks.json file where each task has an id, a split of train or val, a prompt, a sandbox and a grader.
  2. Choose a grader type from exact, contains, json_field or code_stdout that fits each task's output.
  3. Run wikiskill init <name> to create a workspace, then place your tasks in its bench/tasks folder (inferred). (inferred, check the README)
  4. Run wikiskill status to check the workspace loaded the tasks.
  5. Run wikiskill evolve --iters 3 to evolve skills on your tasks.

Prompt for your agent

In the ashutoshsinghpr7/wikiskill repo, add my tasks.json with train and val splits and at least one task per grader type I use. Run `wikiskill evolve --iters 2` and report the baseline score and any rejected proposals. Verify that every task has a grader and that missing deliverables score 0 rather than crashing.

Caveat. The README gives the tasks.json format only as an example, so you must check task ids and splits carefully before a long run.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page