DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to check paper discovery against the 15-paper smoke corpus with Veri Repro

Run the deterministic discovery and evidence gate on pinned public papers.

  • Research
  • Advanced
  • 30 minutes
  • From the README
  • Any agent

Who it is forA maintainer or evaluator who wants to confirm Veri Repro's discovery and evidence stages work on a fixed set of real papers.

The problemTool results can drift between versions, and there is no quick way to check discovery and evidence behavior on known inputs.

The outcomeA pass or fail result from the deterministic gate for the 15 pinned papers in benchmarks/real-paper-smoke.json.

What you need first

  • Python with pip to install the verirepro package from PyPI
  • A pinned arXiv ID, DOI, PDF URL, or local PDF of the paper
  • Docker, only if you want third-party experiment code to execute
  • An OpenAI-compatible model endpoint, only if you want model-assisted analysis

Steps

  1. Clone the repository with git clone https://github.com/XiantingWu/VeriRepro.git.
  2. Enter the checkout with cd VeriRepro.
  3. Install the development extras with python -m pip install -e '.[dev]'.
  4. Run the discovery and evidence gate with python scripts/run_real_paper_smoke.py --require-top1 --require-evidence. (inferred, check the README)

Prompt for your agent

Using the VeriRepro repo (XiantingWu/VeriRepro), clone the repository, install it with `python -m pip install -e '.[dev]'`, and run `python scripts/run_real_paper_smoke.py --require-top1 --require-evidence`. Expected output is a gate result over the 15 pinned papers. Verify the gate result and report any failing papers without claiming that all 15 were fully reproduced.

Caveat. The README says this gate measures discovery and evidence only and does not claim that all 15 papers were fully reproduced.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page