# AI use cases for Veri Repro (XiantingWu/VeriRepro) Repo: [XiantingWu/VeriRepro](https://github.com/XiantingWu/VeriRepro) | Use cases: 4 | Page: https://dedupedai.com/use-cases/xiantingwu-verirepro/ VeriRepro is a Python package that checks whether a computational paper's claims are backed by its repository, environment, and execution evidence, and writes a PASS, FAIL, or PARTIAL reproduction report. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Plan a paper's reproduction without running its code: https://dedupedai.com/use-cases/xiantingwu-verirepro/plan-paper-reproduction-without-running-code/ - Verify a paper's metrics by running its code in a sandbox: https://dedupedai.com/use-cases/xiantingwu-verirepro/verify-paper-metrics-in-sandbox/ - Check paper discovery against the 15-paper smoke corpus: https://dedupedai.com/use-cases/xiantingwu-verirepro/validate-paper-discovery-with-smoke-gate/ - Score agent reproduction runs with the ReproBench adapter: https://dedupedai.com/use-cases/xiantingwu-verirepro/score-agent-reproduction-tasks-reprobench/ ## How to plan a paper's reproduction without running its code with Veri Repro Repo: [XiantingWu/VeriRepro](https://github.com/XiantingWu/VeriRepro) | Difficulty: Intermediate | Time: 15 minutes | Basis: From the README Map a paper's claims to repository and environment evidence with no third-party code execution. **Who it is for:** A researcher or reviewer who wants to see what it would take to reproduce a computational paper before running anyone's experiment code. **The problem:** Reading a paper and its repository by hand takes time, and running unknown code is risky before you know what it needs. **The outcome:** A reproduction workspace with planning artifacts such as repository-plan.json and environment-plan.json, plus a report, and no third-party experiment execution. ### Steps 1. Install the published package with `python -m pip install verirepro`. 2. Check the local setup with `verirepro doctor --json`. 3. Plan the reproduction of a pinned smoke-corpus paper with `verirepro plan 2103.00020v1`. 4. Run the planning-only reproduction with `verirepro reproduce 2103.00020v1 --no-execute --no-llm` and open the generated report.json or report.md. ### Prompt to paste into your agent ```text Using the VeriRepro repo (XiantingWu/VeriRepro), install the verirepro package and run `verirepro reproduce --no-execute --no-llm` for the paper . Confirm the workspace contains repository-plan.json and environment-plan.json and that no experiment code was executed, then summarize the report.json status and the planned steps. ``` ### Caveat The planning-only path does not run the paper's experiments, so it cannot tell you whether the scientific result reproduces. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/xiantingwu-verirepro/plan-paper-reproduction-without-running-code/ ## How to verify a paper's metrics by running its code in a sandbox with Veri Repro Repo: [XiantingWu/VeriRepro](https://github.com/XiantingWu/VeriRepro) | Difficulty: Advanced | Time: 1 hour | Basis: From the README Run a paper's repository in the Docker boundary and get a PASS, FAIL, or PARTIAL verdict. **Who it is for:** A reproducibility engineer or lab member who needs a grounded verdict on whether a paper's reported numbers can be reproduced. **The problem:** A script exiting successfully does not show the paper's scientific result was reproduced, and manual comparison of metrics is error-prone. **The outcome:** A verdict in report.json and report.md, with metrics, figures, tables, and stdout and stderr logs preserved in the workspace. ### Steps 1. Install the package with `python -m pip install verirepro`. 2. Confirm the sandbox prerequisites with `verirepro doctor --strict` before any third-party code runs. 3. Run the reproduction of the target paper with `verirepro reproduce `, adding `--allow-network` or `--allow-gpu` only when you have decided to grant those authorities. (inferred, check the README) 4. Read the verdict in report.json and check the metric comparisons and the experiment.stdout.log and experiment.stderr.log files before you trust the result. ### Prompt to paste into your agent ```text Using the VeriRepro repo (XiantingWu/VeriRepro), run `verirepro doctor --strict` and then `verirepro reproduce ` for the paper . Expected output is a workspace with report.json and a PASS, FAIL, or PARTIAL status. Verify that any PASS is backed by evidence-authorized scalar comparisons and not just a zero exit code. ``` ### Caveat Veri Repro states that Docker is an execution boundary, not a formal sandbox proof, and that a PARTIAL verdict is intentional when evidence is insufficient. Risk, remote script: This runs third-party experiment code from the paper's repository, so use disposable infrastructure if the repository might be hostile, since Docker is not a formal sandbox proof. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/xiantingwu-verirepro/verify-paper-metrics-in-sandbox/ ## How to check paper discovery against the 15-paper smoke corpus with Veri Repro Repo: [XiantingWu/VeriRepro](https://github.com/XiantingWu/VeriRepro) | Difficulty: Advanced | Time: 30 minutes | Basis: From the README Run the deterministic discovery and evidence gate on pinned public papers. **Who it is for:** A maintainer or evaluator who wants to confirm Veri Repro's discovery and evidence stages work on a fixed set of real papers. **The problem:** Tool results can drift between versions, and there is no quick way to check discovery and evidence behavior on known inputs. **The outcome:** A pass or fail result from the deterministic gate for the 15 pinned papers in benchmarks/real-paper-smoke.json. ### Steps 1. Clone the repository with `git clone https://github.com/XiantingWu/VeriRepro.git`. 2. Enter the checkout with `cd VeriRepro`. 3. Install the development extras with `python -m pip install -e '.[dev]'`. 4. Run the discovery and evidence gate with `python scripts/run_real_paper_smoke.py --require-top1 --require-evidence`. (inferred, check the README) ### Prompt to paste into your agent ```text Using the VeriRepro repo (XiantingWu/VeriRepro), clone the repository, install it with `python -m pip install -e '.[dev]'`, and run `python scripts/run_real_paper_smoke.py --require-top1 --require-evidence`. Expected output is a gate result over the 15 pinned papers. Verify the gate result and report any failing papers without claiming that all 15 were fully reproduced. ``` ### Caveat The README says this gate measures discovery and evidence only and does not claim that all 15 papers were fully reproduced. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/xiantingwu-verirepro/validate-paper-discovery-with-smoke-gate/ ## How to score agent reproduction runs with the ReproBench adapter with Veri Repro Repo: [XiantingWu/VeriRepro](https://github.com/XiantingWu/VeriRepro) | Difficulty: Advanced | Time: 1 hour | Basis: From the README Turn each agent's reproduction task into a result file and a summary with the ReproBench commands. **Who it is for:** An evaluator building or comparing AI agents on paper reproduction tasks who needs consistent, auditable results. **The problem:** Agents report reproduction outcomes in different formats, which makes them hard to compare and audit. **The outcome:** A per-task result JSON with outcome, comparisons, failure taxonomy, and provenance, plus one summary JSON across tasks. ### Steps 1. Install the package with `python -m pip install verirepro`. 2. Prepare a task JSON file for one reproduction task, using only the allowed paper sources and confined paths. 3. Run the adapter on the task with `verirepro-reprobench task.json --output result.json`. 4. Summarize all results with `verirepro-reprobench-summary results/*.json --output summary.json` and check that each result records its operator interventions. ### Prompt to paste into your agent ```text Using the VeriRepro repo (XiantingWu/VeriRepro), run `verirepro-reprobench task.json --output result.json` for the task file , then run `verirepro-reprobench-summary results/*.json --output summary.json`. Expected output is a per-task result and a combined summary. Verify that the task file had no local, file, or insecure HTTP paper sources and that operator interventions are recorded. ``` ### Caveat The adapter rejects local, file, and insecure HTTP paper sources, so tasks must use allowed paper sources to be accepted. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/xiantingwu-verirepro/score-agent-reproduction-tasks-reprobench/