DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to audit whether Phil beats the market price with Phil

Run the offline score report to check if Phil's probabilities beat Polymarket's own prices.

Short answer: git clone https://github.com/bennyjo/phil.git; then cd phil; python3 core/score.py --skip-mtm. All steps.

  • Top pick
  • Money
  • Intermediate
  • 15 minutes
  • From the README
  • Claude Code

Who it is forA researcher or forecaster who wants evidence, not P&L, on whether an AI agent forecasts better than the market.

The problemProfit alone does not show skill. You need to know whether the agent's probability is a better forecast than the price the market already showed.

The outcomeA calibration and P&L report from the journal, with the brier_delta metric that compares the agent's forecasts to market prices.

What you need first

  • Python 3 (CI runs on Python 3.12); the engine uses the standard library only
  • git and a clone of the repository
  • A Claude Code account and the `claude` command on your PATH, for running cycles
  • A fork with write access to origin, or no origin remote for local-only runs
  • Pearl Connect installed from pearl.you/connect and a funded Safe, only for real-money mode

Steps

  1. git clone https://github.com/bennyjo/phil.git
  2. cd phil
  3. python3 core/score.py --skip-mtm (inferred, check the README)
  4. Read the calibration section and find brier_delta.
  5. Open the newest file in journal/retros/ and compare its conclusions with the score.

Prompt for your agent

Clone https://github.com/bennyjo/phil, run `python3 core/score.py --skip-mtm` offline, and report the calibration results and the brier_delta value. Verify that the report states whether the agent's probabilities beat the market price, and say so if it does not.

Caveat. The score is from paper trades that Phil made in its own repository, so the result describes this one experiment and does not show general trading skill.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for bennyjo/phil and "Audit whether Phil beats the market price"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer