# AI use cases for Brier (Noisyxl/brier)

Repo: [Noisyxl/brier](https://github.com/Noisyxl/brier) | Use cases: 4 | Page: https://dedupedai.com/use-cases/noisyxl-brier/

brier is a prediction ledger for language models that seals each probability with a hash chain before the answer exists, then grades the sealed forecasts with proper scoring rules.

This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL.

- See a forecaster's calibration gap in four seconds: https://dedupedai.com/use-cases/noisyxl-brier/see-calibration-gap-in-four-seconds/
- Put your own model on a sealed forecasting board: https://dedupedai.com/use-cases/noisyxl-brier/score-own-model-against-sealed-questions/
- Detect any edited forecast in a ledger file: https://dedupedai.com/use-cases/noisyxl-brier/check-ledger-for-edited-forecasts/
- Date a forecast ledger on Robinhood Chain: https://dedupedai.com/use-cases/noisyxl-brier/date-forecasts-on-robinhood-chain/

## How to see a forecaster's calibration gap in four seconds with Brier

Repo: [Noisyxl/brier](https://github.com/Noisyxl/brier) | Difficulty: Intermediate | Time: 15 minutes | Basis: From the README

Run the bundled demo and score it to see how stated confidence compares with actual hit rate.

**Who it is for:** A developer or analyst who wants to see how a forecaster's stated probabilities line up with real outcomes before trusting it.

**The problem:** Forecasts are usually judged from memory, and a confident model can look right while being consistently overconfident.

**The outcome:** A scoreboard showing each forecaster's said percentage, how often it was right, the gap between them, and its skill against the base rate.

### Steps

1. Open a terminal in an empty folder with Node.js installed.
2. Run `npx brier demo && npx brier score`.
3. Read the row for each forecaster and compare the said column with the right column.
4. Check the always-yes row to see how a trivial forecaster scores on the same question set.

### Prompt to paste into your agent

```text
Use the Noisyxl/brier repo. Run `npx brier demo && npx brier score` in a clean directory and report the said, right, gap and skill columns for each forecaster. Verify that the output includes an always-yes row before summarizing.
```

### Caveat

The demo shows the ledger's own bundled questions and forecasters, so its numbers describe that set and do not generalize to your domain.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/noisyxl-brier/see-calibration-gap-in-four-seconds/

## How to put your own model on a sealed forecasting board with Brier

Repo: [Noisyxl/brier](https://github.com/Noisyxl/brier) | Difficulty: Advanced | Time: 1 hour | Basis: From the README

Add an API key so your model answers sealed questions and gets graded alongside the built-in panel.

**Who it is for:** A team evaluating whether an LLM's stated probabilities deserve trust before using it in a decision workflow.

**The problem:** Model answers about future events are rarely recorded before the outcome, so their accuracy cannot be checked fairly.

**The outcome:** Your model's sealed answers appear on the same scoreboard as the four built-in forecasters, graded the same way after questions resolve.

### Steps

1. Clone the repo and install it so the Brier command is available.
2. Add your model's API key to a .env file in the project root, as the README describes. (inferred, check the README)
3. Run `brier ask` to pose a question with its date, source and test, then let the panel answer. (inferred, check the README)
4. Wait until the resolution date passes, then run `npx brier score` to see your model's row.

### Prompt to paste into your agent

```text
Work in the Noisyxl/brier repo and add the API key for <model-provider> to .env. Ask one question with a future resolution date, wait for it to settle, and run `npx brier score`. Verify that <model-name> appears on the scoreboard with a Brier score before reporting.
```

### Caveat

Questions only settle after their resolution date, so a meaningful score requires waiting weeks and building up a set of resolved questions.

Risk, credentials: Your model API key is stored in a local .env file, so keep that file out of version control and never paste the key into shared logs.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/noisyxl-brier/score-own-model-against-sealed-questions/

## How to detect any edited forecast in a ledger file with Brier

Repo: [Noisyxl/brier](https://github.com/Noisyxl/brier) | Difficulty: Beginner | Time: 15 minutes | Basis: From the README

Run the verify command to find the exact line where a sealed probability was changed.

**Who it is for:** Someone who received a forecast ledger from another person or team and needs to confirm no answer was edited after it was sealed.

**The problem:** A forecast file can be quietly altered after the fact, which makes past accuracy claims impossible to trust.

**The outcome:** A verification result confirming the chain is intact, or naming the first line where the hashes stop matching.

### Steps

1. Place the ledger file in the Brier project folder.
2. Run `brier ledger --verify`.
3. If it reports a break, note the line number it names.
4. Compare that line against the original source you trust before drawing any conclusion.

### Prompt to paste into your agent

```text
In the Noisyxl/brier repo, run `brier ledger --verify` on the ledger file at <path-to-ledger>. Report whether the chain is intact, and if not, the exact line the command names. Verify the output text before stating a result.
```

### Caveat

The hash chain proves the file was not edited, but it does not by itself prove when a forecast was sealed, which is what anchoring is for.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/noisyxl-brier/check-ledger-for-edited-forecasts/

## How to date a forecast ledger on Robinhood Chain with Brier

Repo: [Noisyxl/brier](https://github.com/Noisyxl/brier) | Difficulty: Advanced | Time: 1 hour | Basis: From the README

Anchor the ledger head hash to a block so its existence date does not rest on the author's word.

**Who it is for:** A researcher or fund-adjacent analyst who needs proof that forecasts existed before an outcome, using a clock they do not control.

**The problem:** A hash chain shows a file was not edited but not when it existed, so an author could still claim a forecast was sealed earlier than it was.

**The outcome:** An anchor record with the block's time, which a third party can check against the transaction in an explorer.

### Steps

1. Run `brier anchor` to see the head, the bytes and the command to send them.
2. Broadcast the transaction from your own wallet, using the command the anchor step printed.
3. Record the transaction by running `brier anchor 0x<tx-hash>`, filling in your transaction hash. (inferred, check the README)
4. Run `brier anchor --check` to re-read every anchor and confirm it still matches the ledger.

### Prompt to paste into your agent

```text
In the Noisyxl/brier repo, run `brier anchor` and report the head hash and the command it prints. After the owner broadcasts the transaction, run `brier anchor --check` and verify that every anchor still matches before reporting success.
```

### Caveat

Brier prepares the bytes but never broadcasts them, so the anchor only exists once you send the transaction yourself from a wallet you control.

Risk, credentials: Broadcasting the anchor transaction uses your own wallet and its funds, so only sign the bytes Brier prints after you have checked them.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/noisyxl-brier/date-forecasts-on-robinhood-chain/