DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to measure human-written rewrite pairs with the benchmark script with Not AI

Compare original and rewritten text pairs using the repo's reproducible diagnostics.

Short answer: Clone https://github.com/udaysharmadev/Not-Ai and move into the folder; then Place your human-provided original and rewritten pairs in the fixture format the benchmark script expects, with metadata for the expected action; run python3 scripts/benchmark.py against the fixtures (inferred, check the script's help output for options). All steps.

  • Content
  • Advanced
  • 60 minutes
  • From the README
  • Any agent

Authorization notice. Not AI is dual-use. Only use it on software, sites and systems you own or have explicit permission to test.

Who it is forA researcher or editor who has human-provided original and rewritten pairs and wants repeatable diagnostics for a regression check.

The problemJudging whether a rewrite preserved content is hard to repeat by eye, and the team needs consistent numbers.

The outcomeA benchmark report with token-overlap proxy, structural delta, readability delta, word-count change, and protected-literal preservation.

What you need first

  • A terminal and an agent that supports the skill, such as Claude Code, Codex, or Cursor
  • Python 3 to run the gate script from a local clone of the repo (inferred, the README uses python3 and says the gate is dependency-free)
  • The draft text saved as a plain text file

Steps

  1. Clone https://github.com/udaysharmadev/Not-Ai and move into the folder. (inferred, check the README)
  2. Place your human-provided original and rewritten pairs in the fixture format the benchmark script expects, with metadata for the expected action.
  3. Run python3 scripts/benchmark.py against the fixtures (inferred, check the script's help output for options). (inferred, check the README)
  4. Read the token-overlap, structural, readability, and protected-literal results for each pair.
  5. Treat the numbers as diagnostics and review any pair that changes meaning by hand.

Prompt for your agent

In the udaysharmadev/Not-Ai repo, run scripts/benchmark.py on the fixture pairs in <path to fixtures> and return the per-pair token-overlap, structural delta, readability delta, and protected-literal results. Verify that every pair reports a result and that no pair is labeled as a quality judgment.

Caveat. The README says these figures are reproducible diagnostics, not proof that a rewrite is good or semantically identical.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for udaysharmadev/Not-Ai and "Measure human-written rewrite pairs with the benchmark script"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer