DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to run a factuality eval on a prompt before release with Dodoguard

Score an LLM prompt against test cases from the command line in about one minute of run time.

Short answer: Run git clone https://github.com/dodoguardai/dodoguard.git; then run cd dodoguard && npm ci && npm run build; run cp -R examples/openai-eval-factuality ./my-eval && cd my-eval. All steps.

Authorization notice. Dodoguard is dual-use. Only use it on software, sites and systems you own or have explicit permission to test.

Who it is forA developer who changed a prompt or switched models and needs to confirm answer quality did not drop.

The problemChanging a prompt or model can quietly lower answer quality, and spot-checking chat does not catch it.

The outcomeA pass or fail result for each test case, so a prompt or model change can be checked the same way every time.

What you need first

  • Node.js 22.12 or newer for the CLI
  • An OpenAI API key for the OpenAI example, or a local Ollama setup for the local red-team example
  • Docker with Docker Compose for the web console
  • Access to the Dify, Coze, FastGPT, or n8n app you want to check, if you run platform checks

Steps

  1. Run git clone https://github.com/dodoguardai/dodoguard.git
  2. Run cd dodoguard && npm ci && npm run build
  3. Run cp -R examples/openai-eval-factuality ./my-eval && cd my-eval
  4. Set your key with export OPENAI_API_KEY=" ", replacing the space with your real key
  5. Run ../node_modules/.bin/dodoguard eval

Prompt for your agent

In the dodoguard repo, copy examples/openai-eval-factuality to a new folder, set OPENAI_API_KEY, and run ../node_modules/.bin/dodoguard eval from that folder. Report the pass and fail count for each test case. Verify that the run finished and wrote a result for every case before you report success.

Caveat. The quick start uses OpenAI, and the README lists other providers, so check the example config before you switch providers.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for dodoguardai/dodoguard and "Run a factuality eval on a prompt before release"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer