DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to use Promptfoo: what you can do with it

What is Promptfoo? Promptfoo is a CLI and library for evaluating and red-teaming LLM-based apps. It runs automated evaluations of prompts and models, compares providers such as OpenAI, Anthropic, Azure, Bedrock and Ollama side by side, scans your own apps for vulnerabilities, automates checks in CI/CD, reviews pull requests for LLM security issues, and runs evals locally so prompts never leave your machine. It is now part of OpenAI and stays open source under MIT.

How do I use it? Install with npm install -g promptfoo, or brew install promptfoo, or use npx promptfoo@latest; then create a starter project with promptfoo init --example getting-started; set your provider key, for example export OPENAI_API_KEY=sk-abc123 with your real key. Full steps below.

Trended earlier

View repo on GitHub

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer

What do you want to do?

3 use cases

Quick start with Promptfoo

  1. Check you have what it needs: Node.js for npm, or Homebrew, or pip; npx works without installing; API keys for the LLM providers you test, set as environment variables; Only run red teaming against apps you own or are authorised to test.
  2. Open promptfoo/promptfoo on GitHub and read the README and license.
  3. Install with npm install -g promptfoo, or brew install promptfoo, or use npx promptfoo@latest
  4. Create a starter project with promptfoo init --example getting-started
  5. Set your provider key, for example export OPENAI_API_KEY=sk-abc123 with your real key
  6. Run cd getting-started then promptfoo eval

These steps come from our top use case, Compare prompts and models with automated evals, which also has a copy-paste agent prompt.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for promptfoo/promptfoo

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Best AI repos for Software and DevOps

About this repo

  • Who it is forDevelopers and teams shipping prompts, RAG systems or agents who want measured quality and security checks instead of trial and error.
  • Why it is hot27k stars, talked about in 2 AI sources we track
  • LanguageTypeScript
  • Sourcegithub.com/promptfoo/promptfoo ยท Back to Hot repos

Requirements

  • Node.js for npm, or Homebrew, or pip; npx works without installing
  • API keys for the LLM providers you test, set as environment variables
  • Only run red teaming against apps you own or are authorised to test

Add this badge to your README

Maintain Promptfoo? Show readers where to find these use cases. Use cases on Deduped

Markdown 

[![Use cases on Deduped](https://dedupedai.com/badge/promptfoo/promptfoo.svg)](https://dedupedai.com/use-cases/promptfoo-promptfoo/?utm_source=github&utm_medium=badge&utm_campaign=promptfoo-promptfoo)

HTML 

<a href="https://dedupedai.com/use-cases/promptfoo-promptfoo/?utm_source=github&amp;utm_medium=badge&amp;utm_campaign=promptfoo-promptfoo"><img src="https://dedupedai.com/badge/promptfoo/promptfoo.svg" alt="Use cases on Deduped" height="20"></a>

A static image, no tracking script. About the badge

Questions about Promptfoo

What is Promptfoo?

Promptfoo is a CLI and library for evaluating and red-teaming LLM-based apps. It runs automated evaluations of prompts and models, compares providers such as OpenAI, Anthropic, Azure, Bedrock and Ollama side by side, scans your own apps for vulnerabilities, automates checks in CI/CD, reviews pull requests for LLM security issues, and runs evals locally so prompts never leave your machine. It is now part of OpenAI and stays open source under MIT. It is an open-source project on GitHub: https://github.com/promptfoo/promptfoo.

How do I use Promptfoo?

Install with npm install -g promptfoo, or brew install promptfoo, or use npx promptfoo@latest; then create a starter project with promptfoo init --example getting-started; set your provider key, for example export OPENAI_API_KEY=sk-abc123 with your real key. The use cases on this page give the full steps and a copy-paste agent prompt for each job.

Who is Promptfoo for?

Developers and teams shipping prompts, RAG systems or agents who want measured quality and security checks instead of trial and error.

What can I do with Promptfoo?

3 use cases so far, for example: Compare prompts and models with automated evals; Red team your own LLM app for vulnerabilities; Block prompt regressions in CI. Each has steps and a copy-paste agent prompt.

What do I need to run Promptfoo?

Node.js for npm, or Homebrew, or pip; npx works without installing; API keys for the LLM providers you test, set as environment variables; Only run red teaming against apps you own or are authorised to test.

What are alternatives to Promptfoo?

Other hot AI repos in Software and DevOps: ECC, Thinkrail, Superpowers, Deep Infer, Instructor.

Is Promptfoo safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.