DEDUPEDWhat do you want to do?
3 use cases
-
Scaffold a RAG evaluation project in one command
Use the Ragas quickstart CLI to generate a complete example project for evaluating a RAG system.
View prompt
Using vibrantlabsai/ragas, install it with `pip install ragas`, run `ragas quickstart rag_eval -o ./my-project`, then wire the generated project to <my RAG pipeline> and explain each file it created.
-
Score LLM outputs with a custom pass or fail metric
Define an Aspect Critique style metric with DiscreteMetric and let an LLM judge label each response.
View prompt
With vibrantlabsai/ragas, write a script that defines a DiscreteMetric named summary_accuracy with allowed values accurate and inaccurate, scores each response in <my file of summaries> with metric.ascore, and writes value and reason to a CSV.
-
Build test data and feedback loops for an LLM app
Use Ragas test data generation to cover more scenarios, then rescore after each change.
View prompt
Using vibrantlabsai/ragas, generate a test dataset from <my documents folder> with its test data generation feature, score my app with the pre-built metrics, and save a baseline report I can compare against next week.
Quick start with Ragas
- Check you have what it needs: Python with pip; An LLM provider key for LLM-based metrics, for example OPENAI_API_KEY in the README example; uv or another way to run the quickstart CLI is optional.
- Open vibrantlabsai/ragas on GitHub and read the README and license.
- Install Ragas with
pip install ragas - Run
ragas quickstartto list the available templates - Run
ragas quickstart rag_eval -o ./my-projectto create the RAG evaluation project - Open the project, point it at your own RAG pipeline and run it
These steps come from our top use case, Scaffold a RAG evaluation project in one command, which also has a copy-paste agent prompt.
Use this in your AI agent
Ask your agent
Using the Deduped MCP, get the setup steps for vibrantlabsai/ragas
Add Deduped to Claude Code once
claude mcp add --transport http deduped https://dedupedai.com/mcp
Every Monday: the 5 hottest repos and exactly how to use them. Free, one email a week, unsubscribe anytime.
Get them freeAbout this repo
- Who it is forDevelopers shipping RAG systems or LLM features who want objective, repeatable scores instead of eyeballing outputs.
- Why it is hot16k stars, talked about in 2 AI sources we track
- LanguagePython
- Sourcegithub.com/vibrantlabsai/ragas ยท Back to Hot repos
Requirements
- Python with pip
- An LLM provider key for LLM-based metrics, for example OPENAI_API_KEY in the README example
- uv or another way to run the quickstart CLI is optional
Add this badge to your README
Maintain Ragas? Show readers where to find these use cases. 
Markdown
[](https://dedupedai.com/use-cases/vibrantlabsai-ragas/?utm_source=github&utm_medium=badge&utm_campaign=vibrantlabsai-ragas)
HTML
<a href="https://dedupedai.com/use-cases/vibrantlabsai-ragas/?utm_source=github&utm_medium=badge&utm_campaign=vibrantlabsai-ragas"><img src="https://dedupedai.com/badge/vibrantlabsai/ragas.svg" alt="Use cases on Deduped" height="20"></a>
A static image, no tracking script. About the badge
Questions about Ragas
What is Ragas?
Ragas is a Python toolkit for evaluating LLM applications. It offers LLM-based and traditional metrics, automatic test data generation, integrations with frameworks like LangChain, and a `ragas quickstart` command that scaffolds ready-made evaluation projects such as a RAG evaluation template. It is an open-source project on GitHub: https://github.com/vibrantlabsai/ragas.
How do I use Ragas?
Install Ragas with pip install ragas; then run ragas quickstart to list the available templates; run ragas quickstart rag_eval -o ./my-project to create the RAG evaluation project. The use cases on this page give the full steps and a copy-paste agent prompt for each job.
Who is Ragas for?
Developers shipping RAG systems or LLM features who want objective, repeatable scores instead of eyeballing outputs.
What can I do with Ragas?
3 use cases so far, for example: Scaffold a RAG evaluation project in one command; Score LLM outputs with a custom pass or fail metric; Build test data and feedback loops for an LLM app. Each has steps and a copy-paste agent prompt.
What do I need to run Ragas?
Python with pip; An LLM provider key for LLM-based metrics, for example OPENAI_API_KEY in the README example; uv or another way to run the quickstart CLI is optional.
What are alternatives to Ragas?
Other hot AI repos in Software and DevOps: Deep Infer, Thinkrail, ECC, Instructor, Aider.
Is Ragas safe to run?
Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.