Who it is forA teacher or researcher deciding how to index course material in Deep Tutor.
The problemIt is unclear which retrieval engine or mode actually finds the right passages for your documents.
The outcomeSaved metric reports you can compare per metric and per case.
What you need first
- Docker, or Python 3.11 to 3.14 with Node.js 20.9+ for the PyPI web app
- An LLM provider and model configured in Settings, Providers
- Optional embedding and search providers for knowledge bases
Steps
- Write a QA set as JSON lines, each with a query and the gold passages an ideal retrieval should return
- Run
deeptutor kb eval <name> --dataset qa.jsonl --top-k 5 [--mode hybrid] [--save baseline.json] - Change the engine or mode and save a second report
- Compare them with
deeptutor kb eval-diff baseline.json candidate.json
Prompt for your agent
For my DeepTutor knowledge base <name>, write 10 QA cases from these documents as qa.jsonl, run `deeptutor kb eval` with --top-k 5 and --save baseline.json, and summarize Recall@k and MRR.
Caveat. Matching uses exact text or lexical overlap; semantic equivalence is not assessed, and PageIndex is excluded.
Use this in your AI agent
Ask your agent
Using the Deduped MCP, get the setup steps for HKUDS/DeepTutor and "Score which retrieval setup works best on your knowledge base"
Add Deduped to Claude Code once
claude mcp add --transport http deduped https://dedupedai.com/mcp
Every Monday: the 5 hottest repos and exactly how to use them. Free, one email a week, unsubscribe anytime.
Get them freeShare: X LinkedIn Hacker News Email Copy link
Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer