What you can do with AI Engineering From Scratch

4 use cases

• Learn AI engineering lesson by lesson in your agent

  Install the skills so a coding agent quizzes you, teaches one lesson at a time, and tracks progress.

  • AI Engineering From Scratch
  • Claude Code
  • Codex
  • Cursor
  • Software and DevOps
  • Intermediate
  View prompt

  Using the repo rohitg00/ai-engineering-from-scratch, install the skills with npx skills add and run the start-learning skill for <target host>. Expected output: a LEARNING.md file with a phase-by-phase plan. Verify that LEARNING.md exists and names a starting phase.

  Share Save

• Catch a retrieval ranking regression before release

  Compare two retrieval systems query by query and fail a release check when one query gets worse.

  • AI Engineering From Scratch
  • Software and DevOps
  • Intermediate
  View prompt

  In the repo rohitg00/ai-engineering-from-scratch, run the retrieval-evaluation-lab project with --init into learning-artifacts/retrieval-evaluation-lab, then implement stages until the --all --strict grader passes. Expected output: a per-query delta report that lists the query whose most relevant evidence ranks lower. Verify that the grader reports all stages as passing.

  Share Save

• Separate child work from parent time in an agent trace

  Parse a JSONL agent trace, fix overlapping span timing, and output an HTML timeline and regression report.

  • AI Engineering From Scratch
  • Software and DevOps
  • Advanced
  View prompt

  In the repo rohitg00/ai-engineering-from-scratch, run the agent-trace-debugger project with --init into learning-artifacts/agent-trace-debugger, then implement the stages until the --all --strict grader passes. Expected output: an HTML timeline and a JSON regression report for the supplied trace. Verify that per-span token counts are not double counted between parent and child spans.

  Share Save

• Block replayed or altered agent write approvals

  Check role and path on agent tool calls, consume each approval once, and write an audit receipt.

  • AI Engineering From Scratch
  • Software and DevOps
  • Advanced
  View prompt

  In the repo rohitg00/ai-engineering-from-scratch, run the tool-call-firewall project with --init into learning-artifacts/tool-call-firewall, then implement the stages until the --all --strict grader passes. Expected output: an audit receipt for each tool call with the policy decision. Verify that a replayed approval and an altered write are both rejected.

  Share Save

How to use AI Engineering From Scratch

1. Check you have what it needs: Node.js, npx, and python3 installed for the coding agent tutor; A skill-capable coding agent such as Claude Code or Codex, with a writable skills scope; Python 3.10 or newer for the retrieval and firewall projects.
2. Open rohitg00/ai-engineering-from-scratch on GitHub and read the README and license.
3. Run node --version and confirm Node.js is installed.
4. Run npx --version and python3 --version to confirm npx and Python are installed.
5. Run npx skills add rohitg00/ai-engineering-from-scratch, then choose your host and scope when the installer asks.
6. In Codex, use start-learning. In Claude Code, run /start-learning.

These steps come from our top use case, Learn AI engineering lesson by lesson in your agent, which also has a copy-paste agent prompt.

AI Engineering From Scratch alternatives · Best AI repos for Software and DevOps

About this repo

• Who it is forDevelopers who want to build and test AI model internals, retrieval pipelines, and agent runtimes with local graders and kept evaluation results.
• Why it is hot
• LanguagePython
• Sourcegithub.com/rohitg00/ai-engineering-from-scratch · Back to Hot repos

Requirements

• Node.js, npx, and python3 installed for the coding agent tutor
• A skill-capable coding agent such as Claude Code or Codex, with a writable skills scope
• Python 3.10 or newer for the retrieval and firewall projects
• Node.js 22.18 or newer for the agent trace debugger
• A clone of the repository so the project_test.py grader is available

Related repos

• #1 this week Harness2 Harness² is a Python framework from Google Research that refines the harness around a fixed model (prompts, tools, memory, skills, orchestration) so it performs better on agent tasks.
• #8 this week iFixAi iFixAi is an open-source Python tool that audits whether an AI agent is doing the job it is supposed to do, using 60 inspections that combine red-teaming with operational checks.
• #9 this week Orca Orca is a desktop app, with a mobile companion, for running several CLI coding agents side by side, each in its own git worktree, and tracking them in one place.