DEDUPED Hot AI repos, turned into use cases Get 5 a day

What you can do with AI Engineer Notebooks

A set of free, framework-free Jupyter notebooks in Google Colab that teach the applied-LLM stack for AI Engineer and Forward Deployed Engineer roles, running on the free Groq API.

Trended earlier

calmrocks/ai-engineer-notebooks on GitHub

Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk.

Authorization notice. This tool is dual-use. Only use it on software, sites and systems you own or have explicit permission to test, and follow the relevant licenses and terms of service.

What do you want to do?

5 use cases

  • Build a RAG question-answering prototype with a golden set

    Build retrieval-augmented answers from raw API calls, then score them against a golden set.

    View prompt
    Using the repo calmrocks/ai-engineer-notebooks, work through 03-rag/00-what-is-rag.ipynb and 04-evals/01-golden-sets.ipynb against the documents in <placeholder folder>. Report the golden-set pass rate and list the three questions that failed. Verify that each failed answer is checked against the retrieved passages, not only the final text.

  • Catch quality regressions before a prompt or model change ships

    Run an eval suite on every prompt or model change and compare results to the baseline.

    View prompt
    Using the repo calmrocks/ai-engineer-notebooks, run the regression eval in 04-evals/03-regression-evals.ipynb on <placeholder prompt file> before and after a change. Expected output: a table of baseline and candidate scores per metric. Verify that the judge's agreement with human labels was checked before you trust its scores.

  • Test an agent against prompt injection before it touches real data

    Run direct and indirect injection attacks on a tool-using agent, then add defenses and retest.

    View prompt
    Using the repo calmrocks/ai-engineer-notebooks, run the injection examples in 07-security/01-prompt-injection-and-trust.ipynb against <placeholder agent or tool list>. Expected output: a table of attacks with the tool calls each one triggered, before and after defenses. Verify that the test covers both direct and indirect injection.

  • Decide between a fixed pipeline and an agent using cost and latency data

    Compare a fixed pipeline with an agent loop on cost, latency, and quality before you pick one.

    View prompt
    Using the repo calmrocks/ai-engineer-notebooks, build the agent loop from 05-agents/01-agent-loop-from-scratch.ipynb and a fixed pipeline for <placeholder task>. Expected output: a table comparing calls per task, latency, and quality for each. Verify that both versions were scored on the same set of inputs.

  • Get reliable JSON and tool calls from a model in a backend service

    Make a model return valid structured output and handle tool-call errors in a service.

    View prompt
    Using the repo calmrocks/ai-engineer-notebooks, work through 01-model-apis/01-structured-output.ipynb and 01-model-apis/02-tool-calling.ipynb for <placeholder service task>. Expected output: a parser that returns valid objects for every test input and a log of tool-call errors handled. Verify that malformed model output is caught and retried rather than passed on.

How to use AI Engineer Notebooks

  1. Check you have what it needs: A Google account to open Colab notebooks; A free Groq API key from console.groq.com (no credit card needed); The key saved as a Colab secret, as the setup notebook describes.
  2. Open calmrocks/ai-engineer-notebooks on GitHub and read the README and license.
  3. Open the setup notebook at 00-setup/00-environment.ipynb from the Colab badge and add your Groq key as a Colab secret.
  4. Run 03-rag/00-what-is-rag.ipynb to see the retrieve, augment, generate loop in about 15 lines.
  5. Work through 03-rag/01-embeddings-retrieval.ipynb until retrieval returns the right passages for your questions.
  6. Open 04-evals/01-golden-sets.ipynb and build a golden set of questions with expected answers for your RAG system.

These steps come from our top use case, Build a RAG question-answering prototype with a golden set, which also has a copy-paste agent prompt.

AI Engineer Notebooks alternatives · Best AI repos for Software and DevOps

About this repo

Requirements

  • A Google account to open Colab notebooks
  • A free Groq API key from console.groq.com (no credit card needed)
  • The key saved as a Colab secret, as the setup notebook describes
  • Basic Python and comfort reading API responses
Maintain AI Engineer Notebooks? Add the Featured on Deduped badge

Paste this into your README:

[![Featured on Deduped](https://dedupedai.com/badge/calmrocks-ai-engineer-notebooks.svg)](https://dedupedai.com/use-cases/calmrocks-ai-engineer-notebooks/?ref=badge)

More badge styles and HTML

Questions about AI Engineer Notebooks

What is AI Engineer Notebooks?

A set of free, framework-free Jupyter notebooks in Google Colab that teach the applied-LLM stack for AI Engineer and Forward Deployed Engineer roles, running on the free Groq API. It is an open-source project on GitHub: https://github.com/calmrocks/ai-engineer-notebooks.

Who is AI Engineer Notebooks for?

Backend or full-stack engineers moving into AI Engineer, Forward Deployed Engineer, Applied AI, or Solutions Engineer (AI) roles who can already ship production code.

What can I do with AI Engineer Notebooks?

5 use cases so far, for example: Build a RAG question-answering prototype with a golden set; Catch quality regressions before a prompt or model change ships; Test an agent against prompt injection before it touches real data. Each has steps and a copy-paste agent prompt.

What do I need to run AI Engineer Notebooks?

A Google account to open Colab notebooks; A free Groq API key from console.groq.com (no credit card needed); The key saved as a Colab secret, as the setup notebook describes; Basic Python and comfort reading API responses.

What are alternatives to AI Engineer Notebooks?

Other hot AI repos in Software and DevOps: Harness2, iFixAi, Orca, Diagram Design, AI Memory.

Is AI Engineer Notebooks safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.