Choose an independent judge so the agent is not grading its own homework.
Intermediate
45 minutes
Any agent
Seen in the wild
Evidence (similar to what people reported): X, Oct 2026
WhoTeam lead who does not trust a model to score its own output
The problemIf the same vendor's model both runs and grades the agent, blind spots can go unnoticed.
The outcomeAn audit graded by a separate vendor's judge (or a multi-judge ensemble), giving a more independent score.
Steps
Install extras for both the tested and judge providers, e.g. pip install "ifixai[openai]"
Run ifixai setup and pick a self, single independent vendor, or ensemble judge
Run: ifixai run --dry-run to inspect the call estimate first
Run: ifixai run and compare the scorecard to a self-graded run if you have one
Prompt for your agent
Use ifixai-ai/iFixAi to audit <my agent> with <vendor A> as the tested model and <vendor B> as the judge. Do a --dry-run first, then a full run, and note any category where the independent judge scored lower. Verify the judge vendor in the report matches what I chose.
Caveat. Using a second vendor means a second API key and added model-call cost on top of the tested model.
WhoFounder or engineer about to put a support or ops agent in front of real customers
The problemYour tests check that answers look good, but nobody has checked whether the agent stays inside the rules it was given.
The outcomeA graded scorecard (A to F, broken down by category) that shows where the agent breaks policy before users find out.
Steps
Install the package with the extra for the provider you test: pip install "ifixai[openai]"
Run the guided wizard: ifixai setup
Run the audit with no flags: ifixai run
Open the reports in ./ifixai-results/ and read the scorecard by category
Prompt for your agent
Use the ifixai-ai/iFixAi repo to audit my agent at <agent endpoint or provider and model>. Run the guided setup, then a default run, and summarize the scorecard by category. Verify the report files were written to ./ifixai-results/ before you report results.
Caveat. The engine is free but every check makes paid model calls, so use --dry-run first to see the estimated call count.
Install the plugin and ask in plain English; the agent builds the fixture, runs it and walks you through the scorecard.
Beginner
30 minutes
Claude Code
Codex
Seen in the wild
Evidence (similar to what people reported): X, Oct 2026
WhoDeveloper already working in Claude Code or Codex who wants a guided, explained audit without learning flags
The problemYou do not want to hand-write configs or decode raw JSON reports just to learn whether your agent behaves.
The outcomeA completed audit with an interactive scorecard and a plain-English walkthrough of what failed.
Steps
In Claude Code run: /plugin marketplace add ifixai-ai/iFixAi
Then run: /plugin install ifixai@ifixai-community
Ask the agent: run iFixAi on my setup (or type /ifixai:ifixai)
Confirm the stated cost, let it run, then read the scorecard walkthrough
Prompt for your agent
Use the ifixai plugin from ifixai-ai/iFixAi to audit <my agent or provider>. Discover my config, tell me the cost before anything is billed, run the diagnostic, then explain the scorecard. Verify the final result file exists and cite which categories scored worst.
Caveat. Restart Claude Code or run /reload-plugins if the plugin does not appear; on Codex you must approve the plugin's hook once.
WhoEngineering manager evaluating whether a coding agent is safe to trust with team repos
The problemYou can't tell from demos whether the agent silently fails, fakes tool use or falls for injected instructions.
The outcomeA quick graded scorecard covering checks like prompt injection, fake tool use and silent failures.
Steps
Install your provider extra: pip install "ifixai[anthropic]"
Run the wizard: ifixai setup
Run the default 10-inspection suite: ifixai run
Read which categories lost points and decide what to fix first
Prompt for your agent
Use ifixai-ai/iFixAi to run the default audit on my coding agent <agent or model name>. Summarize the grade and the three weakest categories. Verify the run used the default suite and not a full run by checking the report.
Caveat. The default run covers 10 inspections; a full run of all 60 takes longer and costs more in model calls.
WhoEngineer about to give an internal agent write or delete rights on real systems
The problemMost agent testing checks answers and never checks whether the agent stays within what it is allowed to do.
The outcomeEvidence on whether the agent's permission-related checks pass, so you can decide to grant or withhold access.
Steps
Install the extra for your setup: pip install "ifixai[openai]"
Run ifixai setup and choose your agent's real endpoint
Run: ifixai run
Review the permission and policy categories in the scorecard before changing any access
Prompt for your agent
Use ifixai-ai/iFixAi against my agent's HTTP endpoint <endpoint URL> to check whether it respects role and permission limits. Report the permission-related check results from ./ifixai-results/ and verify which checks failed before I grant it delete rights.
Caveat. A passing audit is not proof of safety; it is one independent check, so keep human approval on destructive actions.
Scaffold a native /ifixai-skill command into the coding agent you already use, no plugin needed.
Beginner
30 minutes
From the README
Cursor
Gemini CLI
Any agent
WhoDeveloper using Cursor, VS Code, Windsurf, Gemini or another agent that lacks the Claude Code plugin
The problemYou want the guided, agent-driven audit but your editor has no plugin install path.
The outcomeA working /ifixai-skill command in your agent that reads your setup, shows the cost, and runs the audit after you approve.
Steps
Scaffold the skill for your agent: uvx ifixai install --agents cursor
Or scaffold every agent: uvx ifixai install --agents all
Check supported agents and file locations: uvx ifixai install --list
Run /ifixai-skill in your agent and approve the cost shown by the free --dry-run
Prompt for your agent
Scaffold the iFixAi skill for my agent from ifixai-ai/iFixAi using uvx ifixai install --agents <agent slug>, then run /ifixai-skill. Show me the --dry-run cost before running. Verify the skill file landed where uvx ifixai install --list says it should.
Caveat. Auto-detect only finds agents whose folder already exists, so on a fresh project name the agent explicitly with --agents.
Script the audit with explicit CLI flags so every release is graded and the report is saved as an artifact.
Intermediate
1 hour
From the README
Any agent
WhoPlatform or DevOps engineer who ships agent changes weekly and wants a repeatable policy check
The problemPrompt or tool changes can quietly break permission rules, and nobody notices until production.
The outcomeA scripted, flag-driven audit run in CI that writes JSON and Markdown reports for every change.
Steps
Install the provider extra in your CI image: pip install "ifixai[anthropic]"
Export the provider API key as an environment variable in CI secrets
Run an explicit-flag audit (see the Explicit flags section of the README for the exact options)
Upload ./ifixai-results/ as a build artifact and fail the job on a low grade
Prompt for your agent
Add an ifixai-ai/iFixAi audit step to the CI config in <repo path>, using explicit CLI flags and an API key from secrets. Save the reports from ./ifixai-results/ as an artifact. Verify with a --dry-run that the step works before enabling it.
Caveat. The README's explicit-flag example was cut off in the text provided, so check the repo docs for the exact flag names before wiring up CI.
Use the offline Policy Lab to find the shortest harmful sequence of permitted actions and test permission fixes.
Advanced
1.5 hours
From the README
Any agent
WhoSecurity-minded engineer reviewing a refund or payout workflow where each step is allowed but the combination is not
The problemSingle permissions look fine, yet a chain of permitted actions can still cause harm.
The outcomeA shortest harmful sequence found offline, plus a check that your permission repair still allows legitimate work.
Steps
Open the Policy Lab doc at docs/business-policy-lab.md in the repo
Follow its offline refund-store example
Adapt the example's policy and actions to your own workflow
Replay with an explicit local callable to confirm the repair holds
Prompt for your agent
Read docs/business-policy-lab.md in ifixai-ai/iFixAi and run the offline refund-store example. Then adapt it to <my workflow>, report the shortest harmful sequence, and verify the proposed permission repair still allows the legitimate completion goal.
Caveat. The README only introduces Policy Lab briefly, so details and commands live in docs/business-policy-lab.md and should be confirmed there.
Who it is forFor teams and solo builders who have an AI agent running or about to ship and want an independent check that it follows business rules, roles and permissions, not only that its answers look right.
DEDUPED