# AI use cases for Typesafe Computer Use (awlevin/typesafe-computer-use) Repo: [awlevin/typesafe-computer-use](https://github.com/awlevin/typesafe-computer-use) | Use cases: 4 | Page: https://dedupedai.com/use-cases/awlevin-typesafe-computer-use/ typesafe-computer-use (jev) is a macOS tool that takes a plain-English goal, reads the screen with OCR, asks a small TypeSafe classifier for the next action, and clicks. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Dry-run a goal to see the next step before any clicks: https://dedupedai.com/use-cases/awlevin-typesafe-computer-use/dry-run-a-goal-before-letting-it-click/ - Run a multi-step browser task on your Mac: https://dedupedai.com/use-cases/awlevin-typesafe-computer-use/automate-a-multi-step-browser-task/ - Find out why a run failed and replay it offline: https://dedupedai.com/use-cases/awlevin-typesafe-computer-use/debug-a-failed-run-offline/ - Run jev on OSWorld desktop tasks to compare results: https://dedupedai.com/use-cases/awlevin-typesafe-computer-use/benchmark-jev-on-osworld-tasks/ ## How to dry-run a goal to see the next step before any clicks with Typesafe Computer Use Repo: [awlevin/typesafe-computer-use](https://github.com/awlevin/typesafe-computer-use) | Difficulty: Intermediate | Time: 30 minutes | Basis: From the README Preview what the agent would do for one plain-English goal without touching your mouse or keyboard. **Who it is for:** A Mac user who wants to test whether the agent understands a task before giving it control of the screen. **The problem:** Letting an automation tool drive a real mouse and keyboard is risky, and it is hard to know what it will pick without seeing its plan first. **The outcome:** You see one step's decision and the payload it would act on, and you decide whether to run with --act. ### Steps 1. Run `git clone https://github.com/awlevin/typesafe-computer-use` 2. Run `cd typesafe-computer-use` 3. Run `uv sync` 4. Run `cp .env.example .env` and fill in TYPESAFE_API_KEY 5. Grant your terminal Screen Recording and Accessibility in System Settings, then Privacy & Security 6. Run `uv run clicker "open the Playground"` and read the printed step ### Prompt to paste into your agent ```text In the typesafe-computer-use repo, run `uv run clicker ""` with no --act flag so it performs a dry run of one step. Report the action it would take and the reason it printed. Verify that no mouse or keyboard event was sent by checking that the command exited without the --act flag set. ``` ### Caveat The project is in beta and the README says to start with a dry run because it drives your real mouse and keyboard. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/awlevin-typesafe-computer-use/dry-run-a-goal-before-letting-it-click/ ## How to run a multi-step browser task on your Mac with Typesafe Computer Use Repo: [awlevin/typesafe-computer-use](https://github.com/awlevin/typesafe-computer-use) | Difficulty: Intermediate | Time: 45 minutes | Basis: From the README Give the agent a goal in plain English and let it click through a site for up to a set number of steps. **Who it is for:** A Mac user with a repetitive click-through task in Chrome, such as reaching a page after several menus. **The problem:** Repeating the same series of clicks across several pages takes time, and frontier-model computer use is slow and costly for each step. **The outcome:** The agent works toward the goal, stops on done, low confidence, a stall, or the step limit, and prints an answer when the writer is used. ### Steps 1. Complete the install steps and grant Screen Recording and Accessibility 2. Clear the terminal so its text is not read as OCR input 3. Run `uv run clicker "" --act --steps 20 --delay 3` for a slower, bounded run (inferred, check the README) 4. Press Ctrl-C or move the mouse into the top-left corner to stop a live run 5. Read the printed answer, and open the run folder under runs/ if the result is wrong ### Prompt to paste into your agent ```text In the typesafe-computer-use repo, run `uv run clicker "" --act --steps 20 --delay 3` and report the final answer and the stop reason. Verify that the run folder was written under runs/ and that the stop reason is one of done, low confidence, a stall, or the step limit. ``` ### Caveat The README warns that settings and behavior change between 0.x releases and that the loop can still have rough edges. Risk, credentials: The README's login example and the optional CLICKER_EMAIL variable mean the agent may type account details, so keep credentials in your local .env file only. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/awlevin-typesafe-computer-use/automate-a-multi-step-browser-task/ ## How to find out why a run failed and replay it offline with Typesafe Computer Use Repo: [awlevin/typesafe-computer-use](https://github.com/awlevin/typesafe-computer-use) | Difficulty: Intermediate | Time: 30 minutes | Basis: From the README Inspect one screen with clicker-inspect, then replay a saved run folder without a live screen. **Who it is for:** A developer or power user whose agent run stalled or chose the wrong action and who needs to see what it read. **The problem:** A wrong click on a live screen is hard to explain after the fact, and reruns can change the outcome. **The outcome:** You see the annotated screen and payload for one capture, and you can step through a saved run folder to find the failing decision. ### Steps 1. Complete the install steps in the repo 2. Run `uv run clicker-inspect ""` to capture the screen after a 3-2-1 countdown (inferred, check the README) 3. Open the annotated screen and payload that the command produces 4. Locate the run folder under runs/ for the failed run and follow docs/run-folder.md to replay it offline 5. Attach the run folder to a bug report, as the CONTRIBUTING notes ask ### Prompt to paste into your agent ```text In the typesafe-computer-use repo, run `uv run clicker-inspect ""` and then open the saved run folder for the failed run under runs/. Report which action was chosen, what the OCR and accessibility tree showed, and where the decision went wrong. Verify that the annotated screen and payload were produced before drawing any conclusion. ``` ### Caveat The README says to clear the terminal first because its text is on screen and would be read as OCR input, so the capture can be polluted if you skip that step. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/awlevin-typesafe-computer-use/debug-a-failed-run-offline/ ## How to run jev on OSWorld desktop tasks to compare results with Typesafe Computer Use Repo: [awlevin/typesafe-computer-use](https://github.com/awlevin/typesafe-computer-use) | Difficulty: Advanced | Time: 3 hours | Basis: From the README Set up OSWorld beside jev and run one desktop task to see how the agent performs on a benchmark. **Who it is for:** A researcher or engineer who wants to measure a computer-use agent on a standard set of real desktop tasks. **The problem:** Claims about cost and speed for computer-use agents are hard to check without running them on the same tasks. **The outcome:** You get a result for one OSWorld task run by jev, alongside OSWorld's own GPT agent on that task. ### Steps 1. Provision a Linux host with KVM, or follow docs/osworld.md for the one-command Google Cloud machine 2. Run `scripts/osworld setup` to fetch OSWorld and install jev beside it 3. Run `scripts/osworld run-jev chrome/ --ocr rapidocr` to run one task 4. Read the result files the run produces and compare them with the GPT agent run on the same task ### Prompt to paste into your agent ```text In the typesafe-computer-use repo, run `scripts/osworld setup` and then `scripts/osworld run-jev chrome/ --ocr rapidocr` on a Linux host with KVM. Report the task result and any error from the setup. Verify that the run finished on the chrome/ task before comparing it to the GPT agent result. ``` ### Caveat The README says the benchmark needs a Linux host with KVM, so it will not run on a Mac alone. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/awlevin-typesafe-computer-use/benchmark-jev-on-osworld-tasks/