# AI use cases for macOS Harness (browser-use/macos-harness) Repo: [browser-use/macos-harness](https://github.com/browser-use/macos-harness) | Use cases: 3 | Page: https://dedupedai.com/use-cases/browser-use-macos-harness/ A thin set of shell commands that lets an LLM agent read and control apps running on a Mac, one command per step, in the background. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Let a coding agent operate a running Mac app: https://dedupedai.com/use-cases/browser-use-macos-harness/let-coding-agent-drive-mac-app/ - Fill fields in a desktop app that has no API: https://dedupedai.com/use-cases/browser-use-macos-harness/fill-desktop-app-form-fields/ - Check what an app window shows after each step: https://dedupedai.com/use-cases/browser-use-macos-harness/check-app-state-after-each-action/ ## How to let a coding agent operate a running Mac app with macOS Harness Repo: [browser-use/macos-harness](https://github.com/browser-use/macos-harness) | Difficulty: Intermediate | Time: 30 minutes | Basis: From the README Give an agent read and click access to a Mac app window, one shell command per step. **Who it is for:** A developer using Claude Code or Codex who hits a task that only an app's own UI can do, such as a settings panel with no API. **The problem:** Coding agents can run shell commands but cannot see or use the windows of apps on the Mac. **The outcome:** The agent reads a window as numbered text, clicks the element it needs, and reads the window again to confirm the result. ### Steps 1. Install or upgrade macOS Harness from https://github.com/browser-use/macos-harness with uv using Python 3.12. 2. Run `macos-harness install --agent claude` for Claude Code, or the matching agent name for your CLI. (inferred, check the README) 3. Run `macos-harness doctor` and grant any missing macOS permission when asked. 4. Run `macos-harness get_app_state --app "TextEdit"` to read the window as numbered text. 5. Run `macos-harness click --app "TextEdit" --element_index 12` on the element you need, then run `macos-harness get_app_state --app "TextEdit"` again. ### Prompt to paste into your agent ```text Using the macos-harness repo at https://github.com/browser-use/macos-harness, run `macos-harness get_app_state --app ` on the target app and list the numbered elements in its window. Verify that the output contains at least one numbered element and a screenshot path before you click anything. ``` ### Caveat The project is experimental and macOS only, and no call works until Accessibility and Screen Recording are granted. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/browser-use-macos-harness/let-coding-agent-drive-mac-app/ ## How to fill fields in a desktop app that has no API with macOS Harness Repo: [browser-use/macos-harness](https://github.com/browser-use/macos-harness) | Difficulty: Intermediate | Time: 45 minutes | Basis: From the README Set field values and type text into a Mac app window using numbered elements from a read. **Who it is for:** An operations or admin worker who enters the same values into a desktop app every week and has no API to use. **The problem:** Re-entering the same values by hand in a desktop app takes time and invites typing mistakes. **The outcome:** The agent sets each named field to its value and checks the window afterward, so the worker only reviews the result. ### Steps 1. Install macOS Harness with uv using Python 3.12, then run `macos-harness install --agent claude` or your agent's name. (inferred, check the README) 2. Run `macos-harness doctor` and confirm the permissions are granted. 3. Run `macos-harness get_app_state --app "TextEdit"` and note the element number of each field. 4. Run `macos-harness set_value --app "TextEdit" --element_index 7 --value "report.txt"` for each field. 5. Run `macos-harness get_app_state --app "TextEdit"` again and check that each field shows the new value. ### Prompt to paste into your agent ```text Using the macos-harness repo at https://github.com/browser-use/macos-harness, read the window of with `macos-harness get_app_state --app ` and set each field listed in with `macos-harness set_value`. Verify by reading the window again and reporting every field that does not match its expected value. ``` ### Caveat By default the agent can drive any app except its own host and system password sheets, so set `ask_before_first_use = true` in policy.toml before letting it fill a real form. Risk, credentials: Text the agent types passes through a shell command, so do not give it passwords you cannot rotate. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/browser-use-macos-harness/fill-desktop-app-form-fields/ ## How to check what an app window shows after each step with macOS Harness Repo: [browser-use/macos-harness](https://github.com/browser-use/macos-harness) | Difficulty: Intermediate | Time: 30 minutes | Basis: From the README Read a window, act once, then read again to see only what changed. **Who it is for:** A QA tester or support engineer who needs to confirm how a Mac app responds to a sequence of actions. **The problem:** Checking each step of a Mac app workflow by hand is slow and easy to skip. **The outcome:** A short log of each action and the changed window text after it, so the tester can see what the app did. ### Steps 1. Install macOS Harness with uv using Python 3.12, then run `macos-harness install --agent codex` or your agent's name. (inferred, check the README) 2. Run `macos-harness get_app_state --app "TextEdit"` to record the starting window. 3. Run `macos-harness press_key --app "TextEdit" --key "super+s"` to trigger the action under test. 4. Run `macos-harness get_app_state --app "TextEdit"` again and compare it with the starting read. 5. Run `macos-harness telemetry disable` if you do not want anonymous telemetry on this machine. ### Prompt to paste into your agent ```text Using the macos-harness repo at https://github.com/browser-use/macos-harness, read with `macos-harness get_app_state`, perform the action in , and read the window again. Verify and report only the lines that changed between the two reads, and state whether the expected change appeared. ``` ### Caveat The change report depends on the accessibility tree the app exposes, and Electron or plain web view apps give no URL to check. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/browser-use-macos-harness/check-app-state-after-each-action/