DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to let a coding agent operate a running Mac app with macOS Harness

Give an agent read and click access to a Mac app window, one shell command per step.

Authorization notice. macOS Harness is dual-use. Only use it on software, sites and systems you own or have explicit permission to test.

Who it is forA developer using Claude Code or Codex who hits a task that only an app's own UI can do, such as a settings panel with no API.

The problemCoding agents can run shell commands but cannot see or use the windows of apps on the Mac.

The outcomeThe agent reads a window as numbered text, clicks the element it needs, and reads the window again to confirm the result.

What you need first

  • A Mac running macOS, since the project is macOS only
  • Python 3.12 and uv, to install macOS Harness
  • Accessibility and Screen Recording permissions, which every call needs
  • A coding agent CLI such as Claude Code or Codex to run the commands

Steps

  1. Install or upgrade macOS Harness from https://github.com/browser-use/macos-harness with uv using Python 3.12.
  2. Run macos-harness install --agent claude for Claude Code, or the matching agent name for your CLI. (inferred, check the README)
  3. Run macos-harness doctor and grant any missing macOS permission when asked.
  4. Run macos-harness get_app_state --app "TextEdit" to read the window as numbered text.
  5. Run macos-harness click --app "TextEdit" --element_index 12 on the element you need, then run macos-harness get_app_state --app "TextEdit" again.

Prompt for your agent

Using the macos-harness repo at https://github.com/browser-use/macos-harness, run `macos-harness get_app_state --app <app name>` on the target app and list the numbered elements in its window. Verify that the output contains at least one numbered element and a screenshot path before you click anything.

Caveat. The project is experimental and macOS only, and no call works until Accessibility and Screen Recording are granted.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page