DEDUPEDWho it is forDeveloper whose task requires poking at a native or visual app that has no API
The problemSome checks, like confirming a desktop app renders correctly, cannot be done from a terminal alone.
The outcomeThe agent operates the visible UI as part of the workflow and reports what it saw.
What you need first
- macOS, Windows or Linux desktop (README badge lists all three)
- At least one coding agent CLI installed and signed in with your own subscription, such as Claude Code, Codex or OpenCode
- A git repository to work in, since each agent runs in its own git worktree
- Optional: iPhone or Android phone for the mobile companion app
- Optional: a remote machine reachable over SSH for SSH worktrees
Steps
- Read the Computer Use docs linked from the README.
- Enable the feature for the agent you want to use.
- Give the agent a narrow task on a specific app.
- Watch its actions and stop it if it goes off track.
Prompt for your agent
Using stablyai/orca's Computer Use feature, have an agent open <app name> and perform <specific check>. Verify by reading back what the agent reports it clicked and saw.
Caveat. Computer Use lets an agent control visible apps on your machine, so run it only on non-sensitive sessions and watch what it clicks.
About orca
A desktop and mobile app to run Claude Code, Codex, OpenCode and other coding agents side by side, each in its own workspace, on your own subscriptions. Part of this week's top 10 repos: ranked #5 on Repo Radar. View stablyai/orca on GitHub.
More with orca
- Turn a GitHub PR or Linear issue into its own agent workspace Browse PRs, issues and boards inside the app and open a fresh worktree for any task so an agent can start on it.
- Get a phone alert when an agent finishes and send follow-ups Leave agents running on your desktop and use the mobile companion app to check progress and reply from anywhere.
- Run agents on a bigger remote machine over SSH Point Orca at a beefy remote box and run agents there with full editing, git and terminals from your laptop.
- Fan one prompt across five agents and merge the best result Send the same task to several coding agents in isolated worktrees, compare the diffs, and keep the winner.
- Comment on an agent's diff lines and send fixes back Leave notes on specific diff lines and ship them to the agent so it fixes exactly what you flagged.
- Point at a UI element and send its HTML and CSS to an agent Use Design Mode in the built-in browser to click any element and drop its code and a screenshot into your prompt.
- Let an agent create worktrees and drive the app via Orca CLI Use the Orca CLI so an agent can spin up worktrees and click through a UI on its own.
- Watch Claude and Codex usage and swap accounts at limits Track rate-limit resets across your agent accounts and switch accounts without logging in again.
Disclaimer
Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page