DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to teach an agent a recurring office task by showing it once with UI Mate

Record one successful run, attach it, and have UI-Mate-democua-27B reuse the procedure on a new task.

Short answer: Record one successful run of the procedure with the UI Mate desktop application's demonstration capture; then edit or attach the saved demonstration in the desktop application so it is linked to the new task; Serve UI-Mate-democua-27B from its Hugging Face checkpoint using the same vLLM pattern as the general model, with a served model name you choose. All steps.

Authorization notice. UI Mate is dual-use. Only use it on software, sites and systems you own or have explicit permission to test.

Credentials. Demonstration recordings capture screenshots and keystrokes, so they can expose passwords or tokens that appear on screen; redact or avoid recording them.

Who it is forAn operations lead or power user who repeats the same multi-app office procedure with different files or data each time.

The problemSome procedures depend on personal tools, file layouts, and naming rules that are easier to show than to describe in one instruction.

The outcomeThe agent follows the demonstrated procedure where it fits the new task and re-plans from the live screen when layouts or data differ.

What you need first

  • A GPU server able to serve the UI-Mate-27B checkpoint with vLLM (the README example uses two-way tensor parallelism)
  • Python with pip, so you can install openai and pillow
  • A clone of the Tencent/UI-Mate repository and the model weights from Hugging Face
  • A sandboxed or test desktop, since the agent controls real keyboard and mouse input

Steps

  1. Record one successful run of the procedure with the UI Mate desktop application's demonstration capture.
  2. Edit or attach the saved demonstration in the desktop application so it is linked to the new task.
  3. Serve UI-Mate-democua-27B from its Hugging Face checkpoint using the same vLLM pattern as the general model, with a served model name you choose.
  4. Start the run with the new task instruction and the attached demonstration, and use pause or interjection if the agent drifts.
  5. Review the session and demonstration artifacts to confirm each subtask completed.

Prompt for your agent

Using the Tencent/UI-Mate repository, attach the demonstration at <demo path> to a run of UI-Mate-democua-27B on the new task <task description>. Report which subtasks the agent completed and whether it re-planned from the live screen anywhere. Verify that the session artifacts show a step-by-step record before you accept the result.

Caveat. The README says the demonstration is advice rather than a script, so results depend on how closely the new task matches the recorded one, and the desktop application setup steps are not in the README excerpt.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for Tencent/UI-Mate and "Teach an agent a recurring office task by showing it once"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer