DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to use UI Mate: what you can do with it

What is UI Mate? UI-Mate is a foundation GUI agent that observes a live desktop screen and acts through keyboard and mouse, with an OpenAI-compatible desktop client and a demonstration-guided mode.

How do I use it? Run pip install openai pillow in the repository environment; then run vllm serve /path/to/UI-Mate-27B --trust-remote-code --served-model-name UI_Mate --port 8000 --tensor-parallel-size 2 --gpu-memory-utilization 0.85 --mm-encoder-tp-mode data --chat-template-content-format openai --limit-mm-per-prompt '{"image":6,"video":0}'; run curl -s http://127.0.0.1:8000/v1/models and confirm the served model name. Full steps below.

Trended earlier

View repo on GitHub

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer

Authorization notice. This tool is dual-use. Only use it on software, sites and systems you own or have explicit permission to test, and follow the relevant licenses and terms of service.

What do you want to do?

3 use cases

  • Test a GUI agent on one screenshot locally

    Serve UI Mate with vLLM and get a predicted next action for a screenshot you provide.

    View prompt
    Using the Tencent/UI-Mate repository, serve UI-Mate-27B with vLLM on port 8000, then run examples/run_agent.py with the screenshot at <screenshot path> and the instruction <instruction>. Report the predicted action and where the screenshot and prediction were saved in outputs/. Verify that the model name returned by /v1/models is UI_Mate before running.

  • Benchmark a GUI agent on OSWorkerBench tasks

    Load the 100 office tasks and 33 plus 45 demonstrations from Hugging Face to measure an agent.

    View prompt
    Use the Tencent/UI-Mate repository and its osworker_bench/ folder to load the OSWorkerBench tasks with load_dataset("tencent/OSWorkerBench") and the demonstrations configuration. Report the number of tasks and demonstration steps you loaded, and confirm the default configuration returns 100 tasks before running any evaluation.

  • Teach an agent a recurring office task by showing it once

    Record one successful run, attach it, and have UI-Mate-democua-27B reuse the procedure on a new task.

    View prompt
    Using the Tencent/UI-Mate repository, attach the demonstration at <demo path> to a run of UI-Mate-democua-27B on the new task <task description>. Report which subtasks the agent completed and whether it re-planned from the live screen anywhere. Verify that the session artifacts show a step-by-step record before you accept the result.

Quick start with UI Mate

  1. Check you have what it needs: A GPU server able to serve the UI-Mate-27B checkpoint with vLLM (the README example uses two-way tensor parallelism); Python with pip, so you can install openai and pillow; A clone of the Tencent/UI-Mate repository and the model weights from Hugging Face.
  2. Open Tencent/UI-Mate on GitHub and read the README and license.
  3. Run pip install openai pillow in the repository environment.
  4. Run vllm serve /path/to/UI-Mate-27B --trust-remote-code --served-model-name UI_Mate --port 8000 --tensor-parallel-size 2 --gpu-memory-utilization 0.85 --mm-encoder-tp-mode data --chat-template-content-format openai --limit-mm-per-prompt '{"image":6,"video":0}'. (inferred, check the README)
  5. Run curl -s http://127.0.0.1:8000/v1/models and confirm the served model name.
  6. Run python examples/run_agent.py --image /path/to/screen.png --instruction "<your instruction>" --base-url http://127.0.0.1:8000/v1. (inferred, check the README)

These steps come from our top use case, Test a GUI agent on one screenshot locally, which also has a copy-paste agent prompt.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for Tencent/UI-Mate

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

UI Mate alternatives · Best AI repos for Work and operations

About this repo

  • Who it is forBuilt for developers and teams who want a computer-use agent to run desktop office workflows, and who can host a large model or call an OpenAI-compatible endpoint.
  • Why it is hot111 stars, launched 73 days ago
  • LanguageJavaScript
  • Sourcegithub.com/Tencent/UI-Mate · Back to Hot repos

Requirements

  • A GPU server able to serve the UI-Mate-27B checkpoint with vLLM (the README example uses two-way tensor parallelism)
  • Python with pip, so you can install openai and pillow
  • A clone of the Tencent/UI-Mate repository and the model weights from Hugging Face
  • A sandboxed or test desktop, since the agent controls real keyboard and mouse input

Add this badge to your README

Maintain UI Mate? Show readers where to find these use cases. Use cases on Deduped

Markdown 

[![Use cases on Deduped](https://dedupedai.com/badge/Tencent/UI-Mate.svg)](https://dedupedai.com/use-cases/tencent-ui-mate/?utm_source=github&utm_medium=badge&utm_campaign=Tencent-UI-Mate)

HTML 

<a href="https://dedupedai.com/use-cases/tencent-ui-mate/?utm_source=github&amp;utm_medium=badge&amp;utm_campaign=Tencent-UI-Mate"><img src="https://dedupedai.com/badge/Tencent/UI-Mate.svg" alt="Use cases on Deduped" height="20"></a>

A static image, no tracking script. About the badge

Questions about UI Mate

What is UI Mate?

UI-Mate is a foundation GUI agent that observes a live desktop screen and acts through keyboard and mouse, with an OpenAI-compatible desktop client and a demonstration-guided mode. It is an open-source project on GitHub: https://github.com/Tencent/UI-Mate.

How do I use UI Mate?

Run pip install openai pillow in the repository environment; then run vllm serve /path/to/UI-Mate-27B --trust-remote-code --served-model-name UI_Mate --port 8000 --tensor-parallel-size 2 --gpu-memory-utilization 0.85 --mm-encoder-tp-mode data --chat-template-content-format openai --limit-mm-per-prompt '{"image":6,"video":0}'; run curl -s http://127.0.0.1:8000/v1/models and confirm the served model name. The use cases on this page give the full steps and a copy-paste agent prompt for each job.

Who is UI Mate for?

Built for developers and teams who want a computer-use agent to run desktop office workflows, and who can host a large model or call an OpenAI-compatible endpoint.

What can I do with UI Mate?

3 use cases so far, for example: Test a GUI agent on one screenshot locally; Benchmark a GUI agent on OSWorkerBench tasks; Teach an agent a recurring office task by showing it once. Each has steps and a copy-paste agent prompt.

What do I need to run UI Mate?

A GPU server able to serve the UI-Mate-27B checkpoint with vLLM (the README example uses two-way tensor parallelism); Python with pip, so you can install openai and pillow; A clone of the Tencent/UI-Mate repository and the model weights from Hugging Face; A sandboxed or test desktop, since the agent controls real keyboard and mouse input.

What are alternatives to UI Mate?

Other hot AI repos in Work and operations: Solidcraft, Browser Use, Next AI Draw.io, Pilot Deck, OpenDots.

Is UI Mate safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.