DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to talk to an Agent in the browser with live camera frames with Qwen Audio Agent

Use the WebUI to stream microphone audio and bounded camera frames to a visual-capable voice frontend.

Credentials. The frontend needs a cloud API Key, and camera frames are sent to that cloud service, so use a key and camera setup you trust.

Who it is forA user who wants a browser-based voice assistant that can see what the camera shows, such as someone describing an object or a workspace.

The problemA text or audio-only assistant cannot respond to what is in front of the camera, so the user has to describe everything by hand.

The outcomeYou open the browser UI, speak to the Agent, and the Agent answers using live audio plus bounded camera frames.

What you need first

  • Node.js 22.22.2+ or 24.15.0+ and npm 10+
  • A Bailian (DashScope) API Key or another supported voice frontend key, such as an OpenAI or Google API Key
  • A backend Agent you want to use, such as Qwen Code, OpenCode, or Claude Code, installed through the one-click install option
  • Microphone access on your computer

Steps

  1. Install the runtime with npm install -g qwen-audio-agent.
  2. Create your config and add your API Key with qwenaudio config.
  3. Pick a visual-capable Realtime frontend such as Google Gemini Live or Qwen Omni Realtime and set its key in the config.
  4. Start the Gateway with qwenaudio.
  5. Open the browser UI with qwenaudio webui and allow camera and microphone access.

Prompt for your agent

Use the repo QwenAudio/qwen-audio-agent to start the WebUI with a visual-capable Realtime frontend. Report the local WebUI URL and confirm that camera frames are streamed with audio. Verify that the Agent's reply references something visible on camera before reporting success.

Caveat. The README says camera frames are streamed only with a visual-capable Realtime frontend, so choosing a frontend without video input will not give the Agent any camera view.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page