DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to collect data from web pages using a browser agent with Doma

Have DomA browse a set of pages and gather the information you need into one place.

Authorization notice. Doma is dual-use. Only use it on software, sites and systems you own or have explicit permission to test.

Who it is forA researcher or analyst who needs to pull the same kinds of details from many web pages and does not want to copy them manually.

The problemGathering the same details from many pages by hand takes hours and is easy to get wrong.

The outcomeDomA visits the pages you name and returns the collected details, which you check against the source pages.

What you need first

  • Chrome or Edge browser
  • Your own LLM API key from an OpenAI-compatible provider
  • Node.js 20+ only if you build from source

Steps

  1. Download the latest package from Releases at https://github.com/dom-actions/doma/releases and unzip it.
  2. Load the folder as an unpacked extension using chrome://extensions or edge://extensions with Developer mode on.
  3. Enter your own LLM API key in the extension.
  4. Give DomA a list of page URLs and the fields you want collected.
  5. Spot-check several collected values against the original pages before using the data.

Prompt for your agent

Use the dom-actions/doma repo as a browser agent to visit the pages listed in <URL list file> and collect the fields <field names>. Expected output: a table with one row per page and one column per field. Verify by opening three random pages and comparing their values with the table.

Caveat. The README lists collecting as a task but does not describe an export format, so you may need to copy results out of the extension yourself.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page