DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to turn any URL into LLM-ready markdown with Reader

Prefix a URL with r.jina.ai to get clean markdown, with front matter or JSON when your pipeline needs structure.

Short answer: Prepend https://r.jina.ai/ to the page URL and fetch it with curl; then For YAML front matter run curl -H 'X-Respond-With: frontmatter' 'https://r.jina.ai/https://example.com'; For JSON output run curl -H "Accept: application/json" https://r.jina.ai/https://en.m.wikipedia.org/wiki/Main_Page. All steps.

  • Top pick
  • Research
  • Beginner
  • 15 minutes
  • From the README
  • Claude Code
  • Codex
  • Cursor
  • Gemini CLI

privacy. The hosted service fetches the URLs you send; self-host for internal or private pages.

legal. Respect each site's terms when converting its content.

Who it is forA developer feeding articles, docs or PDFs to a model.

The problemRaw HTML wastes tokens and confuses models.

The outcomeMarkdown with a title and source header, YAML front matter, or JSON.

What you need first

  • curl or any HTTP client for the hosted r.jina.ai and s.jina.ai endpoints
  • Docker to self-host with `docker pull ghcr.io/jina-ai/reader:oss`
  • An optional Jina API key for higher rate limits on the hosted service

Steps

  1. Prepend https://r.jina.ai/ to the page URL and fetch it with curl
  2. For YAML front matter run curl -H 'X-Respond-With: frontmatter' 'https://r.jina.ai/https://example.com'
  3. For JSON output run curl -H "Accept: application/json" https://r.jina.ai/https://en.m.wikipedia.org/wiki/Main_Page
  4. Add x-max-tokens to cap the size or x-retain-images none to drop images

Prompt for your agent

Using jina-ai/reader, fetch each URL in <my list> through r.jina.ai with the X-Respond-With frontmatter header and save the markdown files for my RAG index.

Caveat. Anonymous requests are the most rate limited; an API key raises the quota.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for jina-ai/reader and "Turn any URL into LLM-ready markdown"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer