privacy. The hosted service fetches the URLs you send; self-host for internal or private pages.
legal. Respect each site's terms when converting its content.
Who it is forA developer feeding articles, docs or PDFs to a model.
The problemRaw HTML wastes tokens and confuses models.
The outcomeMarkdown with a title and source header, YAML front matter, or JSON.
What you need first
- curl or any HTTP client for the hosted r.jina.ai and s.jina.ai endpoints
- Docker to self-host with `docker pull ghcr.io/jina-ai/reader:oss`
- An optional Jina API key for higher rate limits on the hosted service
Steps
- Prepend https://r.jina.ai/ to the page URL and fetch it with curl
- For YAML front matter run
curl -H 'X-Respond-With: frontmatter' 'https://r.jina.ai/https://example.com' - For JSON output run
curl -H "Accept: application/json" https://r.jina.ai/https://en.m.wikipedia.org/wiki/Main_Page - Add x-max-tokens to cap the size or x-retain-images none to drop images
Prompt for your agent
Using jina-ai/reader, fetch each URL in <my list> through r.jina.ai with the X-Respond-With frontmatter header and save the markdown files for my RAG index.
Caveat. Anonymous requests are the most rate limited; an API key raises the quota.
Use this in your AI agent
Ask your agent
Using the Deduped MCP, get the setup steps for jina-ai/reader and "Turn any URL into LLM-ready markdown"
Add Deduped to Claude Code once
claude mcp add --transport http deduped https://dedupedai.com/mcp
Every Monday: the 5 hottest repos and exactly how to use them. Free, one email a week, unsubscribe anytime.
Get them freeShare: X LinkedIn Hacker News Email Copy link
Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer