DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to parse a long multi-page scan with cross-page context with Unlimited OCR

Load Unlimited OCR 2 in Transformers and parse all pages of a document in one request.

Short answer: Install the tested versions with pip install torch==2.13.0 torchvision pillow transformers==4.52.1 huggingface-hub==0.36.0 plus the other pinned packages in the README; then load baidu/Unlimited-OCR-2 with AutoModel and AutoTokenizer using trust_remote_code=True and bfloat16 on CUDA; Convert your document to page images in page order. All steps.

remote-code. Loading the model uses trust_remote_code=True, which runs code from the model repository; review it first.

Who it is forA data engineer turning long scanned reports into text where tables and sections cross page breaks.

The problemPage-by-page OCR loses context across pages and breaks tables and paragraphs that span them.

The outcomeText for the whole document from one multi-page call, saved to an output folder.

What you need first

  • An NVIDIA GPU (tested on an A800 80GB) with CUDA
  • Python 3.10 with torch 2.13.0 and transformers 4.52.1, or uv for the SGLang setup
  • PyMuPDF to convert PDFs to page images

Steps

  1. Install the tested versions with pip install torch==2.13.0 torchvision pillow transformers==4.52.1 huggingface-hub==0.36.0 plus the other pinned packages in the README
  2. Load baidu/Unlimited-OCR-2 with AutoModel and AutoTokenizer using trust_remote_code=True and bfloat16 on CUDA
  3. Convert your document to page images in page order
  4. Call model.infer_multi with image_files set to the pages and save_results=True
  5. For a single page, call model.infer, which uses dynamic tiling (gundam mode)

Prompt for your agent

Using baidu/Unlimited-OCR-2 in Transformers as the README shows, convert <document.pdf> to page images, run infer_multi over all pages, and save the text to output/multi. Report token count and any pages that look wrong.

Caveat.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for baidu/Unlimited-OCR and "Parse a long multi-page scan with cross-page context"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer