DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to turn PDFs, Word files and emails into clean text elements with Unstructured

Run partition on any supported file and get a list of elements ready for chunking and embedding.

Short answer: Install everything with pip install "unstructured[all-docs]", or just the formats you need with pip install "unstructured[docx,pptx]"; then install system dependencies such as poppler-utils and tesseract-ocr for PDFs and images; call partition(filename=...) from Unstructured.partition.auto so the file type is detected for you. All steps.

  • Top pick
  • Data and analytics
  • Beginner
  • 45 minutes
  • From the README
  • Claude Code
  • Codex
  • Cursor
  • Gemini CLI

Who it is forAn engineer building a RAG index over a folder of mixed file types.

The problemEvery format needs its own parser, and layouts break naive text extraction.

The outcomeOne element list per document, with text you can print, chunk or store.

What you need first

  • Python with pip; `pip install "unstructured[all-docs]"` for every document type
  • System packages as needed: libmagic-dev, poppler-utils, tesseract-ocr and libreoffice
  • Or Docker, to use the prebuilt image instead of installing dependencies

Steps

  1. Install everything with pip install "unstructured[all-docs]", or just the formats you need with pip install "unstructured[docx,pptx]"
  2. Install system dependencies such as poppler-utils and tesseract-ocr for PDFs and images
  3. Call partition(filename=...) from Unstructured.partition.auto so the file type is detected for you
  4. Print the result with print("\n\n".join([str(el) for el in elements])) and pass the elements to your chunker

Prompt for your agent

Using Unstructured-IO/unstructured, install `unstructured[all-docs]`, partition every file in <my documents folder> with partition from unstructured.partition.auto, and report the element count per file and any file that failed.

Caveat. The README notes the hosted Transform API scores higher on tables than the open-source library.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for Unstructured-IO/unstructured and "Turn PDFs, Word files and emails into clean text elements"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer