Who it is forAn engineer building a RAG index over a folder of mixed file types.
The problemEvery format needs its own parser, and layouts break naive text extraction.
The outcomeOne element list per document, with text you can print, chunk or store.
What you need first
- Python with pip; `pip install "unstructured[all-docs]"` for every document type
- System packages as needed: libmagic-dev, poppler-utils, tesseract-ocr and libreoffice
- Or Docker, to use the prebuilt image instead of installing dependencies
Steps
- Install everything with
pip install "unstructured[all-docs]", or just the formats you need withpip install "unstructured[docx,pptx]" - Install system dependencies such as poppler-utils and tesseract-ocr for PDFs and images
- Call partition(filename=...) from Unstructured.partition.auto so the file type is detected for you
- Print the result with
print("\n\n".join([str(el) for el in elements]))and pass the elements to your chunker
Prompt for your agent
Using Unstructured-IO/unstructured, install `unstructured[all-docs]`, partition every file in <my documents folder> with partition from unstructured.partition.auto, and report the element count per file and any file that failed.
Caveat. The README notes the hosted Transform API scores higher on tables than the open-source library.
Use this in your AI agent
Ask your agent
Using the Deduped MCP, get the setup steps for Unstructured-IO/unstructured and "Turn PDFs, Word files and emails into clean text elements"
Add Deduped to Claude Code once
claude mcp add --transport http deduped https://dedupedai.com/mcp
Every Monday: the 5 hottest repos and exactly how to use them. Free, one email a week, unsubscribe anytime.
Get them freeShare: X LinkedIn Hacker News Email Copy link
Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer