GitHub / Unstructured-IO/unstructured / commits
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
| SHA | Message | Author | Date | Stats |
|---|---|---|---|---|
| 104b585d |
fix: fallback character-set detection for file-like objects (#4438)
Signed-off-by: fei <2****5@u****m> |
fei <2****5@u****m>
Committed by: GitHub <n****y@g****m> |
8 days ago | |
| 6d383401 |
fix: linear-time inline element merging in HTML partitioning (ML-1713) (#4440)
Co-authored-by: Claude Opus 4.8 <n****y@a****m> |
Matt Aadland <a****6@u****m>
Committed by: GitHub <n****y@g****m> |
8 days ago | |
| 44f9d747 |
fix(security): bound quadratic array-stream decoding in is_pdf_too_complex (S...
Co-authored-by: Claude Sonnet 5 <n****y@a****m> |
Matt Aadland <a****6@u****m>
Committed by: GitHub <n****y@g****m> |
10 days ago | |
| 8c4592a1 | chore: restore telemetry by default (#4431) |
cragwolfe <c****g@u****o>
Committed by: GitHub <n****y@g****m> |
13 days ago | |
| 114a1d51 |
test: stop GLOBAL_WORKING_DIR tests from disturbing other xdist workers (#4426)
Co-authored-by: Claude Opus 5 (1M context) <n****y@a****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
21 days ago | |
| 4c61d871 |
feat: add lazy chunking entry points (#4423)
Co-authored-by: Claude Opus 5 (1M context) <n****y@a****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
22 days ago | |
| 441b9d68 | perf(html): use indexed parent lookup (#4417) |
Kevin Turcios <1****7@u****m>
Committed by: GitHub <n****y@g****m> |
30 days ago | |
| d309caf8 | Terminology update: Platform -> Pipelines in README (#4404) |
Paul-Cornell <p****l@u****o>
Committed by: GitHub <n****y@g****m> |
about 1 month ago | |
| c38745b3 | feat: partition arbitrary valid JSON and NDJSON files (#4391) |
Simon <s****n@c****o>
Committed by: GitHub <n****y@g****m> |
about 1 month ago | |
| 8447a52b |
Update README.md (#4393)
Co-authored-by: ajaykrishnan23 <k****g@g****m> |
ajaykrish-uns <a****y@u****o>
Committed by: GitHub <n****y@g****m> |
about 1 month ago | |
| cda16b3f |
fix: sanitize v2 HTML output to prevent stored XSS (GHSA-v5mq-3xhg-98m9) (#4394)
Co-authored-by: Claude Opus 4.8 (1M context) <n****y@a****m>, Copilot Autofix powered by AI <6****]@u****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
about 2 months ago | |
| 445c9573 | refactor: centralize URL fetching with host validation and default timeouts (... |
William <2****d@u****m>
Committed by: GitHub <n****y@g****m> |
about 2 months ago | |
| f6eea758 |
fix(ci): stabilize partition benchmark with a rolling-median baseline (#4375)
Co-authored-by: Claude Opus 4.8 (1M context) <n****y@a****m> |
qued <6****d@u****m>
Committed by: GitHub <n****y@g****m> |
2 months ago | |
| 37635f74 |
feat: derive category_depth from heading level in the v2 (ontology) HTML pars...
Co-authored-by: Claude Opus 4.8 (1M context) <n****y@a****m> |
qued <6****d@u****m>
Committed by: GitHub <n****y@g****m> |
2 months ago | |
| 5ead69ad |
feat: extract filled AcroForm field text in PDF partitioning (#4372)
Co-authored-by: Claude Opus 4.8 (1M context) <n****y@a****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
2 months ago | |
| dacae2cc |
feat: add enrichment origins metadata field (#4370)
Co-authored-by: Claude Opus 4.8 (1M context) <n****y@a****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
3 months ago | |
| dedf1441 |
fix: keep extracted text aligned with rotated PDF page images in hi_res (#4367)
Co-authored-by: Claude Opus 4.8 (1M context) <n****y@a****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
3 months ago | |
| 19857c19 |
fix: stop decimating embedded text on dense PDF pages (#4368)
Co-authored-by: Claude Opus 4.8 (1M context) <n****y@a****m>, cubic-dev-ai[bot] <1****]@u****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
3 months ago | |
| 620cde78 |
fix(hi_res): recover text inside PDF figure overlays (#4363)
Co-authored-by: Claude Opus 4.8 (1M context) <n****y@a****m>, ryannikolaidis <1****s@u****m>, qued <q****d@u****m> |
qued <6****d@u****m>
Committed by: GitHub <n****y@g****m> |
3 months ago | |
| 7c8f6752 |
fix: rename `isolate_tables` chunking option to `isolate_table` (#4355)
Co-authored-by: Claude Opus 4.7 (1M context) <n****y@a****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
3 months ago | |
| 346cffff |
feat: add option for table chunking (#4354)
Co-authored-by: Claude Opus 4.7 (1M context) <n****y@a****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
3 months ago | |
| bfd78b26 |
fix: handle text too long for spacy issue (#4353)
Co-authored-by: cubic-dev-ai[bot] <1****]@u****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
3 months ago | |
| 238657f6 | fix: chunking dropping table content (#4352) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
3 months ago | |
| 8daa154a | fix: ndjson file type detection (#4349) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
4 months ago | |
| 199f2555 | feat: Track the table extraction method (#4346) |
Vladimir Kirilenko <v****o@d****i>
Committed by: GitHub <n****y@g****m> |
4 months ago | |
| f51769b1 | feat(cli): add unstructured doctor diagnostics command (#4342) |
Clayton <1****0@u****m>
Committed by: GitHub <n****y@g****m> |
4 months ago | |
| b909cf4f | Reject oversized PDF renders before bitmap allocation (#4345) |
Nick Franck <4****e@u****m>
Committed by: GitHub <n****y@g****m> |
4 months ago | |
| 879e1269 | fix: first table chunk preserve col/row span (#4343) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
4 months ago | |
| ed76bfe6 |
feat: replace PyPI opencv wheels with ffmpeg-free builds in Dockerfiles (#4336)
Co-authored-by: Claude Opus 4.6 (1M context) <n****y@a****m> |
Lawrence Elitzer (LoLo) <l****e@u****o>
Committed by: GitHub <n****y@g****m> |
4 months ago | |
| 3ac44430 |
feat: add option to skip table chunking (#4338)
Co-authored-by: cragwolfe <c****g@u****o> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
4 months ago | |
| dfb1653f |
Enable vertical text detection for rotated images (#4328)
Co-authored-by: ryannikolaidis <1****s@u****m>, vladimir-kivi-ds <v****s@u****m> |
Vladimir Kirilenko <v****o@d****i>
Committed by: GitHub <n****y@g****m> |
4 months ago | |
| d0aa8eb5 |
feat: add GHA workflow to build opencv wheels without ffmpeg (#4335)
Co-authored-by: Claude Opus 4.6 (1M context) <n****y@a****m> |
Lawrence Elitzer (LoLo) <l****e@u****o>
Committed by: GitHub <n****y@g****m> |
4 months ago | |
| 029f4917 |
fix(deps): upgrade vulnerable transitive dependencies [security] (#4334)
Co-authored-by: utic-renovate[bot] <u****]@u****m> |
utic-github-cicd-token-generator[bot] <2****]@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 2437078b | Fix fixtures update CI to regenerate markdown (#4332) |
Vladimir Kirilenko <v****o@d****i>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| d2990950 | feat: add page number support to v1 html partition (#4327) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 615782a4 | fix(chunking): preserve semantic headers in carried table chunks (#4313) |
cragwolfe <c****g@u****o>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 264d569b | feat: render Formula elements as $$ blocks with optional normalization (#4308) |
Clayton <1****0@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 051b3583 |
fix(deps): upgrade vulnerable transitive dependencies [security] (#4318)
Co-authored-by: utic-renovate[bot] <u****]@u****m> |
Lawrence Elitzer (LoLo) <l****e@u****o>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| affb9d60 |
refactor: deduplicate PDF rendering by delegating to unstructured-inference (...
Co-authored-by: codeflash-ai[bot] <1****]@u****m>, Kevin Turcios <t****r@g****m> |
codeflash-ai[bot] <1****]@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 8929336e |
perf: speed up standardize_quotes with str.translate() (#4314)
Co-authored-by: codeflash-ai[bot] <1****]@u****m> |
Kevin Turcios <1****7@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 6ada488f |
fix: pdfminer drops extractable text (#4310)
Co-authored-by: Claude Opus 4.6 (1M context) <n****y@a****m> |
qued <6****d@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| a3172f8e | mem: exclude unused spaCy pipeline components to reduce model memory (#4296) |
Kevin Turcios <1****7@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| b6cf5106 | feat(chunking): repeat table headers on continuation chunks (#4298) |
cragwolfe <c****g@u****o>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 6360ef7f | fix: isolate Table elements in pre-chunks (#4307) |
Clayton <1****0@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 47f4728e | mem: reduce PaddleOCR rec_batch_num from 6 to 1 (#4295) |
Kevin Turcios <1****7@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 7c5855bd | Replace lazyproperty with functools.cached_property (#4282) |
Kevin Turcios <1****7@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 94b3ffd0 | fix(chunking): preserve nested table structure in reconstruction (#4301) |
cragwolfe <c****g@u****o>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| b0e86a41 | fix: Self-contained script for version extraction in release CI (#4304) |
Vladimir Kirilenko <v****o@d****i>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 6447dab9 |
fix(deps): Update security updates [SECURITY] (#4303)
Co-authored-by: utic-renovate[bot] <2****]@u****m> |
utic-renovate[bot] <2****]@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 78dfb309 |
feat: tablechunks can reconstruct table (#4291)
Co-authored-by: Claude Opus 4.6 (1M context) <n****y@a****m>, cragwolfe <c****g@u****o> |
qued <6****d@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 47f42b1a | feat: custom Markdown extensions for partition_md (#4292) |
Clayton <1****0@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 1d66b0c5 | feat: Store routing in ElementMetadata (#4293) |
Vladimir Kirilenko <v****o@d****i>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| cc89c8cf |
fix(deps): Update security vulnerability in pypdf to v6.9.1 [SECURITY] (#4248)
Co-authored-by: utic-renovate[bot] <2****]@u****m> |
utic-renovate[bot] <2****]@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| cb168530 |
feat: make telemetry off by default (#4281)
Co-authored-by: Lawrence Elitzer (LoLo) <l****e@u****o> |
Clayton <1****0@u****m>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| 5585e982 |
chore: disable fail-build on Anchore container scan (#4285)
Co-authored-by: Claude Opus 4.6 <n****y@a****m> |
Lawrence Elitzer (LoLo) <l****e@u****o>
Committed by: GitHub <n****y@g****m> |
5 months ago | |
| f6fcba43 | Add a check for complex pdfs (#4268) |
Matt Aadland <a****6@u****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 4da154bc | feat: audio speech to text partition (#4264) |
Clayton <1****0@u****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 6aeb74f0 | Adds Form Element (#4272) |
Matt Aadland <a****6@u****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| ac14f573 | add check if libmagic fails (#4273) |
Matt Aadland <a****6@u****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| aef3bc4d |
fix: avoid O(N²) re-scanning in _patch_current_chars_with_render_mode (#4266)
Co-authored-by: codeflash-ai[bot] <1****]@u****m>, Alan Bertl <a****n@u****o> |
Kevin Turcios <1****7@u****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 031b0cf8 | Bump dependencies (#4265) |
Emily Voss <g****b@e****v>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| d0f86208 | feat: add create_file_from_elements() to re-create document files from elemen... |
Clayton <1****0@u****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 53023522 | fix: relax lower bound for pdfminer.six (#4262) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 16482f96 | Add Github action for time regressions (#4261) |
Matt Aadland <a****6@u****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| afbda958 | feat: custom fallback for language detection (#4238) |
Clayton <1****0@u****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 4a77a8c9 |
fix: self-install pinned spaCy model at runtime with SHA256 verification (#4258)
Co-authored-by: Claude Opus 4.6 <n****y@a****m>, Yao You <y****o@u****o> |
Lawrence Elitzer (LoLo) <l****e@u****o>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 47b8b5ee | bump version (#4257) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 3db7b4f9 |
Fix: replace nltk with spacy CVE 2025 14009 (#4255)
Co-authored-by: Lawrence Elitzer <l****e@u****o>, Claude Opus 4.6 <n****y@a****m>, ryannikolaidis <1****s@u****m>, badGarnet <b****t@u****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| a8f14ba3 | fix: update depdencies (#4247) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| c6c74628 | fix: set max decompressed size for elements JSON (#4244) |
qued <6****d@u****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| e2d8b7ae | fix: remap parent id after hashing (#4245) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| c1f819c5 | fix: gracefully handle invalide html string during chunking (#4243) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| e1f75a39 | Improve fast partition cold start (#4242) |
Nick Franck <4****e@u****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 8096b5af | fix: remove duplicate characters caused by fake bold rendering in PDFs (#4215) |
BitToby <b****6@g****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 104d29f8 | Automate pypi publishing (#4239) |
Emily Voss <e****0@g****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 78e21ca4 | Use bigger runner to publish images (#4237) |
Emily Voss <e****0@g****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| b536fc44 | Add Python 3.11 and 3.13 support (#4236) |
Emily Voss <e****0@g****m>
Committed by: GitHub <n****y@g****m> |
6 months ago | |
| 95b953de | Fix ARM64 image issues (#4233) |
Emily Voss <e****0@g****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 8745e7c9 | Fix Docker ARM64 image failure, use 8-core github runners (#4232) |
Emily Voss <e****0@g****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 2ae8a972 | Fix ARM64 paddlepaddle image builder bug (#4228) |
Emily Voss <e****0@g****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 69770c65 | Migrate to uv (#4226) |
Emily Voss <e****0@g****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 3e426fd1 | feat: increase PIL's max image pixel value for pdf partition (#4220) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| fef9959a |
chore: bump dependencies for 0.18.34 (#4221)
Co-authored-by: Claude Opus 4.5 <n****y@a****m> |
luke-kucing <l****e@u****o>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 2a8f7c6a |
Fix: make pdf image dpi consistent (#4217)
Co-authored-by: ryannikolaidis <1****s@u****m>, badGarnet <b****t@u****m> |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| b1e4b009 |
Preserve newlines in Table and TableChunk elements during PDF partitioning (#...
Co-authored-by: Claude Opus 4.5 <n****y@a****m> |
Dream <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| e131439d | feat: add group_elements_by_parent_id utility function (#4207) |
MkDev11 <9****1@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 4bbb1fff | feat: put pdfium call behind a threadlock (#4211) |
Yao You <t****u@g****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| d1f1bdf1 |
chorse sep bump to resolve open CVEs (#4205)
Co-authored-by: Claude Opus 4.5 <n****y@a****m>, ryannikolaidis <1****s@u****m>, luke-kucing <l****g@u****m> |
luke-kucing <l****e@u****o>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| d4caedf0 | fix: Preserve Line Breaks in Code Blocks During Chunking (#4196) |
Dream <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 8f32550d |
fix(deps): Update semitechnologies/weaviate Docker tag to v1.35.3 (#4135)
Co-authored-by: utic-renovate[bot] <2****]@u****m> |
utic-renovate[bot] <2****]@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| dbe96e22 | fix(deps): Update opensearchproject/opensearch Docker tag to v2.19.4 (#4134) |
utic-renovate[bot] <2****]@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 7b366c53 | fix(deps): Update docker.elastic.co/elasticsearch/elasticsearch Docker tag to... |
utic-renovate[bot] <2****]@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| f0b0e7c9 | fix: filter coordinates kwargs to prevent TypeError in hi_res PDF processing ... |
MkDev11 <9****1@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 01c3f7c2 | Token-Based Chunking Support (#4203) |
Dream <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| c0323a61 | fix: remove sandbox=True from pypandoc to fix ODT conversion (#4193) |
MkDev11 <9****1@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 95fea7e5 |
fix(deps): switch from pip-compile to uv pip compile (#4202)
Co-authored-by: Claude Sonnet 4.5 (1M context) <n****y@a****m> |
Lawrence Elitzer (LoLo) <l****e@u****o>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 8cb62786 | fix: reduce default dpi to 350 (#4199) |
qued <6****d@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| b70758ae |
Luke/update dockerfile (#4192)
Co-authored-by: Claude Opus 4.5 <n****y@a****m>, qued <6****d@u****m> |
luke-kucing <l****e@u****o>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| a55810de |
enhancement: Speed up method `_DocxPartitioner._style_based_element_type` by ...
Co-authored-by: codeflash-ai[bot] <1****]@u****m>, qued <6****d@u****m> |
Aseem Saxena <a****s@g****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 6abc5dfa |
enhancement: Speed up function `_get_optimal_value_for_bbox` by 2,883% (#4181)
Co-authored-by: codeflash-ai[bot] <1****]@u****m>, qued <6****d@u****m> |
Aseem Saxena <a****s@g****m>
Committed by: GitHub <n****y@g****m> |
7 months ago |