# Docling (v2): Notes > MIT-licensed local document-conversion toolkit parsing PDF, images, MS Office, and HTML into a unified DoclingDocument representation with modular pipelines - Authors: Hunter Heidenreich - Published: 2025-01-01 - Updated: 2026-03-04 - Code: Docling Code - Model: Docling Models - Source: Bevaya Labs, --- ## TL;DR Docling is an MIT-licensed, fully local document-conversion toolkit that parses multiple formats (PDF, images, MS Office, HTML) into a unified **DoclingDocument** representation, then exports lossless JSON or lossy Markdown/HTML and supports chunking for RAG. Architecturally it combines modular parser backends, pipelines, and task-specific models (layout detection, OCR, table structure), with an explicit design goal of faithful extraction where text comes from PDF tokens or OCR rather than being generated. The v2 paper focuses on speed benchmarking across CPU and GPU settings, showing OCR dominates runtime and GPU acceleration changes the relative cost of pipeline stages. * * * ## What kind of paper is this? **Dominant:** $\\Psi\_{\\text{Resource}}$ (releases an open-source toolkit with modular architecture, reusable model components, and permissive licensing suitable for air-gapped and sensitive deployments). **Secondary:** $\\Psi_{\\text{Evaluation}}$ (benchmarks conversion speed and profiles pipeline stages across hardware configurations); $\\Psi_{\\text{Method}}$ (describes modular pipeline architecture and faithful extraction design). * * * ## What is the motivation? - **Fragmented ecosystem:** Document conversion is scattered across formats and often locked behind SaaS or restrictive licensing. The authors aim for a self-contained local library suitable for sensitive or air-gapped environments. - **VLM concerns:** The paper positions Docling against VLM-based converters, arguing that hallucinations and cost/compute requirements (often SaaS-only) are risks when faithful transcription is required. - **Faithful extraction approach:** Their alternative uses modular, task-specific models that recover structure and features, while text is sourced from PDF tokens or OCR to reduce "generated" false content. - **Extensibility:** Need for a system that supports swapping or adding models, custom pipelines, and integration with downstream RAG and processing workflows. * * * ## What is the novelty? ### Unified DoclingDocument representation - **DoclingDocument** is a Pydantic model capturing document elements, hierarchy, layout (bounding boxes), and provenance (page numbers, origin). - API surface supports building and traversing in reading order, exporting to lossless JSON versus lossy Markdown/HTML, and chunking abstraction that produces text chunks plus metadata. ### Modular architecture - Three main concepts: **pipelines**, **parser backends**, and the **DoclingDocument** data model as the centerpiece. - Two standard pipelines: - **StandardPdfPipeline** (PDF/images): parse pages, OCR, layout analysis, table structure, assemble document, then export/chunk. - **SimplePipeline** (markup formats like Office/HTML/AsciiDoc): parse markup, build/assemble, optional enrichment. - Design emphasizes extensibility: swap models, add custom pipelines, integrate new backends. ### PDF parsing backends - Default: `docling-parse` built on low-level `qpdf`. - Alternative backend: `pypdfium`. - Parsing step retrieves programmatic text tokens (strings with coordinates) and renders a bitmap per page for downstream operations. ### Task-specific models - **Layout analysis:** Object detector for page elements, architecture derived from **RT-DETR**, re-trained on **DocLayNet** plus proprietary datasets. Inference via Hugging Face `transformers` and `safetensors`. Post-processing removes overlaps and intersects detections with PDF tokens to form paragraphs, titles, lists, captions, figures, and tables. - **Table structure recognition:** **TableFormer**, a vision-transformer model predicting table row/column structure and header/body roles. Pipeline provides image crop with included text cells; predictions are matched back to PDF cells, avoiding re-transcription and making it language-agnostic. - **OCR:** Integrations with **EasyOCR** and **Tesseract**. Authors note EasyOCR is "fairly slow on CPU" and is the biggest compute expense in their benchmark. - **Assembly/post-processing:** Outputs assembled into DoclingDocument via `docling-core`; post-processing algorithms augment features like reading order and figure-caption matching. * * * ## What experiments were performed? ### Speed benchmark and comparisons - Compared against **Unstructured**, **Marker**, and **MinerU**. Explicitly excludes SaaS or remote services. - **Benchmark dataset:** 89 PDFs, derived largely from DocLayNet and augmented with CCpdf. Totals: 4,008 pages, 56,246 text items, 1,842 tables, 4,676 pictures. - **Hardware configurations:** - AWS EC2 `g6.xlarge` (8 vCores, 32GB RAM, Nvidia L4 24GB). - MacBook Pro M3 Max 64GB. - CPU-only and GPU-enabled runs. - **Methodology controls:** Clean environments, latest versions, "state-of-the-art" options. CPU runs constrained to 8 threads; GPU runs enable CUDA where supported. ### Profiling and ablations - Break down per-stage contribution: PDF parse, OCR, layout, table structure. - Measure runtime impact of disabling OCR and/or table structure recognition. - Analyze stage-specific GPU speedups versus CPU baseline. * * * ## What are the outcomes and limitations? ### Outcomes **Per-page conversion time distribution** (benchmark pages): - **x86 CPU:** 0.6s (5th percentile) / 0.79s (median) / 16.3s (95th percentile). - **M3 Max:** 0.26s / 0.32s / 6.48s. - **L4 GPU:** 57ms / 114ms / 2,081ms. **Ablation results:** - Disabling OCR saves approximately 60% runtime on x86 and M3, approximately 50% on L4. - Disabling table structure saves 16% (x86/M3) and 24% (L4). - Disabling both OCR and table structure saves approximately 75% across configurations. **OCR dominance:** - OCR engaged on 578 pages. - EasyOCR average transcription time: 1.6s (L4), 13s (x86), 5s (M3). **Tool comparison (average seconds per page):** - **CPU-only:** Docling leads (3.1s x86, 1.27s M3), followed by MinerU and Unstructured. Marker is substantially slower. - **GPU-enabled:** MinerU leads (0.21s) versus Docling (0.49s) and Marker (0.86s). Unstructured does not benefit from GPU. **Stage-specific GPU speedups** (versus x86 CPU): - OCR: approximately $8 \\times$ faster. - Layout analysis: approximately $14 \\times$ faster. - Table structure: approximately $4.3 \\times$ faster. ### Limitations and open questions **Faithful extraction trade-offs:** - The authors argue faithful extraction avoids hallucinations, but note this requires maintaining a diverse model set for components like formulas and figures. It is unclear how well this scales to highly specialized domains or degraded document quality. **Model coverage gaps:** - Future work includes adding figure classification, equation recognition, and code recognition models. Current system does not handle these elements as robustly as text and tables. **Benchmark scope:** - Speed evaluation is comprehensive, but there is no accuracy or quality benchmark reported in this paper. The authors mention future plans for an open-source quality evaluation framework (referencing DP-Bench, OmniDocBench), but results are not included here. **Hardware generalization:** - Benchmarks use specific configurations (L4 GPU, M3 Max). It is unclear how performance scales to other GPU generations, ARM architectures, or resource-constrained environments. **Comparison fairness:** - Baseline tools are configured with "state-of-the-art" options, but version choices and configuration decisions may not represent optimal settings for all tools. Authors provide version numbers but not full hyperparameter details for baselines. > **Contrast to MinerU:** MinerU is faster on GPU (0.21s/page versus 0.49s for Docling) but slower on CPU. Both use modular pipelines with task-specific models, but MinerU focuses on optimized inference speed while Docling emphasizes extensibility and faithful extraction design. > **Contrast to Marker:** Marker is substantially slower than Docling on both CPU and GPU. Marker uses Surya OCR by default, which may contribute to slower performance compared to Docling's EasyOCR integration. > **Contrast to olmOCR and Infinity-Parser:** olmOCR and Infinity-Parser are end-to-end VLMs that generate structured output directly from images, whereas Docling uses a pipeline of task-specific models and extracts text from PDF tokens or OCR. Docling explicitly avoids generation to reduce hallucination risk, trading potential accuracy for faithful extraction. * * * ## Model ### System architecture Three main concepts: - **Pipelines:** Coordinate parsing, enrichment, and assembly steps. - **Parser backends:** Handle format-specific extraction (PDF, images, Office, HTML). - **DoclingDocument data model:** Pydantic model capturing elements, hierarchy, layout, and provenance. ### StandardPdfPipeline Processing flow: 1. Parse pages via backend (extract text tokens with coordinates, render bitmap). 2. OCR (optional, triggered for scanned pages or regions without programmatic text). 3. Layout analysis (detect page elements via RT-DETR-based model). 4. Table structure recognition (TableFormer predicts row/column structure). 5. Assemble document (build DoclingDocument with reading order, hierarchy). 6. Export (lossless JSON or lossy Markdown/HTML) and chunking. ### SimplePipeline For markup formats (Office, HTML, AsciiDoc): 1. Parse markup structure. 2. Build and assemble DoclingDocument. 3. Optional enrichment steps. ### Layout analysis model - **Architecture:** Derived from **RT-DETR** (real-time detection transformer). - **Training data:** Re-trained on **DocLayNet** plus proprietary datasets (specifics not disclosed). - **Inference:** Hugging Face `transformers` and `safetensors`. - **Post-processing:** Remove overlaps, intersect detections with PDF tokens to form semantic units (paragraphs, titles, lists, captions, figures, tables). ### Table structure model (TableFormer) - **Architecture:** Vision transformer predicting table row/column structure and header/body roles. - **Input:** Image crop of detected table region plus included text cells from PDF tokens. - **Output:** Structural predictions matched back to PDF cells, avoiding re-transcription. This makes it language-agnostic. - **Inference:** Implemented in PyTorch. - **Training details:** Refined with a custom structure token language (per cited prior work), but this paper does not provide end-to-end training hyperparameters. ### OCR integrations - **EasyOCR:** Default, supports many languages. Authors note it is "fairly slow on CPU" and dominates compute cost in benchmarks. - **Tesseract:** Alternative OCR backend. - **Trigger logic:** OCR is applied to scanned pages or regions without programmatic text. ### PDF parsing backends - **Default:** `docling-parse` built on `qpdf` (low-level PDF library). - **Alternative:** `pypdfium`. - **Output:** Text tokens (strings with bounding box coordinates) and page bitmaps. - **Average parsing time:** 81ms per page (x86), 44ms per page (M3). No GPU support for this stage. * * * ## Data ### Benchmark dataset - **Source:** 89 PDFs, largely from DocLayNet, augmented with CCpdf. - **Statistics:** 4,008 pages, 56,246 text items, 1,842 tables, 4,676 pictures. - **Purpose:** Speed benchmark and profiling; no accuracy evaluation reported in this paper. ### Layout model training data - Re-trained on **DocLayNet** plus proprietary datasets. - DocLayNet is a public dataset for document layout analysis with diverse document types and detailed annotations. - Proprietary data specifics are not disclosed (size, domain distribution, quality control). * * * ## Algorithms / Training This paper is primarily a system and tool report; it does not provide end-to-end training hyperparameters or detailed training procedures. Key details: - **Layout detector:** RT-DETR-based architecture re-trained on DocLayNet and proprietary data. Training hyperparameters (learning rate, optimizer, schedule, batch size) are not reported. - **TableFormer:** Described as refined with a custom structure token language (per cited prior work). Inference is implemented in PyTorch, but training details are omitted. - **OCR models:** Docling integrates existing OCR engines (EasyOCR, Tesseract) rather than training custom OCR models. * * * ## Evaluation ### Benchmark configurations Tool versions and settings (Table 1 in paper): - **Docling:** v2.5.2, EasyOCR (default), TableFormer (fast mode). - **Marker:** 0.3.10, Surya OCR (default). - **MinerU:** 0.9.3, auto OCR, doclayout_yolo, rapid_table. - **Unstructured:** 0.16.5, "hi res with table structure" mode. ### Speed results **Per-page conversion time** (median and percentiles reported in Section 5.4): - See Outcomes section above for detailed numbers. **Average seconds per page** (CPU and GPU): - **CPU-only:** - Docling: 3.1s (x86), 1.27s (M3). - MinerU: higher than Docling on CPU. - Marker: substantially slower than Docling. - Unstructured: between Docling and Marker. - **GPU-enabled:** - MinerU: 0.21s (fastest). - Docling: 0.49s. - Marker: 0.86s. - Unstructured: no GPU benefit. ### Profiling insights - **OCR cost:** OCR engaged on 578 of 4,008 pages. EasyOCR is the dominant compute expense, especially on CPU. - **Stage-specific speedups:** GPU provides large speedups for OCR and layout analysis, moderate speedup for table structure, but no speedup for PDF parsing. - **Ablation impact:** Disabling OCR and table structure dramatically reduces runtime but sacrifices extraction quality for scanned or complex documents. * * * ## Hardware / Production ### Benchmark systems - **AWS EC2 g6.xlarge:** - 8 vCores, 32GB RAM, Nvidia L4 24GB. - Used for x86 CPU and L4 GPU benchmarks. - **MacBook Pro M3 Max:** - 64GB RAM. - Used for ARM CPU benchmarks. ### Experimental controls - **CPU runs:** Constrained to 8 threads via `OMP_NUM_THREADS` and tool-specific configuration. - **GPU runs:** Enable CUDA where supported. Not all tools benefit from GPU (e.g., Unstructured). ### Performance characteristics - **PDF parsing:** 81ms per page (x86), 44ms per page (M3). No GPU support for this stage. - **OCR (EasyOCR):** 1.6s per page (L4), 13s (x86), 5s (M3). Dominates total runtime when engaged. - **Layout analysis:** Approximately $14 \\times$ faster on L4 GPU versus x86 CPU. - **Table structure:** Approximately $4.3 \\times$ faster on L4 GPU versus x86 CPU. ### Deployment considerations - **Local, air-gapped use:** MIT license and fully local execution make Docling suitable for sensitive or offline environments. - **Extensibility:** Modular architecture allows swapping OCR engines, adding custom models, or integrating new pipelines. - **Memory and throughput:** Paper focuses on per-page latency; batch processing, memory footprint, and concurrent throughput are not quantified. * * * _Note: This analysis follows the Bevaya research note guidelines and classification taxonomy. For academic or production use, consult the original paper and verify claims through independent evaluation._