# DocLayout-YOLO: Real-time Layout Analysis via Synthetic Data & Adaptive Perception > A high-speed unimodal model for document layout analysis that achieves competitive accuracy on DocLayNet and D4LA benchmarks by leveraging diverse synthetic pre-training data (DocSynth-300K) and a global-to-local receptive field module (GL-CRM). - Authors: Bevaya AI Team - Published: 2024-10-16 - Updated: 2026-03-12 - Paper: DocLayout-YOLO Paper (arXiv) - Code: DocLayout-YOLO - Model: Weights: DocSynth-300K Pre-training - Model: Weights: DocLayNet Fine-tuned - Model: Weights: D4LA Fine-tuned - Model: Weights: DocStructBench Fine-tuned - Dataset: DocSynth-300K - Source: Bevaya Labs, --- ## TL;DR DocLayout-YOLO is a unimodal document layout detector built on YOLOv10 that combines large-scale synthetic pre-training (DocSynth-300K, 300K pages) with a multi-scale receptive field module (GL-CRM). It reports 79.7% mAP on the DocLayNet validation split and 85.5 FPS on an A100 (measured on DocStructBench), outperforming several transformer-based multimodal baselines in both speed and accuracy on the benchmarks tested. Code, weights, and the synthetic dataset are all publicly available. --- ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$: a new unimodal architecture (YOLOv10 variant) and training recipe optimized for real-time document layout analysis. **Secondary:** $\Psi\_{\text{Resource}}$: introduction of DocSynth-300K, a large-scale synthetic dataset generated via a 2D bin-packing algorithm ("Mesh-candidate BestFit"), and DocStructBench, a diverse in-house evaluation benchmark. --- ## What is the motivation? Document Layout Analysis (DLA) faces a tension between speed and accuracy: 1. **Multimodal methods** (e.g., LayoutLMv3, DiT) achieve high accuracy by fusing visual and textual features but are slow (6-14 FPS on the paper's benchmarks) due to heavy transformer backbones. 2. **Unimodal methods** (e.g., YOLO, DINO) are fast but struggle with complex documents because they lack: - **diverse pre-training data** (existing datasets like PubLayNet are homogeneous and skewed toward academic formats). - **multi-scale perception** mechanisms to handle extreme size variance between document elements (e.g., a tiny page number vs. a full-page table). The authors aim to close this gap by creating a model that matches the speed of unimodal detectors while exceeding the accuracy of heavy multimodal transformers. --- ## What is the novelty? The paper introduces two complementary innovations: ### A. Mesh-candidate BestFit & DocSynth-300K ($\Psi\_{\text{Resource}}$) To address data scarcity, the authors frame document generation as a **2D bin-packing problem** rather than a generative diffusion task. - **Algorithm:** "Mesh-candidate BestFit" iteratively fills a page grid with elements (Text, Tables, Figures) sampled from a diverse pool sourced from the M6Doc test set (~2,800 pages, 74 categories). It maximizes fill rate (density) while keeping layouts visually aligned. The authors note that M6Doc itself is not open source due to copyright restrictions, which creates a license ambiguity for DocSynth-300K (see Reproducibility). - **Outcome:** DocSynth-300K, a 300K-page synthetic dataset with enforced layout and element diversity, intended to prevent the model from overfitting to specific academic formats. ### B. Global-to-Local Controllable Receptive Module (GL-CRM) ($\Psi\_{\text{Method}}$) To handle scale variation, the authors modify the YOLOv10 backbone with **GL-CRM**, a hierarchical module that adapts the receptive field across three levels. Features are extracted at varying dilation rates: $$F\_i = \text{GELU}(\text{BN}(\text{Conv}(X, w, d\_i)))$$ These are concatenated across all dilation rates: $$\hat{F} = \text{Concat}([F\_1, F\_2, \ldots, F\_n])$$ A gating mask $M$ controls feature selection: $$M = \sigma(\text{GELU}(\text{BN}(\text{Conv}\_{gate}(\hat{F}))))$$ The masked features are fused back into the residual stream: $$X\_{CRM} = X + \text{GELU}(\text{BN}(\text{Conv}\_{out}(M \otimes \hat{F})))$$ The three levels of GL-CRM are: - **Global level:** Large kernels ($k=5$) with dilations ($d=1,2,3$) to capture texture details and local patterns at whole-page scale. - **Block level:** Medium kernels ($k=3$) with dilations ($d=1,2,3$) to handle medium-scale document elements. - **Local level:** Standard bottleneck (no dilation) to capture fine-grained semantic details. --- ## What experiments were performed? ### Training Setup - **Pre-training:** YOLOv10-M initialized on DocSynth-300K (30 epochs, LR=0.02, batch=128, image size 1600). - **Fine-tuning:** Separately fine-tuned on DocLayNet (LR=0.02, patience=100, image size 1120), D4LA (LR=0.04, patience=100, image size 1600), and DocStructBench (LR=0.04, patience=100, image size 1280). - **Hardware:** 8 x A100 GPUs. ### Baselines - **Unimodal:** YOLOv10, DINO-4scale (ResNet-50 backbone, ImageNet1K pretrained). - **Multimodal:** LayoutLMv3, DiT-Cascade (Base/Large), VGT (Da et al., 2023). ### Evaluation Metrics Primary metric is mAP (COCO-style); secondary is AP50. Speed is measured as FPS on a single A100 GPU. ### Key Datasets - **D4LA:** A 27-class dataset of 11,092 images (8,868 train / 2,224 test) manually annotated from RVL-CDIP across 12 document types, requiring fine-grained class distinction (e.g., `LetterSign`, `Date`, `RegionKV`). - **DocLayNet:** A standard diverse layout benchmark containing 80,863 pages from 7 document types, manually annotated with 11 categories. - **DocStructBench:** An in-house benchmark (not publicly released) covering Academic, Textbook, Market Analysis, and Financial document types; 7,310 train / 2,645 test images across 10 categories (the appendix reports different totals of 9,082/2,232, an internal inconsistency in the paper; the main-text numbers are what the reported results were trained and evaluated on). --- ## What are the outcomes/conclusions? ### Quantitative Performance **DocLayNet:** | Model | FPS | mAP | |---|---|---| | DiT-Cascade-L | 6.0* | 72.6% | | DocLayout-YOLO | 85.5* | 79.7% | \* FPS figures are from the DocStructBench comparison (Table 3 of the paper); the DocLayNet evaluation (Table 2) does not report FPS. DocLayout-YOLO achieves 79.7% mAP on DocLayNet (evaluated on the validation split), 14.3x faster than DiT-Cascade-L based on the DocStructBench speed comparison, with higher accuracy. **D4LA:** DocLayout-YOLO reports 70.3% mAP on D4LA, outperforming multimodal baselines on this benchmark as well. **DocStructBench:** Overall 78.8% mAP across the benchmark's four document subsets. On the Financial subset specifically, the model reaches 90.1% mAP. Note that DocStructBench is an in-house evaluation set and is not publicly available for independent verification. ### Ablation Studies Table 1 in the paper isolates each component against a YOLOv10-M baseline (no document-specific pre-training): - **GL-CRM alone:** +1.2% on D4LA, +1.0% on DocLayNet. - **DocSynth-300K pre-training alone:** +1.2% on D4LA, +2.6% on DocLayNet. - **Full DocLayout-YOLO (GL-CRM + DocSynth):** +1.7% on D4LA, +3.0% on DocLayNet, and up to +3.5% on the Textbook subset of DocStructBench. ### Limitations - The bin-packing synthesis is heuristic-based and may not capture the semantic coherence or organic noise of real documents (e.g., text content does not flow logically across elements). - DocStructBench is not publicly available, so the generalization results on that benchmark cannot be independently verified. - The model underperforms DiT-Cascade-L on the Market Analysis subset of DocStructBench (69.4% vs 70.8% mAP), suggesting that DocSynth-300K pre-training may not fully cover the most complex layout distributions. The paper does not discuss this gap explicitly. - The model performs layout detection only; a separate OCR step is still required for text extraction. --- ## Reproducibility ### Models - Backbone: YOLOv10-M with GL-CRM modifications inserted into the backbone feature extraction stages. - Pre-trained and fine-tuned weights all released on HuggingFace under Apache-2.0: pre-training checkpoint, DocLayNet-finetuned, D4LA-finetuned, and DocStructBench-finetuned variants (see frontmatter artifacts). - Inference runs within the `ultralytics` Python framework with standard YOLOv10 inference APIs. ### Algorithms - **Pre-training:** LR=0.02, batch=128, 30 epochs, image size 1600 on DocSynth-300K. - **Fine-tuning (DocLayNet):** LR=0.02, patience=100 epochs, image size 1120. - **Fine-tuning (D4LA):** LR=0.04, patience=100 epochs, image size 1600. - **Fine-tuning (DocStructBench):** LR=0.04, patience=100 epochs, image size 1280. - The paper does not explicitly report the optimizer type, warmup schedule, gradient clipping, or mixed-precision settings for pre-training. ### Data - **DocSynth-300K:** 300K synthetic pages generated by the Mesh-candidate BestFit algorithm from a pool sourced from the M6Doc test set specifically (~2,800 pages, 74 element categories). The DocLayout-YOLO paper itself states that M6Doc "is not open source due to copyright restrictions." DocSynth-300K is therefore built from crops of copyright-encumbered source images; synthetic rearrangement of those crops does not dissolve the underlying copyright. The HuggingFace dataset card lists Apache-2.0 in its metadata, but that claim is legally dubious given the M6Doc provenance. A public inquiry on the HuggingFace discussion page (opened March 2025) has received no response from the authors. Treat the license as **unknown**. - **DocLayNet:** Public benchmark (80,863 pages, 11 categories, 7 document types). The validation split (6,480 images) is used for evaluation, not the test split. - **D4LA:** Public dataset; 11,092 images from RVL-CDIP, 27 categories, 12 document types; 8,868 train / 2,224 test. The ModelScope page lists Apache-2.0, but the underlying images come from RVL-CDIP (a subset of the IIT-CDIP tobacco litigation archive), which carries no Apache-2.0 grant. The VGT paper that introduced D4LA makes no license claim beyond "will be made publicly available." Treat the image license as **unknown**; the annotation schema may be separately permissible. - **DocStructBench:** In-house; NOT publicly available. 7,310 train / 2,645 test images (main-text Section 5.1), 10 categories, covering Academic, Textbook, Market Analysis, and Financial subsets. The appendix reports different totals (9,082/2,232), an internal inconsistency in the paper. ### Evaluation - Primary metric: mAP (COCO-style AP at IoU 0.50:0.95); secondary: AP50. - Speed: FPS measured on a single A100 GPU (batch size not explicitly stated; single-image inference implied). - DocLayNet results are on the **validation split**, not the test split, as stated in Section 5.1 of the paper. This affects direct comparability with papers that report test-split numbers. - There is an internal inconsistency in the paper between Table 1 and Table 2 for the YOLO-v10 DocLayNet baseline: Table 1 (ablations) reports 76.7 mAP, while Table 2 (method comparison) reports 76.2 mAP. The paper gives no explanation. Ablation deltas in the note use Table 1's 76.7 baseline. - Baselines span both unimodal (YOLOv10, DINO-4scale) and multimodal (LayoutLMv3, DiT-Cascade B/L, VGT) models. - No error bars, seed sensitivity, or multi-run statistics are reported; all results appear to be single runs. - DocStructBench results cannot be independently reproduced since the benchmark is not public. ### Hardware - **Training:** 8 x A100 GPUs. - **Inference:** 85.5 FPS on a single A100 GPU (measured on DocStructBench; per-benchmark FPS is not reported separately). - Total GPU-hours and cloud compute cost are not reported. - No guidance is provided on CPU-only or consumer GPU inference feasibility. --- ## BibTeX ```bibtex @misc{zhao2024doclayoutyolo, title={DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perception}, author={Zhiyuan Zhao and Hengrui Kang and Bin Wang and Conghui He}, year={2024}, eprint={2410.12628}, archivePrefix={arXiv}, primaryClass={cs.CV} } ```