# DocBank: A Benchmark Dataset for Document Layout Analysis > A 500K-page benchmark dataset for document layout analysis constructed using weak supervision from arXiv LaTeX sources. - Authors: Bevaya AI Team - Published: 2020-06-01 - Updated: 2026-03-15 - Paper: Preprint (arXiv) - Paper: Published (COLING 2020) - Dataset: DocBank Dataset (Hugging Face) - Code: Code (GitHub) - Model: Model Zoo - Source: Bevaya Labs, --- ## TL;DR DocBank introduces a 500K-page benchmark for document layout analysis with fine-grained token-level annotations across 12 semantic categories, constructed via weak supervision from arXiv LaTeX sources. The authors inject semantic-specific colors into LaTeX documents, recompile them, and extract token-level labels by mapping RGB values back to structure types. This enables both sequence labeling and object detection workflows. * * * ## What kind of paper is this? - **Dominant:** $\Psi_{\text{Resource}}$ (benchmark dataset with construction pipeline, splits, statistics, and baselines) - **Secondary:** $\Psi_{\text{Method}}$ (weak supervision construction procedure and object detection conversion) - **Secondary:** $\Psi_{\text{Evaluation}}$ (custom area-based metric and multimodal baseline comparisons) * * * ## What is the motivation? Document layout analysis typically emphasizes visual features while underutilizing textual content, despite text providing strong signals for semantic role classification. Existing labeled datasets are either smaller-scale, image-only, or lack token-level annotations, making it difficult to fairly compare NLP, computer vision, and multimodal approaches. High-quality manual annotation at token-level is expensive; the authors target a scalable, low-cost labeling approach using LaTeX structure. * * * ## What is the novelty? The paper formalizes document layout analysis as follows: given a document $D$ composed of a discrete token set $t = \\{t\_0, t\_1, \dots, t\_n\\}$, where each token $t\_i = (w, (x\_0, y\_0, x\_1, y\_1))$ consists of a word $w$ and its bounding box, and a set of semantic categories $C = \\{c\_0, c\_1, \dots, c\_m\\}$, the goal is to find a function $F : (C, D) \to S$ that produces a prediction set: $$ S = \\{(\\{t\_0^0, \dots, t\_{n\_0}^0\\}, c\_0), \dots, (\\{t\_0^k, \dots, t\_{n\_k}^k\\}, c\_k)\\} $$ The core innovations are: - **Weak supervision from LaTeX semantics:** The authors inject structure-specific font colors into LaTeX source code for semantic units (abstract, author, caption, etc.), recompile the documents, then recover token labels by mapping extracted RGB colors to structure types. - **Token-level annotations at scale:** Each token is represented as `(word, bounding box)`, enabling NLP-style sequence labeling while remaining convertible to object detection annotations. - **Conversion to object detection format:** Same-label tokens are grouped into connected components using BFS with x/y proximity thresholds, then bounding boxes are computed for each component to produce region-level annotations. * * * ## What experiments were performed? ### Dataset Construction - **Scale:** 500K document pages from arXiv papers published 2014-2018 (400K Train, 50K Val, 50K Test), sampled proportionally to preserve the natural year distribution. - **Labels:** 12 semantic structure types: Abstract, Author, Caption, Equation, Figure, Footer, List, Paragraph, Reference, Section, Table, Title. - **Pipeline:** 1. **Semantic coloring:** Inject `\color{fontcolor}{...}` edits into LaTeX source. 2. **PDF Compilation:** Generate visually distinct pages. 3. **Token Extraction:** Use PDFPlumber (built on PDFMiner) to extract text lines and non-text elements (like `#LTFigure#`, `#LTLine#`) with bounding boxes. 4. **Labeling:** Assign labels based on the RGB color of the first character in the token. - **Reading Order:** Tokens are sorted top-to-bottom, left-to-right to support sequence models. ### Baselines & Training The authors frame layout analysis as sequence labeling over a serialized 2D document. - **NLP Baselines:** BERT and RoBERTa (text-only sequence labeling). - **Multimodal Baseline:** LayoutLM (text + 2D layout embeddings, no image embeddings). - **Vision Baseline:** Faster R-CNN with ResNeXt-101 (trained on converted object detection annotations). - **Ensemble:** Combining ResNeXt-101 object detections with LayoutLM token predictions. ### Evaluation Metric Instead of standard BIO tagging, the authors propose an area-based metric to handle the spatial nature of the task: $$ \text{Precision} = \frac{\text{area of GT tokens} \cap \text{detected tokens}}{\text{area of all detected tokens}} $$ $$ \text{Recall} = \frac{\text{area of GT tokens} \cap \text{detected tokens}}{\text{area of all GT tokens}} $$ $$ F\_1 = \frac{2 \times \text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}} $$ * * * ## What are the outcomes/conclusions? **Outcomes** LayoutLM substantially improves over text-only baselines and pure vision baselines. The ensemble achieves the best performance. - **LayoutLM$_{\text{LARGE}}$ (0.9350 F1)** outperforms BERT$_{\text{BASE}}$ (0.8770 F1) and Faster R-CNN (0.9051 F1). - The ensemble of **ResNeXt-101 + LayoutLM$_{\text{LARGE}}$** reaches **0.9488 F1**. **Limitations** - **Domain restriction:** Limited to arXiv papers (LaTeX source required). - **Tokenization artifacts:** Mixed-color tokens are assigned the color of the first character, introducing potential label noise. - **Non-text handling:** Non-text elements are treated as special tokens (e.g., `##LTFigure##`), which may underspecify their semantic content compared to visual features. - **Dependency reliability:** The pipeline depends on successful LaTeX compilation; compilation errors directly degrade annotation quality. * * * ## Mapping to Unified Taxonomy DocBank's token-level annotation allows for fine-grained mapping. | DocBank Class | Visual Primitive | Logical Role | Notes | | :--- | :--- | :--- | :--- | | **Abstract** | `Text` | `Abstract` | The paper abstract section. | | **Author** | `Text` | `Author` | Author names and affiliations. | | **Caption** | `Text` | `Caption` | Text describing a `Figure` or `Table`. | | **Equation** | `Formula` | `DisplayEquation` | Block-level mathematical regions. | | **Figure** | `Image` | `Figure` | Visual content (charts, plots, images). | | **Footer** | `Text` | `PageFooter` | Running footers (page numbers, dates). | | **List** | `Text` | `ListItem` | Tokens part of a list structure. **Dataset Violation:** Label is `List` (container) but applies to content tokens. | | **Paragraph** | `Text` | `Body` | Standard prose. | | **Reference** | `Text` | `BibEntry` | Bibliography items. | | **Section** | `Text` | `SectionHeader` | Section and alignment titles. | | **Table** | `Table` | `Table` | Tabular regions. | | **Title** | `Text` | `Title` | The main paper title. | * * * ## Reproducibility ### Models - **Text-only baselines:** BERT (BASE: 110M params, LARGE: 340M params) and RoBERTa (same sizes), used as sequence labeling models over tokenized document text. - **Multimodal baseline:** LayoutLM (BASE and LARGE), pre-trained on IIT-CDIP Test Collection 1.0. Only text and 2D layout position embeddings were used during fine-tuning; image embeddings were disabled. - **Vision baseline:** Faster R-CNN with ResNeXt-101 backbone, pre-trained on ImageNet. - **Checkpoints:** Fine-tuned weights for all baselines are available in the Model Zoo (linked in artifacts above) under Apache-2.0. ### Data - **Source:** arXiv papers from 2014-2018, covering Physics, Mathematics, Computer Science, and other fields. - **Scale:** 500K pages total (400K Train / 50K Val / 50K Test), randomly sampled to preserve the natural year distribution without balancing. - **Availability:** Annotations and code released publicly on GitHub under Apache-2.0; also mirrored on Hugging Face. The underlying arXiv PDFs are not redistributed - users download them directly from arXiv, where each paper carries its own author-chosen license. Apache-2.0 covers only the annotation layer. ### Algorithms - **Sequence labeling:** AdamW optimizer, initial learning rate $5 \times 10^{-5}$, max sequence block size $N = 512$. - **Object detection:** Faster R-CNN with ResNeXt-101 backbone, pre-trained on ImageNet via Detectron2. - **LayoutLM pre-training:** Pre-trained on IIT-CDIP Test Collection 1.0 using two objectives: Masked Visual-Language Model (MVLM) and Multi-label Document Classification (MDC). Image embeddings were not used during fine-tuning on DocBank; only text and 2D layout position embeddings were active. ### Hardware - **Training:** 8x NVIDIA V100 GPUs, batch size 10 per GPU. Fine-tuning LayoutLM took approximately 5 hours per epoch over 400K pages. ### Evaluation - **Toolkit:** HuggingFace Transformers for BERT/RoBERTa/LayoutLM baselines; Detectron2 for Faster R-CNN. - **Note:** LayoutLM was used without image embeddings; only text and 2D layout position embeddings were used during fine-tuning. * * * ## BibTeX ```bibtex @inproceedings{li-etal-2020-docbank, title = "{D}oc{B}ank: A Benchmark Dataset for Document Layout Analysis", author = "Li, Minghao and Xu, Yiheng and Cui, Lei and Huang, Shaohan and Wei, Furu and Li, Zhoujun and Zhou, Ming", booktitle = "Proceedings of the 28th International Conference on Computational Linguistics", month = dec, year = "2020", address = "Barcelona, Spain (Online)", publisher = "International Committee on Computational Linguistics", url = "https://aclanthology.org/2020.coling-main.82", doi = "10.18653/v1/2020.coling-main.82", pages = "949--960" } ```