# DocParser: Hierarchical Document Structure Parsing from Renderings > DocParser is a two-stage end-to-end system that parses document renderings into complete hierarchical structures: Mask R-CNN detects all entity types, then a rule-based component classifies parent-child and reading-order relations. Scalable weak supervision from LaTeX source via synctex reduces labeled-data requirements substantially. - Authors: Bevaya AI Team - Published: 2019-11-05 - Updated: 2026-04-02 - Code: DocParser Code - Dataset: arXivdocs Dataset - Paper: Preprint (arXiv) - Paper: Published (AAAI 2021) - Source: Bevaya Labs, --- ## TL;DR DocParser is, per the authors, the first system to parse the complete hierarchical structure of document renderings into a tree of entities and relations. Stage 1 applies a modified Mask R-CNN to detect 23 entity categories (text, figures, tables, table cells, equations, etc.). Stage 2 uses rule-based heuristics to classify `parent_of` and `followed_by` relations, recovering both document hierarchy and reading order. A LaTeX-based weak supervision mechanism using synctex generates large-scale noisy labels and reduces reliance on costly human annotation, improving entity detection mAP by 39.1% and relation F1 by 35.8% compared to a baseline trained on clean labels alone. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$: The primary contribution is the DocParser system itself: a two-stage pipeline for end-to-end hierarchical document structure parsing, including the Mask R-CNN adaptation for document entities, the rule-based relation-classification heuristics, and the weak supervision framework. The paper is organized around system design and ablation experiments. **Secondary:** $\Psi\_{\text{Resource}}$: DocParser introduces the arXivdocs dataset, which the authors describe as the first resource designed for evaluating hierarchical document structure parsing. It comes in two variants: arXivdocs-target (362 manually annotated documents) and arXivdocs-weak (127,472 weakly labeled documents generated from arXiv LaTeX source). ## What is the motivation? Document structure is fundamental to downstream NLP tasks such as information extraction, question answering, and document conversion, yet most rendered document formats (notably PDF) store content as flat streams of character positions with no structural information. Prior work addressed narrow subtasks: table detection locates table bounding boxes; table structure parsing decomposes individual tables; OCR systems extract text without hierarchy. The authors argue that no prior system had inferred the complete hierarchical structure of a rendered document, including the nesting of sections, figures, captions, equations, and table cells, together with reading order. A second obstacle is data scarcity. Annotating full document hierarchies is expensive: a typical page contains dozens of entities with complex nesting, and the labeling process requires domain expertise. At the time of this work, existing datasets covered only tables (ICDAR 2013) and not complete document structures. An approach that can learn from abundant unlabeled data was needed to make training tractable. ## What is the novelty? DocParser makes three coordinated contributions. **Two-stage parsing system.** Given a set of document renderings $D\_1, \ldots, D\_n$, the system outputs hierarchical structures $T\_1, \ldots, T\_n$, each consisting of entities $E\_j$ and relations $R\_j$ of type $\Psi \in \\{\text{parent\_of}, \text{followed\_by}, \text{null}\\}$. Stage 1 (entity detection) uses Mask R-CNN with a ResNet-110 backbone and Feature Pyramid Network adapted for document-specific entity shapes. Key modifications include: an aspect ratio limit of 1:8 for region proposals (to handle narrow text lines and columns), output classes expanded to 23 document categories, treating all pixels within ground-truth bounding boxes as foreground during mask training, and a mask loss weighting factor of 0.5 to keep bounding-box and category accuracy primary. The 63.9M-parameter model is pre-trained on MS COCO. Stage 2 (relation classification) applies a set of geometry-based heuristics to convert the flat entity list into a hierarchy: - **Nesting (`parent_of`)**: Four heuristics run in sequence: $h\_1$ builds candidate parent-child pairs from bounding-box overlap ($\theta\_1 = 0.45$, $\theta\_2 = 1.2$); $h\_2$ validates against a predefined document grammar; $h\_3$ prunes sub-children to retain only direct children; $h\_4$ resolves ambiguous parents by IoU, then confidence score, then bounding-box area. - **Ordering (`followed_by`)**: Two heuristics determine reading order. $o\_1$ groups entities into left ($G\_l$), center ($G\_c$), and right ($G\_r$) columns by computing overlap with page halves (threshold 0.7), then orders left before right in multi-column layouts. $o\_2$ sorts within groups top-to-bottom, left-to-right by the top-left corner coordinates of bounding boxes. A structure-based refinement loop (Component 4) iteratively updates bounding boxes and repairs grammar violations after relation classification, running until convergence or a maximum iteration count. **arXivdocs dataset.** The dataset provides two splits. arXivdocs-target contains 362 manually annotated scientific documents (160 train / 79 val / 123 test) with 30 entity categories averaging 86.32 entities per document. arXivdocs-weak contains 127,472 documents labeled automatically via the weak supervision mechanism. Both splits are drawn from arXiv and include only documents with at most 30 pages and at least one table. **Weak supervision from LaTeX source.** For documents whose LaTeX source is available, synctex maps TeX commands to PDF rendering coordinates. DocParser uses this to automatically generate noisy bounding-box annotations: figure environments yield FIGURE and FIGURE\_GRAPHIC entities; itemize environments yield ITEMIZE and ITEM entities; sectioning commands yield SECTION and HEADING entities; math environments yield EQUATION entities; and table environments yield TABLE, TABULAR, TABLE ROW, TABLE CELL, and TABLE COLUMN entities. The resulting annotations are noisy (imprecise boxes, missing entities, incorrect labels) but produced at scale without human effort. The system is pre-trained on these weak labels and then fine-tuned on the small manually-annotated target set. ## What experiments were performed? **Document structure parsing on arXivdocs-target.** Three system variants are compared: DocParser Baseline (trained only on arXivdocs-target clean labels), DocParser WS (pre-trained on arXivdocs-weak only), and DocParser WS+FT (pre-trained on arXivdocs-weak then fine-tuned on arXivdocs-target). Entity detection is measured by mAP at IoU thresholds of 0.5 and 0.65 across 20 entity categories. Relation classification is measured by F1 over the full relation triple $(E\_{\text{subj}}, E\_{\text{obj}}, \Psi)$, with and without structure-based refinement. **Robustness check: table structure parsing on ICDAR 2013.** The three DocParser variants are adapted for the ICDAR 2013 table structure dataset to evaluate whether the weak supervision strategy generalizes beyond the arXivdocs domain. Results are compared to DeepDeSRT (Schreiber et al., 2018), a well-regarded image-based method at the time. **Data efficiency analysis.** Figure 3 in the paper tracks mAP as a function of the number of fine-tuning samples, showing at what point DocParser WS+FT surpasses the baseline trained on the full target training set. **Hardware.** All models are trained on 8 GPUs with 12 GB VRAM each (SGD, lr=0.001, momentum=0.9, batch size 8, 80,000 iterations across three phases). Inference speed is measured on a single Titan Xp GPU. ## What are the outcomes/conclusions? **Entity detection.** DocParser WS+FT achieves an mAP of 69.4 at IoU=0.5, compared to 49.9 for the baseline and 34.6 for the WS-only variant. The 39.1% relative improvement over the baseline confirms that weak pre-training provides a substantially better initialization even for entity categories absent from the weak supervision labels (e.g., AUTHOR, FOOTER, HEADER). Only 20 fine-tuning samples are sufficient for WS+FT to surpass the baseline trained on 160 samples, a roughly 8x reduction in labeling effort. **Relation classification.** With refinement at IoU=0.5, DocParser WS+FT achieves an F1 of 0.615 overall (0.581 for `followed_by`, 0.679 for `parent_of`), compared to 0.453 for the baseline. The 35.8% relative improvement is consistent across both relation types. Refinement adds approximately +0.111 F1 absolute for the WS+FT system. The strict evaluation protocol (requiring exact entity matching at IoU threshold) likely understates qualitative accuracy: qualitative analysis shows that pages with F1 below 0.5 can still have visually accurate document structure reconstructions. **Table structure parsing.** On ICDAR 2013, DocParser WS+FT achieves F1 of 0.9292, surpassing the DeepDeSRT image-based baseline (0.9144 on a different subset) and a PDF-heuristic system (F1 of 0.9221) that has access to raw PDF structure. Results suggest the approach generalizes beyond the arXiv scientific paper domain, though the two baselines use different test subsets so the comparison is approximate. **Inference speed.** Entity detection requires approximately 340 ms per document on a single Titan Xp GPU. Relation classification adds 5.67 ms per document on a single CPU at 2.1 GHz (10.81 ms with refinement). **Limitations.** The system relies on Mask R-CNN, which requires substantial GPU memory and training time. The relation classification component is rule-based and does not learn from data, which limits its ability to handle unusual or ambiguous layouts not covered by the predefined document grammar. The arXivdocs dataset is restricted to scientific papers from arXiv, so generalization to other document types (business documents, forms, newspapers) is not validated. The strict tuple-matching evaluation metric penalizes minor bounding-box mismatches, making the raw F1 scores conservative relative to perceived quality. **Lineage.** DocParser is the direct predecessor of HRDoc (Ma et al., AAAI 2023), which extends the hierarchical parsing formulation to a multi-page learned approach with a new multi-modal encoder-decoder and the STEDS metric. The DS3Lab line of work continues in Detect-Order-Construct (Wang et al., 2024), which adds an explicit reading-order prediction stage and joint evaluation across detection, ordering, and structure reconstruction. ## Reproducibility ### Models Mask R-CNN with a ResNet-110 backbone and Feature Pyramid Network. The model has 63,891,032 parameters and is initialized from MS COCO pretrained weights. Key architectural modifications are documented: maximum RPN aspect ratio 1:8, classifier output expanded to 23 categories, mask loss weight 0.5. Source code is released under MIT License at [https://github.com/DS3Lab/DocParser](https://github.com/DS3Lab/DocParser). Pre-trained model weights (including COCO pre-trained weights) are distributed via Google Drive links provided in the repository README; they should be extracted into `DocParser/docparser/default_models/`. No separate license is stated for the weights; they are bundled with the MIT-licensed repository. ### Algorithms Three-phase training: phase 1 (20,000 iterations, CNN backbone frozen), phase 2 (40,000 iterations, stages 4-5 unfrozen), phase 3 (20,000 iterations, all layers trainable). Optimizer: SGD with learning rate 0.001 and momentum 0.9. Batch size 8 (one image per GPU, 8 GPUs). Early stopping based on validation mAP measured every 2,000 iterations (IoU threshold 0.8 for validation). During training, up to 100 entities are sampled per image; at prediction time, the maximum is 200 entities, with a confidence threshold of 0.7. Fine-tuning from weak supervision weights follows phase-1 settings for 2,000 steps. The relation classification component is entirely rule-based with fixed thresholds ($\theta\_1 = 0.45$, $\theta\_2 = 1.2$ for overlap; 0.7 for column assignment) and does not require training. ### Data arXivdocs-target: 362 manually annotated documents (160 / 79 / 123 train/val/test split) drawn from arXiv scientific papers, each with at most 30 pages and at least one table. Annotations cover 30 entity categories; average 86.32 entities per document. arXivdocs-weak: 127,472 documents with 593,583 pages generated by synctex reverse rendering of LaTeX source. Only 80% of arXivdocs-weak is used in experiments; 20% is held back for future annotation. The arXivdocs-target dataset is distributed via Google Drive links in the repository README and should be extracted into `DocParser/datasets/`. No explicit license is stated for the dataset annotations; the repository carries an MIT license that covers its code, but dataset annotations and underlying arXiv papers carry separate and unstated terms. The underlying arXiv papers carry their individual submission licenses. ### Evaluation Entity detection: mAP at IoU thresholds of 0.5 and 0.65, following Pascal VOC convention. Multiple predictions for the same ground-truth entity: only the highest-IoU match is counted as a true positive. Relation classification: F1 over the complete relation triple $(E\_{\text{subj}}, E\_{\text{obj}}, \Psi)$; the triple is counted as correct only when both entity endpoints match (at the specified IoU threshold) and the relation type is correct. No statistical significance tests or confidence intervals are reported. The ICDAR 2013 robustness check uses a random 50% subset of the competition test set; the Schreiber et al. baseline uses a different (non-public) 50% subset, introducing a comparability caveat. ### Hardware Training: 8 GPUs with 12 GB VRAM each (GPU model not specified for training, but described as equivalent to Titan Xp class). Total training: 80,000 iterations. Inference: Titan Xp GPU (12 GB VRAM), approximately 340 ms per document for entity detection. Relation classification: single CPU at 2.1 GHz, 5.67 ms per document (10.81 ms with refinement). No cloud cost estimates are reported. ## BibTeX ```bibtex @inproceedings{rausch2021docparser, title={DocParser: Hierarchical Document Structure Parsing from Renderings}, author={Rausch, Johannes and Martinez, Octavio and Bissig, Fabian and Zhang, Ce and Feuerriegel, Stefan}, booktitle={Proceedings of the AAAI Conference on Artificial Intelligence}, volume={35}, number={5}, pages={4328--4338}, year={2021}, doi={10.1609/aaai.v35i5.16558} } ```