# DocHieNet: A Large and Diverse Dataset for Document Hierarchy Parsing
> Research note on DocHieNet, a 1,673-document bilingual dataset for multi-page document hierarchy parsing across legal, financial, educational, and scientific domains.
- Authors: Hunter Heidenreich
- Published: 2026-03-15
- Paper: Published (EMNLP 2024)
- Dataset: DocHieNet Dataset
- Project: DocHieNet Project Page
- Source: Bevaya Labs,
---
## TL;DR
DocHieNet is a manually annotated, multi-page, bilingual (English + Chinese) dataset of 1,673 documents (15,610 pages) spanning legal, financial, educational, and scientific domains for document hierarchy parsing. It annotates 187K+ layout elements across 19 categories with hierarchical and sequential relationships, including 37.4% cross-page relations. The authors also propose DHFormer, a transformer-based baseline that uses sparse text-layout encoding and a global layout element decoder, reaching 77.82 F1 on DocHieNet while achieving 93-99 F1 on prior datasets.
## What kind of paper is this?
**Dominant:** $\Psi\_{\text{Resource}}$. The headline contribution is the DocHieNet dataset: a multi-domain, bilingual, multi-page dataset for document hierarchy parsing with manually verified annotations, filling a gap left by existing single-domain or single-page resources. The paper devotes its primary sections to data collection, annotation design, dataset statistics, and comparisons with existing resources.
**Secondary:** $\Psi\_{\text{Method}}$. DHFormer is a non-trivial methodological contribution: a sparse text-layout encoder with chunked attention plus a global layout element decoder for multi-page hierarchy prediction, evaluated across five datasets with ablations.
## What is the motivation?
Existing document hierarchy parsing datasets are limited in scope:
- **arXivdocs** is single-page and English-only; **E-Periodica** is single-page (though multilingual: EN, DE, FR, IT). Neither captures cross-page relationships.
- **HRDoc** covers only scientific papers with automatically generated (not manually verified) annotations at the text-line level
- No existing dataset spans multiple document domains (legal, financial, educational, technical)
The authors argue that layout-block-level annotation is more practical than text-line-level because (a) it leverages existing layout analysis models, (b) text-line evaluation inflates scores since trivial "connect" relations dominate, and (c) layout-block evaluation better reflects actual hierarchy prediction quality.
## What is the novelty?
### DocHieNet Dataset
- **1,673 documents** (1,110 English, 563 Chinese) across **15,610 pages** with **187K+ layout elements**
- **19 layout element categories:** title, sub-title, section-title, text, formula, TOC-title, TOC, figure, figure-title, figure-caption, table, table-title, table-caption, header, footer, page-number, footnote, endnote, sidebar
- **Two relationship types:** hierarchical (parent-child) and sequential (reading order)
- **Multi-domain:** Legal, financial, educational, technical, scientific, government publications
- **Cross-page relations:** 37.4% macro-average ratio with 5.4-page average span (vs. HRDoc's 24.9% ratio and 2.4-page span)
- **Manual annotation:** 12 experienced annotators with 3 rounds of specialist quality checks
- **Split:** 1,512 train / 161 test documents. 835 training documents are truncated (partially annotated) to reduce pattern redundancy; test documents are fully annotated.
### DHFormer
A transformer-based framework for document hierarchy parsing with three components:
1. **Sparse Text-Layout Encoder:** Uses a pretrained layout-aware language model (GeoLayoutLM). Documents are split into $K$ chunks where each chunk contains the maximum number of layout elements whose total tokens do not exceed a limit $l = 512$. Attention is dense within chunks but factorized across chunks, reducing complexity from $O(N^2)$ to $O(l \cdot N)$.
2. **Page and Inner-Layout Embeddings:** Sinusoidal page embeddings encode absolute page number; 1D inner-layout position embeddings encode relative token position within each layout element.
3. **Global Layout Element Decoder:** A 2-layer transformer decoder with Shifted Sparse Attention (window size 48) that pools each layout element's representation via its first token. A learnable root embedding is prepended. Relation prediction uses a bilinear layer with sigmoid activation trained with cross-entropy loss.
## What experiments were performed?
The authors evaluate DHFormer on five datasets, all mapped to DocHieNet's annotation format for comparability:
- **DocHieNet** (1,673 docs, multi-domain, bilingual)
- **arXivdocs** (362 single-page scientific documents)
- **HRDoc-Simple** (1,000 docs) and **HRDoc-Hard** (1,500 docs)
- **E-Periodica** (542 single-page magazine pages)
**Metrics:** F1 and TEDS (Tree Edit Distance Score).
**Baselines:** DocParser, DSPS (HRDoc baseline), DOC (Comp-HRDoc baseline), DSG (graph-based), GPT-4 (in-context learning), Llama2-7b (LoRA fine-tuned).
**Ablations:** (1) encoder choice (GeoLayoutLM vs. LayoutLMv3 vs. BROS vs. XLM-RoBERTa), (2) chunking strategy and window size, (3) page and inner-layout embeddings, (4) text-line vs. layout-block annotation paradigm, (5) LLM comparison across document lengths.
## What are the outcomes/conclusions?
### Main Results (all datasets in DocHieNet format)
| Dataset | DHFormer F1 | DHFormer TEDS | Next Best F1 | Next Best TEDS |
| :--- | :---: | :---: | :---: | :---: |
| arXivdocs | 98.45 | 95.04 | 81.17 (DSG) | 72.47 (DSG) |
| HRDoc-Simple | 99.34 | 98.69 | 84.78 (DSG) | 83.24 (DSG) |
| HRDoc-Hard | 93.40 | 89.14 | 74.04 (DSG) | 64.33 (DSG) |
| E-Periodica | 92.53 | 84.85 | 67.17 (DSG) | 60.14 (DSG) |
| DocHieNet | 77.82 | 57.64 | 53.51 (DSG) | 33.90 (DSG) |
DocHieNet is considerably more challenging than prior datasets: even DHFormer achieves only 77.82 F1, compared to 93+ on the others.
### Key Findings
- **Encoder matters:** GeoLayoutLM (layout-aware) outperforms XLM-RoBERTa (text-only) by 8.7 F1 points (77.82 vs. 69.13).
- **Both embedding types help:** Page embeddings add ~2 F1 and inner-layout position embeddings add ~1.5 F1; combined they add ~4 F1 (73.66 to 77.82).
- **Chunking strategy:** Layout-element-aligned chunking at 512 tokens (77.82 F1) outperforms page-level chunking (75.66) and stride-based chunking (73.98).
- **LLM baselines lag behind:** DHFormer (~80 F1) outperforms fine-tuned Llama2-7b (~55-65 F1) and GPT-4 ICL (~25-50 F1), with LLM performance degrading sharply as document length increases (approximate values read from Figure 5; no table provided).
- **Annotation paradigm:** Text-line-level evaluation (as in HRDoc) inflates scores because trivial "connect" relations dominate. Layout-block evaluation is more informative.
### Limitations
- The dataset, while multi-domain, may not cover all document variations in the wild.
- 835 of 1,673 training documents are truncated (partially annotated), which may affect model training for long-document scenarios.
- The dataset requires layout element detection as a prerequisite; end-to-end evaluation with a layout detector (CenterNet) drops F1 from 99.34 to 94.32 on HRDoc-Simple.
- No arXiv preprint available; the paper is only accessible through ACL Anthology.
## Reproducibility
### Models
- **DHFormer:** Sparse text-layout encoder (GeoLayoutLM default) + 2-layer transformer decoder with Shifted Sparse Attention (window size 48)
- **Text encoder variants evaluated:** GeoLayoutLM, LayoutLMv3, BROS, XLM-RoBERTa
- The [GitHub repo](https://github.com/AlibabaResearch/AdvancedLiterateMachinery/tree/main/DocumentUnderstanding/DocHieNet) contains only a README and LICENSE file; no runnable training or inference code is available. The README states data and model "will be made publicly available in the near future." The dataset is hosted on [ModelScope](https://modelscope.cn/datasets/iic/DocHieNet).
- No pretrained model weights or evaluation scripts released.
### Algorithms
- **Optimizer:** AdamW, base LR 4e-5 decaying to 1e-6
- **Epochs:** 100
- **Max tokens per chunk:** 512
- **Max chunks:** 32 (training), 128 (testing)
- **Decoder:** 2-layer transformer, window size 48 for Shifted Sparse Attention
- **Relation prediction:** Bilinear layer with sigmoid, cross-entropy loss
- **Batch size:** Not reported
- **Framework:** Not explicitly stated (likely PyTorch given GeoLayoutLM dependency)
### Data
- **DocHieNet:** 1,512 train / 161 test documents. Available on [ModelScope](https://modelscope.cn/datasets/iic/DocHieNet) under Apache-2.0.
- **Languages:** English (1,110 docs) + Chinese (563 docs)
- **Annotation:** Manual by 12 annotators with 3 rounds of specialist QC
- **835 training documents are truncated** (partially annotated); all 161 test documents are fully annotated
- Comparison datasets (arXivdocs, HRDoc, E-Periodica) available separately with their own licenses
### Evaluation
- **Metrics:** F1 (relation-level) and TEDS (Tree Edit Distance Score: 1 minus the normalized tree edit distance between predicted and ground-truth document hierarchies; document-level)
- **All datasets mapped to DocHieNet annotation format** for cross-dataset comparison
- Cross-format evaluation also performed (train in one format, test in another)
- No error bars or multi-run statistics reported
- LLM comparison included (GPT-4 ICL, Llama2-7b LoRA)
### Hardware
- **DHFormer training:** 2 NVIDIA Tesla V100 GPUs
- **Llama2-7b fine-tuning:** 2 NVIDIA A100 GPUs, LoRA (rank 8, alpha 32, dropout 0.05)
- No inference latency or throughput reported
## BibTeX
```bibtex
@inproceedings{xing2024dochiennet,
title={DocHieNet: A Large and Diverse Dataset for Document Hierarchy Parsing},
author={Xing, Hangdi and Cheng, Changxu and Gao, Feiyu and Shao, Zirui and Yu, Zhi and Bu, Jiajun and Zheng, Qi and Yao, Cong},
booktitle={Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
pages={1129--1142},
year={2024},
doi={10.18653/v1/2024.emnlp-main.65}
}
```