# ICDAR 2021 SLP Task A: Document Layout Recognition on PubLayNet > Competition overview for Task A of the ICDAR 2021 Scientific Literature Parsing competition: document layout recognition on PubLayNet with 78 teams and 281 submissions. - Authors: Hunter Heidenreich - Published: 2026-04-01 - Paper: Preprint (arXiv) - Paper: Published (ICDAR 2021) - Code: ICDAR2021-SLP (GitHub) - Source: Bevaya Labs, --- ## TL;DR ICDAR 2021 SLP Task A is the competition track for document layout recognition on PubLayNet. With 78 teams and 281 submissions, the top systems pushed $\text{mAP}@[0.50\!:\!0.95]$ on PubLayNet's test set to 0.9733, substantially above previously reported results. Multimodal detection (vision plus PDF-parsed text positions) combined with model ensembles and per-class PDF refinement was the dominant winning approach. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Resource}}$: The primary contribution is a community benchmark event. The paper documents the competition design, evaluation protocol, participation statistics, leaderboard results, and system descriptions from top teams. No new model or dataset is introduced; PubLayNet provides the underlying data. **Secondary:** none. The paper does not make a methodological or evaluative contribution beyond organizing and reporting the competition results. ## What is the motivation? Prior layout analysis benchmarks were small (typically a few hundred pages) because annotation was done manually. PubLayNet automated annotation at scale, enabling a large-scale competition. Task A was designed to drive progress specifically in recognizing layout elements in scientific literature: text blocks, titles, figures, tables, and lists. The competition is part of a two-task event (ICDAR2021-SLP). Task B, which addresses table structure recognition on PubTabNet, is covered in the [TSR notes](../tsr/). ## What is the novelty? The competition itself is the contribution: a standardized evaluation protocol on a fixed test set, an online leaderboard, and a documented snapshot of where the field stood in early 2021. The paper also provides system descriptions from top teams, making it a useful reference for understanding what practical engineering choices drove the largest gains at the time. ## What experiments were performed? ### Data Task A used PubLayNet. The competition test set (11,405 pages) was the held-out split of PubLayNet released as part of the competition; participants had access to the full training set (335,703 pages) and development set (11,245 pages) throughout. | Split | Size | Phase | | :--- | ---: | :--- | | Training | 335,703 | N/A | | Development | 11,245 | N/A | | Mini development | 20 | Format verification | | Test | 11,405 | Evaluation | ### Metric COCO-style mAP averaged over IoU thresholds $[0.50 : 0.95]$ in steps of $0.05$, computed over five region classes: Text, Title, List, Table, Figure. ### Results Top 9 teams on the Evaluation Phase leaderboard: | Team | Text | Title | List | Table | Figure | Overall | | :--- | ---: | ---: | ---: | ---: | ---: | ---: | | Davar-Lab-OCR (Hikvision) | 0.9838 | 0.9607 | 0.9680 | 0.9735 | 0.9804 | **0.9733** | | TAL | 0.9823 | 0.9420 | 0.9700 | 0.9775 | 0.9833 | 0.9710 | | Simo (Shanghai Jiao Tong) | 0.9810 | 0.9536 | 0.9636 | 0.9738 | 0.9796 | 0.9703 | | BIT-VR Lab | 0.9778 | 0.9270 | 0.9645 | 0.9762 | 0.9816 | 0.9654 | | IOD | 0.9774 | 0.9251 | 0.9620 | 0.9773 | 0.9814 | 0.9647 | | 小牛刀 | 0.9797 | 0.9515 | 0.9575 | 0.9635 | 0.9709 | 0.9646 | | JHL | 0.9774 | 0.9245 | 0.9620 | 0.9754 | 0.9814 | 0.9642 | | 刷不了 | 0.9778 | 0.9248 | 0.9634 | 0.9734 | 0.9803 | 0.9639 | | SRK | 0.9767 | 0.9200 | 0.9599 | 0.9737 | 0.9800 | 0.9621 | ## What are the outcomes/conclusions? **Key findings:** - Title is the hardest class across all teams; per-class AP for Title is 3-5 points lower than any other class. The authors note this as an open problem. - The top three teams all exceed 0.97 overall mAP, a substantial improvement over previously reported Mask R-CNN baselines (~0.91). - The dominant pattern across top systems: two-stage Cascade Mask R-CNN or HTC backbone, with post-processing that replaces or refines text/title detections using coordinates extracted directly from the PDF (via PDFMiner or similar). This PDF-native refinement step was decisive for the title category specifically. - Model ensembles (multiple backbones: ResNeXt-101, ResNeSt-101, ResNeSt-200, YOLOv5x, CenterNet2) were standard practice among top teams. - One team (Simo/SJTU) explicitly patched unannotated "text" regions in the training set using heuristic strategies, and credited this for a recall improvement on that class. **Limitations:** - The competition is restricted to scientific literature (PubMed Central). All results are on the PubLayNet domain; generalization to other document types is not evaluated. - Systems using PDF-native text extraction are not directly comparable to image-only approaches. The competition did not distinguish these modes, so the leaderboard conflates vision-only and multimodal systems. - Source code for most top systems was not released as part of the competition. ## Reproducibility ### Models No trained weights were released by the competition organizers. The top teams provided system descriptions but not reproducible code, with one exception: Davar-Lab-OCR (Hikvision) released code and paper references via their lab website and the DAVAR-Lab-OCR GitHub repository. ### Algorithms Top systems shared a common structure: Cascade Mask R-CNN or HTC detection backbone (ResNeXt-101 or ResNeSt-101/200 FPN), trained with augmentation and multi-scale inference, followed by weighted bounding-box fusion (WBF) ensemble. PDF text extraction (PDFMiner or similar) was applied in a post-processing step to refine text-line and title detections by replacing coarse detector outputs with precise PDF-parsed coordinates. ### Data PubLayNet is publicly available under CDLA-Permissive-1.0 (annotations); underlying PMCOA PDFs are non-commercial only. The competition test set is the PubLayNet test split (11,405 pages). See [PubLayNet notes](../layout/) for full dataset details. ### Evaluation COCO $\text{mAP}@[0.50\!:\!0.95]$ via pycocotools. The competition evaluation harness is available at the [ICDAR2021-SLP GitHub repo](https://github.com/IBM/ICDAR2021-SLP) (Apache-2.0). The online leaderboard is no longer active. ### Hardware Not reported by competition organizers. Individual teams used V100-class GPUs for training; no aggregate compute figures are available. ## BibTeX ``` ```bibtex @inproceedings{yepes2021icdar, title={ICDAR 2021 Competition on Scientific Literature Parsing}, author={Jimeno Yepes, Antonio and Zhong, Peter and Burdick, Douglas}, booktitle={Document Analysis and Recognition -- ICDAR 2021}, year={2021}, publisher={Springer}, doi={10.1007/978-3-030-86337-1_40} } ``` ```