# Deep Learning for Table Detection and Structure Recognition: A Survey > A 2022 survey of deep learning approaches to table detection and table structure recognition, covering 19 datasets, three method families (heuristic, ML, deep learning), and comparative experiments across multiple backbone-detector combinations. - Authors: Hunter Heidenreich - Published: 2022-11-15 - Updated: 2026-03-23 - Paper: Preprint (arXiv) - Paper: Published (ACM Computing Surveys 2024) - Code: Survey GitHub Repository - Source: Bevaya Labs, --- ## TL;DR A comprehensive survey of deep learning approaches to table detection (TD) and table structure recognition (TSR) as of late 2022. The authors review 19 datasets, provide a historical taxonomy from heuristic to deep learning methods, and run comparative experiments across multiple backbone-detector combinations on the TNCR dataset. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Systematic}}$: The paper's primary contribution is synthesizing and taxonomizing a large body of work on TD and TSR into a structured landscape. It covers historical progression, datasets, architectures, and open problems without proposing a new model. **Secondary:** $\Psi\_{\text{Evaluation}}$: Beyond literature synthesis, the authors contribute comparative experimental tables. They run a set of backbone-detector combinations (HRNet, ResNeSt, Dynamic R-CNN) on the TNCR dataset and compile cross-paper result comparisons on ICDAR 2013/2017/2019, UNLV, Marmot, and TableBank. ## What is the motivation? Tables appear throughout documents and extracting their content programmatically requires two sequential steps: locating the table region (TD) and then parsing its internal grid structure (TSR). The deep learning literature on both tasks grew rapidly between 2017 and 2022, producing diverse architectures and datasets that were difficult to compare systematically. Existing reviews did not cover both tasks together or were based primarily on classical methods. The authors argue a comprehensive survey that unifies terminology, organizes datasets, and benchmarks representative methods is needed to orient both newcomers and practitioners. ## What is the novelty? The paper does not introduce a new model. Its contributions are organizational: 1. **Historical taxonomy.** Methods are grouped into three eras: (1) heuristic-based (1990s to early 2010s), relying on line detection, spatial features, and rule-based logic; (2) machine learning-based (2000s to 2010s), using SVMs, decision trees, and hidden Markov models on handcrafted features; and (3) deep learning-based (2017 to 2022), covering Faster R-CNN, Mask R-CNN, Cascade Mask R-CNN, DETR variants, and encoder-decoder architectures for TSR. 2. **Dataset inventory.** Table 1 in the paper catalogs 19 datasets with their sizes, task coverage (TD / TSR / classification), annotation source, and scan type. This includes ICDAR 2013, ICDAR 2017 POD, ICDAR 2019, TABLE2LATEX-450K, IIIT-AR-13K, UNLV, Marmot, TableBank, PubTabNet, PubTables-1M, FinTabNet, SciTSR, SynthTabNet, TNCR, and several smaller sets. 3. **Table classification coverage.** The survey explicitly covers table type classification (a five-class problem on the TNCR dataset: full-lined, no-lined, merged, partial-lined, partial-lined-merged), which prior surveys had not addressed. 4. **Comparative benchmark.** Section 7 contains the authors' own experiments running multiple backbone-detector combinations on TNCR with precision, recall, and F1 evaluated at 11 IoU thresholds from 50% to 95%. 5. **Open repository.** The authors maintain a GitHub repository cataloging recent publications, open data, and code. ## What experiments were performed? ### Own Experiments The authors evaluate table detection on the TNCR dataset (9,428 tables from FDA drug label images, MIT license, introduced with a 5-class table type annotation scheme). They test: - **HRNet variants** (HRNetV2p-W18, HRNetV2p-W32) with three detection heads: HTC, Mask R-CNN, Cascade Mask R-CNN. - **ResNeSt variants** (S-50, S-101) with Faster R-CNN. - **Dynamic R-CNN** (ResNet-50 backbone) with Faster R-CNN. Results are reported as precision, recall, and F1 at each IoU threshold from 50% to 95%, plus the averaged 50%:95% score. Selected highlights: | Backbone | Detector | F1@50% | F1@50%:95% | |---|---|---|---| | HRNetV2p-W18 | HTC | 0.933 | 0.840 | | HRNetV2p-W32 | Mask R-CNN | 0.911 | 0.871 | | ResNeSt-101 | Faster R-CNN | 0.934 | 0.748 | | Dynamic R-CNN (ResNet-50) | Faster R-CNN | 0.912 | 0.628 | HRNet-based detectors show higher performance at tighter IoU thresholds (80% to 95%), while ResNeSt achieves similar peak F1 at 50% but degrades faster. The W32 models sometimes overfit on the smaller TNCR training set. ### Literature Compilation Tables 16 to 18 compile results from prior work across multiple datasets. Table detection methods are compared on UNLV, ICDAR 2013, ICDAR 2017, ICDAR 2019, Marmot, and TableBank using F1 at various IoU thresholds. TSR methods are compared on ICDAR 2013 using F1 at IoU 50%. The metrics used across studies are inconsistent: some papers report F1 at a fixed IoU (50%, 60%, or 80%), others report mAP, and IoU thresholds are not always specified. The survey adopts a unified IoU sweep to facilitate comparison where possible, filling in blanks with the original paper's stated threshold when the method did not specify. ## What are the outcomes/conclusions? The survey finds that deep learning has substantially advanced both tasks: - **TD on standard benchmarks is largely saturated, per the survey's results.** Several methods exceed F1 0.99 at IoU 50% on ICDAR 2013. Deformable CNN + Faster R-CNN (Siddiqui et al.) reaches F1 99.6%. - **TSR is more difficult than TD, the survey finds.** Performance drops sharply at tighter IoU thresholds and on complex spanning-cell structures. ICDAR 2019 results show F1 values in the 40% to 55% range at IoU 50% for structure recognition, indicating that parsing internal table structure is harder than locating the table region. - **Dataset diversity matters.** Models trained on scanned scientific articles (ICDAR, PubTabNet) do not transfer cleanly to financial (FinTabNet) or wild-captured images (WTW, CamCap). - **Table classification** (the five-class TNCR task) is relatively underexplored and no clear winner emerges from existing work. The authors conclude that opportunities for improvement remain in TSR and cross-domain generalization, and call for more diverse training sets and unified evaluation protocols. ### Limitations - Coverage stops at 2022; transformer-based TSR methods (Table Transformer, EDD successors) are only briefly mentioned. - The own experiments are restricted to one dataset (TNCR), which is relatively small and domain-specific (FDA drug labels). Results do not transfer to diverse document types. - The metric collection across papers is inconsistent and many cells in the comparison tables are empty, making direct comparisons difficult. - The paper does not discuss statistical significance, variance across runs, or data splits in the own experiments. - There is limited engagement with the more recent methods that emerged in 2021 to 2022 (e.g., PubTables-1M Table Transformer, EDD-family TSR systems). ## Reproducibility ### Models This is a survey; no novel model is proposed. The own experiments use standard implementations from the MMDetection toolbox with HRNet, ResNeSt, and Dynamic R-CNN backbones. All these backbones and detectors have publicly available weights and code. ### Algorithms - Framework: MMDetection (PyTorch) - Detectors: HTC, Mask R-CNN, Cascade Mask R-CNN, Faster R-CNN - Backbones: HRNetV2p-W18, HRNetV2p-W32, ResNeSt-50, ResNeSt-101, ResNet-50 - Training schedules: 1x (12 epochs) and 20e (20 epochs), as reported per-model - No additional training details (learning rate, batch size, augmentation, GPU count) are provided for the own TNCR experiments. ### Data - Own experiments: TNCR dataset, MIT licensed, 6,621 pages / 9,428 table instances, scanned FDA drug label images, 5-class table type classification - Literature comparison: ICDAR 2013 (238 pages), ICDAR 2017 POD (2,417 images), ICDAR 2019 (2,439 images), UNLV (2,889 pages), Marmot (2,000 pages), TableBank (417k tables, research-only data license), PubTabNet (568k tables), FinTabNet (~113k tables), PubTables-1M (~947k tables), SciTSR (15k tables), SynthTabNet (600k synthetic tables) ### Evaluation - Metrics: Precision, Recall, F1 at IoU thresholds 50% to 95% in 5% steps, plus 50%:95% average - The mAP formulation follows COCO convention; AP is computed per-class, then averaged - No error bars, significance tests, or multi-run statistics are reported for the own experiments - Cross-paper comparison tables have many missing cells due to inconsistent metric reporting in original papers ### Hardware No hardware details are provided for the own experiments. The MMDetection framework is standard; all tested backbones (HRNet, ResNeSt) require GPU memory typical of standard detection workflows. ## BibTeX ```bibtex @article{kasem2022deep, title={Deep learning for table detection and structure recognition: A survey}, author={Kasem, Mahmoud and Abdallah, Abdelrahman and Berendeyev, Alexander and Elkady, Ebrahem and Abdalla, Mahmoud and Mahmoud, Mohamed and Hamada, Mohamed and Nurseitov, Daniyar and Taj-Eddin, Islam}, journal={ACM Computing Surveys}, volume={56}, number={12}, articleno={305}, pages={1--41}, year={2024}, publisher={ACM New York, NY}, doi={10.1145/3657281} } ```