# CascadeTabNet: End-to-End Table Detection and Structure Recognition > CascadeTabNet combines Cascade Mask R-CNN with an HRNet backbone for joint table detection and cell structure recognition, using iterative transfer learning and document-specific image augmentation to achieve competitive results on ICDAR 2013, ICDAR 2019, and TableBank. - Authors: Hunter Heidenreich - Published: 2026-03-24 - Paper: Preprint (arXiv) - Paper: Published (CVPRW 2020) - Code: CascadeTabNet Code and Annotations - Source: Bevaya Labs, --- ## TL;DR CascadeTabNet applies a three-stage Cascade Mask R-CNN with an HRNet backbone to jointly detect tables and recognize their cell structure in document images. The authors use two-stage iterative transfer learning from a merged general dataset down to a smaller annotated set, combined with dilation and smudge image augmentations, to achieve competitive results on ICDAR 2013, ICDAR 2019, and TableBank while training on relatively small amounts of labeled data. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$: The central contribution is a complete training pipeline for an instance segmentation model adapted to table understanding. The paper devotes most of its space to architecture choices, augmentation strategies, ablation tables comparing model variants, and detailed transfer learning procedures. **Secondary:** $\Psi\_{\text{Resource}}$: The authors manually annotate 342 images from ICDAR 2019 with three-class labels (bordered table, borderless table, borderless cell masks) and release these annotations publicly. This is a supporting contribution rather than the headline. ## What is the motivation? Table recognition from document images involves two tightly coupled subproblems: locating table regions and identifying the internal cell structure. Earlier systems addressed these independently, often relying on heuristics (line detection, junction analysis) or sequential two-model pipelines where errors in detection propagate into structure recognition. More recent deep-learning approaches, such as DeepDeSRT and TableNet, attempted joint or sequential CNN-based solutions but still required large training sets or introduced separate post-processing steps. The authors aim to handle both tasks with a single model that learns efficiently from limited labeled data, and that outputs cell-level segmentation masks directly without rule-based post-processing for the structure recognition component. ## What is the novelty? The paper combines three ideas that had not been assembled in this way for document table recognition: **1. Cascade Mask R-CNN with HRNet backbone.** The model (CMRcnnHr, referred to as CascadeTabNet in the paper) is a three-stage Cascade Mask R-CNN using HRNetV2p-W32 as the backbone. HRNet maintains high-resolution feature maps throughout the network rather than progressively downsampling, which helps preserve fine-grained spatial detail needed for cell-boundary segmentation. The cascade stages allow the model to refine bounding box proposals at increasingly strict IoU thresholds, improving localization quality. The architecture is implemented via MMdetection using the default `cascade_mask_rcnn_hrnetv2p_w32_20e` configuration. **2. Two-stage iterative transfer learning.** The training strategy proceeds as: $$\theta\_{\text{general}} \leftarrow \text{fine-tune}(\theta\_{\text{ImageNet+COCO}},\ \mathcal{D}\_{\text{general}})$$ $$\theta\_{\text{specific}} \leftarrow \text{fine-tune}(\theta\_{\text{general}},\ \mathcal{D}\_{\text{specific}})$$ where $\mathcal{D}\_{\text{general}}$ is a merged dataset of 1,934 images (ICDAR 2019 Modern, Marmot, and a GitHub borderless table set) with all tables labeled as one class, and $\mathcal{D}\_{\text{specific}}$ is a smaller 342-image set annotated for three classes (bordered tables, borderless tables, borderless cell masks). No layers are frozen during any stage of fine-tuning. **3. Document-specific image augmentations.** Two augmentation transforms are proposed: - **Dilation transform**: binarizes the image and applies morphological dilation with a $2 \times 2$ kernel to thicken text strokes, making text regions more salient to an object detector trained on natural images. - **Smudge transform**: applies Euclidean, linear, and max distance transforms to spread black pixel regions, producing a blurred, smeary representation. This is adapted from Gilani et al. (2017). Adding both transforms to the training set (tripling its size) raises the baseline Faster R-CNN model's weighted-average F1 on ICDAR 2019 from 0.758 to 0.835. ## What experiments were performed? **Augmentation ablation (Table 1).** A Faster R-CNN ResNeXt-101 baseline is trained on four variants of the general dataset: original only, original + dilation, original + smudge, and all three. Results are reported as F1 at IoU thresholds 0.6, 0.7, 0.8, and 0.9, plus a weighted average (WAvg.) that up-weights higher IoU thresholds. The combined augmentation set achieves WAvg. 0.835 vs. 0.758 for the original-only baseline. **Model comparison (Table 2).** Seven model configurations are evaluated on the ICDAR 2019 Track A Modern test set, all trained on the general dataset with both augmentations: RetinaNet (ResNeXt-101), Faster R-CNN HRNet, Cascade R-CNN (ResNeXt-101 and HRNet), Cascade Mask R-CNN (ResNet-50 with deformable convolutions, ResNeXt-101, and HRNet). CascadeTabNet (CMRcnnHr) achieves WAvg. 0.918, outperforming the next best (CMRcnnX at 0.905). **Table detection benchmarks (Tables 3 and 4).** After fine-tuning on ICDAR 2019 Track A training data, the model achieves 3rd place on the post-competition leaderboard (WAvg. 0.901), with the best reported F1 at IoU 0.9 (0.901). On TableBank, training on only 1,500 images per document type (Word, Latex, or both), the model achieves F1 of 94.33 (combined), 96.60 (Latex), and 94.92 (Word), exceeding the TableBank baseline ResNeXt-101 and ResNeXt-152 models trained on the full dataset. **ICDAR 2013 table detection (Table 5).** Fine-tuning the general model on 40 ICDAR 2013 images (using 198 for testing, a harder evaluation than prior work that reserved most images for training) yields perfect precision, recall, and F1. The authors note their test set is larger and harder than those used by DeepDeSRT and TableNet. **ICDAR 2019 Track B2 structure recognition (Table 6).** Evaluated on 100 test images using cell adjacency relations, CascadeTabNet achieves the highest post-competition WAvg. F1 of 0.232 (vs. 0.206 for NLPR-PAL). The absolute scores are low, reflecting the difficulty of precise cell localization at high IoU thresholds. All experiments were run on Google Colaboratory with a P100 PCIE GPU (16 GB VRAM), Intel Xeon CPU at 2.30 GHz, and 12.72 GB RAM. ## What are the outcomes/conclusions? The results suggest that existing instance segmentation architectures originally designed for natural images can be adapted to document table understanding with relatively modest amounts of domain-specific training data, provided the training strategy (iterative transfer learning) and augmentations are chosen to close the gap between natural and document image statistics. For bordered tables, the pipeline falls back to conventional line detection for cell extraction rather than relying on model predictions, which the authors describe as more efficient. The model is applied only to borderless cell segmentation, where line cues are insufficient. Several limitations are worth noting: - The ICDAR 2013 perfect-score result is difficult to interpret fairly because the test/train split differs from prior comparisons (the authors test on more images while fine-tuning on fewer). - Structure recognition F1 scores remain low in absolute terms (WAvg. 0.232), and the authors explicitly note that "high-end post-processing can improve the results significantly," suggesting the model alone is not sufficient for production-quality structure recognition. - The annotated dataset for structure recognition contains only 342 images, which limits diversity and may explain the model's failures on some document types (Figure 5d). - No statistical significance testing or confidence intervals are reported. - The comparison is framed as "no post-processing" vs. competitors that use post-processing, but the bordered-table branch of the pipeline does use conventional line detection and contour-based algorithms, so the system is not fully end-to-end in the strictest sense. ## Reproducibility ### Models - **Architecture:** Cascade Mask R-CNN with HRNetV2p-W32 backbone, three Bbox stages plus one Mask Head at the final stage. Backbone width W32 indicates 32 channels in the high-resolution convolution stream. All models in the comparison use an FPN neck, as stated in the paper's model comparison section. - **Parameter count:** Not reported in the paper. - **Implementation:** MMdetection toolbox, default configuration `cascade_mask_rcnn_hrnetv2p_w32_20e` (20 epochs by default, labeled "20e" in the config name). - **Pretrained initialization:** ImageNet-pretrained HRNet weights (via COCO pretrained model from MMdetection). The paper refers to "imagenet coco model weights" for the first transfer learning stage. - **Weights release:** The authors release code and annotations on GitHub (https://github.com/DevashishPrasad/CascadeTabNet) under the MIT license. The paper does not state whether trained model weight checkpoints are included or specify their license separately. ### Algorithms - **Transfer learning stage 1:** Fine-tune the COCO-pretrained model on the general dataset (1,934 images, single table class). All layers unfrozen. - **Transfer learning stage 2:** Fine-tune the stage-1 model on the specific dataset (342 images, three classes: bordered table, borderless table, borderless cell). All layers unfrozen. - **Optimizer and schedule:** Not explicitly stated in the paper. The paper uses MMdetection default configurations; the 20e schedule in MMdetection convention uses SGD with momentum and step-based learning rate decay, but these details are not confirmed in the text. - **Batch size:** Not reported in the paper. - **Augmentation:** Dilation transform (binary + $2 \times 2$ morphological dilation) and smudge transform (distance transform-based blurring) applied offline; augmented images added to the training set alongside originals, tripling its size. - **No test-time augmentation or ensembling** is mentioned. ### Data - **General dataset:** 1,934 images, 2,835 table annotations. Merged from ICDAR 2019 cTDaR Modern (cTDaR), Marmot (Chinese and English subsets), and a GitHub borderless table dataset. Ground-truth errors in the Marmot dataset were corrected. - **Specific dataset:** 342 manually annotated images selected from ICDAR 2019 Train set. Three-class annotations: 114 bordered tables, 429 borderless tables, 24,920 borderless cell masks. Released publicly via the GitHub repository. - **TableBank fine-tuning:** 1,500 images randomly sampled per document type from the full TableBank dataset. Annotation quality issues in the Word subset led to exclusion of some images from the test set. - **ICDAR 2013 fine-tuning:** 40 randomly selected images used for fine-tuning; 198 used for testing. - **Availability:** ICDAR 2019, ICDAR 2013, Marmot, and TableBank are publicly available datasets; individual licenses are not stated in the paper and should be verified at their respective sources. The custom 342-image annotation set is available through the GitHub repository under the repository's MIT license. ### Evaluation - **ICDAR 2019 (Track A):** Precision, Recall, F1 at IoU thresholds 0.6, 0.7, 0.8, 0.9. The WAvg. metric is defined by the cTDaR competition (Gao et al. 2019) and weights the four F1 scores more heavily at higher IoU thresholds; the exact weights (0.1, 0.2, 0.3, 0.4) are specified in the competition description, not directly in this paper. - **TableBank:** Standard precision, recall, and F1 computed by summing overlap areas across all documents, following the method of Gilani et al. (2017) as described in the TableBank paper. - **ICDAR 2013:** Cell-level completeness/purity metrics; precision and recall per table, then averaged. The paper follows the same protocol as TableNet (Paliwal et al. 2019). - **ICDAR 2019 (Track B2):** Cell adjacency relation-based evaluation (Gobel et al. 2012); F1 at IoU 0.6, 0.7, 0.8, 0.9 and WAvg. - **No error bars, seeds, or significance tests are reported.** - **Comparison fairness concern:** The ICDAR 2013 evaluation uses a larger test split than prior published results, making direct comparison with DeepDeSRT and TableNet potentially misleading in either direction. ### Hardware - **Training hardware:** Google Colaboratory, P100 PCIE GPU (16 GB VRAM), Intel Xeon CPU at 2.30 GHz, 12.72 GB RAM. - **Training time:** Not reported. - **Inference latency and throughput:** Not reported. - **Inference hardware:** Not separately characterized; all reported experiments used the same Colab environment. - **Deployment:** The pipeline relies on MMdetection and standard image processing libraries; local deployment should be feasible on a single GPU with at least 16 GB VRAM based on the reported training environment. ## BibTeX ```bibtex @inproceedings{prasad2020cascadetabnet, title={CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents}, author={Prasad, Devashish and Gadpal, Ayan and Kapadni, Kshitij and Visave, Manish and Sultanpure, Kavita}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops}, pages={572--573}, year={2020} } ```