# RobusTabNet: CornerNet Proposals and Spatial CNN for Robust Table Extraction > RobusTabNet proposes CornerNet-based region proposals for Faster R-CNN table detection and a spatial CNN split-and-merge architecture for table structure recognition, reporting competitive results on SciTSR, PubTabNet, and cTDaR across both tasks. - Authors: Hunter Heidenreich - Published: 2026-03-24 - Paper: Preprint (arXiv) - Paper: Published (Pattern Recognition 2023) - Source: Bevaya Labs, --- ## TL;DR RobusTabNet is a two-stage table extraction system that replaces the standard region proposal network (RPN) in Faster R-CNN with CornerNet for higher-localization-accuracy table detection, and pairs a spatial CNN separator prediction module with a Grid CNN cell merger for table structure recognition. The approach is designed to handle geometrically distorted and even curved tables that confound axis-aligned methods. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$: The paper's center of gravity is the design and evaluation of two architectural components: a CornerNet-as-RPN table detector and a spatial CNN + Grid CNN structure recognizer. Extensive ablations and comparisons against published baselines on six public benchmarks support every claim. **Secondary:** none significant; the authors release manually annotated structure labels for cTDaR TrackA modern, but this is a minor byproduct rather than the paper's purpose. ## What is the motivation? Two gaps motivate this work. For **table detection**, existing CNN detectors (CDeC-Net, Cascade Mask R-CNN variants) achieve good recall but poor localization at high IoU thresholds. The underlying cause is that standard RPN generates proposals whose IoU with ground-truth boxes falls mostly in the 0.7–0.9 range; proposals with IoU > 0.9 represent only 48.1% of positive samples. Because these lower-quality positives survive NMS with high scores, the final detector is penalized whenever evaluation uses tight IoU thresholds. For **table structure recognition (TSR)**, the dominant cell-detection-then-clustering paradigm (TabStruct-Net, LGPMA) assumes tables are axis-aligned. Camera-captured documents in settings such as the "Insert data from picture" feature in Excel routinely contain skewed or curved tables that violate this assumption. Separately, row/column segmentation models built on plain ResNet-FPN cannot propagate enough context to disambiguate large blank regions in borderless tables. ## What is the novelty? **CornerNet-FRCN table detector.** The authors replace RPN with CornerNet to generate table proposals. CornerNet predicts paired top-left and bottom-right corner heatmaps, with offsets to compensate for quantization error from the stride-16 backbone: $$ p\_i^x = \left\lfloor \frac{q\_j^x}{s} \right\rfloor, \quad p\_i^y = \left\lfloor \frac{q\_j^y}{s} \right\rfloor $$ $$ \Delta\_i = \left( \left\lfloor \frac{q\_i^x}{s} \right\rfloor - \frac{q\_i^x}{s},\ \left\lfloor \frac{q\_i^y}{s} \right\rfloor - \frac{q\_i^y}{s} \right) $$ All valid top-left/bottom-right corner pairs are enumerated as proposals (x-left $<$ x-right and y-top $<$ y-bottom) and then passed through a standard Fast R-CNN head for classification and box refinement. The authors report that this design raises the fraction of well-localized proposals (IoU $>$ 0.9 within IoU $>$ 0.7 positives) from 48.1% (RPN) to 96.3% (CornerNet). The combined training loss is: $$ L\_{\text{detector}} = \lambda\_{\text{corner}} \cdot L\_{\text{corner}} + L\_{\text{frcn}}, \quad \lambda\_{\text{corner}} = 0.2 $$ where $L\_{\text{corner}}$ uses focal loss for heatmap classification and Smooth-$L\_1$ for offset regression, and $L\_{\text{frcn}}$ uses cross-entropy plus $L\_1$ box regression. **Spatial CNN separation line prediction.** A plain ResNet-18 + FPN feature map struggles with borderless tables because each pixel's receptive field is local. The spatial CNN module (adapted from lane detection) propagates information sequentially across the feature map. For the row separator branch, slices $\\{s\_i^w\\}$ along the width dimension are updated left-to-right and then right-to-left by convolving with a $9 \times 1$ kernel and adding to the adjacent slice. The column branch performs analogous top-to-bottom and bottom-to-top passes. The resulting context-enriched feature map drives pixel-level binary segmentation: $$ L\_{\text{split}} = \frac{1}{N\_{\text{row}}} \sum\_i L(R\_i, R\_i^\ast) + \frac{1}{N\_{\text{col}}} \sum\_j L(C\_j, C\_j^\ast) $$ **Grid CNN cell merging.** After cell generation via connected-component analysis and polynomial curve fitting, the detected cells are arranged in an $M \times N$ grid. RoI-aligned features per cell form a grid feature map $F\_{\text{grid}} \in \mathbb{R}^{M \times N \times 512}$. Three stacked $3 \times 3$ convolutions aggregate spatial context across this grid. A relation network then predicts whether each pair of 4-adjacent cells should be merged, using an 18-dimensional spatial compatibility vector $l\_{ij}$ encoding the box deltas: $$ \begin{aligned} t\_x^{ij} &= (x^i - x^j) / w^j, \quad & t\_y^{ij} &= (y^i - y^j) / h^j, \\\\ t\_w^{ij} &= \log(w^i / w^j), \quad & t\_h^{ij} &= \log(h^i / h^j), \\\\ t\_x^{ji} &= (x^j - x^i) / w^i, \quad & t\_y^{ji} &= (y^j - y^i) / h^i. \end{aligned} $$ $$ L\_{\text{merge}} = \frac{1}{N\_p} \sum\_i L(r\_i, r\_i^\ast) $$ The total structure recognizer loss is $L\_{\text{recognizer}} = L\_{\text{split}} + L\_{\text{merge}}$. ## What experiments were performed? **Datasets (TD):** cTDaR 2019 TrackA (600/240 modern, plus 600/199 historical), PubLayNet (335k/11k/11k), IIIT-AR-13K (9.3k/2k/2.1k). **Datasets (TSR):** SciTSR (12k/3k), PubTabNet (500k/9k/9k), cTDaR TrackB2-Modern (100 test images). The authors also evaluate TSR on a private in-house dataset of 9,000/700 camera-captured, skewed, or curved table images. **Metrics:** Weighted Average F1 (IoU $\in \\{0.6, 0.7, 0.8, 0.9\\}$) for cTDaR; COCO AP$^{0.5:0.95}$ for PubLayNet; PASCAL VOC AP for IIIT-AR-13K; adjacency relation F1 for SciTSR and cTDaR TrackB2; TEDS-Struct for PubTabNet. **Baselines:** CDeC-Net (strongest published TD baseline), LGPMA (ICDAR 2021 TSR winner), TabStruct-Net, GTE, EDD, SPLERGE (for distorted table comparison). **Ablations:** The paper systematically compares five message-passing methods for separation line prediction (no message passing, projection networks, Bi-GRU, CC Attention, Spatial CNN) and four cell merging methods (no merging, Relation Network, GCN, Grid CNN) on cTDaR TrackB2-Modern. ## What are the outcomes/conclusions? **Table detection:** RobusTabNet (ResNet-18 backbone) achieves WAvg. F1 of 94.9% on cTDaR TrackA (vs. 94.3% for CDeC-Net with a much heavier dual-ResNeXt-101 backbone), AP$^{0.5:0.95}$ of 97.0% on PubLayNet (vs. 96.7% for CDeC-Net), and test AP of 97.7% on IIIT-AR-13K (vs. 96.5% for Mask R-CNN with ResNet-101). The authors attribute these gains to proposal quality: CornerNet raises top-50 recall at IoU $\geq$ 0.9 from 89.3% (RPN) to 97.8%. **Table structure recognition:** RobusTabNet achieves F1 of 99.3% / 98.7% on SciTSR / SciTSR-COMP (vs. 98.8% / 98.0% for LGPMA), TEDS-Struct of 97.0% on PubTabNet (vs. 96.7% for LGPMA), and the highest reported WAvg. F1 among evaluated methods on cTDaR TrackB2-Modern. On the private distorted-table dataset, RobusTabNet scores 94.6% WAvg. F1 against SPLERGE's 63.8%. **Ablation findings:** Spatial CNN outperforms CC Attention (93.9%), Bi-GRU (93.1%), and projection networks (93.0%) for separation line prediction (94.6% vs. all alternatives). Grid CNN outperforms GCN (94.0%) and Relation Network (93.2%) for cell merging (94.6%). **Limitations:** The detector struggles with closely adjacent tables (difficulty disambiguating boundaries). The structure recognizer degrades on cells with multi-line content and on extremely dense tables where separator masks from adjacent rows or columns overlap. All public benchmarks consist of black-line/white-background tables; generalization to colored or stylized documents is untested. ## Reproducibility ### Models - **TD:** ResNet-18 with dilations in Conv5 (stride 16), reduced to 64 channels with a $1 \times 1$ conv. CornerNet corner pooling + 3$\times$3 conv appended to produce Dilated-C5'. Fast R-CNN head with two 1,024-d fc layers. Total parameter count not reported. - **TSR:** ResNet-18 + FPN (64 output channels) shared backbone. Spatial CNN branches with $\frac{H}{4} \times \frac{W}{32}$ intermediate resolution ($3 \times$ downsampling), propagating $9 \times 1$ kernels. Grid CNN: RoI Align $7 \times 7 \times 64$, two 512-d fc layers per cell, $3 \times$ stacked $3 \times 3$ convolutions, 2-hidden-layer MLP relation head. - No pretrained weights or code released. Weights initialized from ImageNet-pretrained ResNet-18; new layers initialized from $\mathcal{N}(0, 0.01)$. ### Algorithms - **Optimizer:** SGD with momentum 0.9, weight decay $5 \times 10^{-4}$. - **Schedule:** 15K iterations; base LR 0.032, decayed by $\times 0.1$ at 10K and 13K iterations. PubLayNet uses $3\times$ schedule. SciTSR and PubTabNet TSR models trained for 12 epochs. - **Batch size:** 4 images per GPU across 8 GPUs; synchronized batch normalization. - **Multi-scale training (TD):** shorter side randomly chosen from $\\{320, 416, 512, 608, 704, 800\\}$; cTDaR also uses random rotations in $\\{0^\\circ, 90^\\circ, 180^\\circ, 270^\\circ\\}$ with $\pm 5^\circ$ jitter. **Multi-scale training (TSR):** shorter side randomly chosen from $\\{416, 512, 608, 704, 800\\}$ using cropped table images. - **OHEM:** Equal hard positive/negative sampling (32/32 proposals for FRCN; 1,024/1,024 separator/background pixels per branch for TSR split; 64/64 cell pairs for merge). - **Loss weights:** $\lambda\_{\text{corner}} = 0.2$ (tuned on in-house dataset; applied without per-dataset tuning). ### Data - Public training sets: cTDaR TrackA (600 modern images annotated by the authors for TSR structure), PubLayNet (86k table-containing images used for TD), IIIT-AR-13K, SciTSR, PubTabNet. - Private in-house dataset: 9,000 training / 700 test camera-captured images with distorted/curved tables; not released. - The authors annotated row/column separation lines and cell bounding boxes for the cTDaR TrackA 600 modern images for TSR training; they state these annotations "will be released publicly," though no repository link was provided in either the preprint or the published journal version. As of this writing, no public release has been identified. ### Evaluation - **Metrics:** Standard per-dataset protocols. TEDS-Struct ignores OCR and evaluates structural HTML trees only. Adjacency F1 uses the official cTDaR measurement tool (https://github.com/cndplab-founder/ctdar_measurement_tool). - **Test-time scaling:** TD images rescaled so the shorter side is 512 px (longer side capped at 1,024 px). TSR: cropped table images rescaled so the longer side is 1,024 px; SciTSR images are not resized. - **Comparisons:** Single-model, single-scale inference throughout. - **Statistical rigor:** No error bars, significance tests, or multi-run variance reported. - **Known limitations:** LGPMA leverages the axis-aligned table constraint to boost PubTabNet performance; RobusTabNet does not use this constraint, making the comparison favorable to RobusTabNet on distorted images. - **Hyperparameter transfer:** All thresholds ($C\_{th}=0.3$, $S\_{th}=0.8$, merge score threshold $0.8$, top-$K$=100) were tuned on the private in-house dataset and applied without modification to public benchmarks. ### Hardware - 8 Nvidia V100 GPUs. - PyTorch 1.6.0. - Training GPU-hours not reported. - Inference: no latency figures reported; the paper does not quantify FPS or memory footprint at test time. ## BibTeX ```bibtex @article{ma2023robustabnet, title={Robust Table Detection and Structure Recognition from Heterogeneous Document Images}, author={Ma, Chixiang and Lin, Weihong and Sun, Lei and Huo, Qiang}, journal={Pattern Recognition}, volume={133}, pages={109006}, year={2023}, publisher={Elsevier}, doi={10.1016/j.patcog.2022.109006} } ```