# TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition > An end-to-end network that jointly predicts cell bounding boxes and logical row/column indices by framing table structure recognition as graph reconstruction - Authors: Hunter Heidenreich - Published: 2021-06-20 - Updated: 2026-03-16 - Paper: Preprint (arXiv) - Dataset: TableGraph-24K (dataset bundle) - Code: TGRNet - Model: Pretrained Checkpoints - Source: Bevaya Labs, --- ## TL;DR TGRNet reframes table structure recognition as a graph reconstruction problem where each cell is a node with both a spatial location (bounding box) and a logical location (start/end row and column indices). A two-branch architecture performs segmentation-based cell detection and GCN-based ordinal classification for logical index prediction. The authors also contribute TableGraph-350K, a large-scale dataset with over 350K table graph annotations derived from TABLE2LATEX-450K. ## What kind of paper is this? * **Dominant Basis:** $\Psi\_{\text{Method}}$ * The central contribution is an end-to-end architecture (TGRNet) that jointly detects cell spatial locations via segmentation and predicts cell logical locations via graph convolutional networks with ordinal regression. The paper devotes most of its pages to architecture design, loss formulation, and ablation studies. * **Secondary Basis:** $\Psi\_{\text{Resource}}$ * The paper introduces the TableGraph-350K dataset, providing table graph annotations (cell bounding boxes and logical indices) for over 350K tables from TABLE2LATEX-450K. Pretrained model checkpoints are also released. ## What is the motivation? Existing table structure recognition methods typically model tables in one of three ways: (1) detecting cell bounding boxes without inferring logical structure, (2) converting table images to markup sequences (LaTeX/HTML), or (3) predicting pairwise adjacency relations between cells. Each has limitations: * **Cell detection** approaches recover spatial locations but ignore the logical row/column indices that downstream applications (e.g., table QA systems like TAPAS) require. * **Markup sequence** approaches face a one-to-many mapping problem: the same table structure can be transcribed into different markup sequences, which introduces noise during training. * **Adjacency relation** approaches only capture pairwise relationships. Recovering global logical structure from pairwise adjacency requires additional graph optimization algorithms, which adds complexity and introduces error propagation. The authors argue that the **logical location** of each cell (its start/end row and column indices) is the most natural and complete representation of table structure. It can be used to derive adjacency matrices, markup sequences, or database formats, while the reverse transformations are harder. ## What is the novelty? ### Table Graph Formulation The paper introduces the concept of a **Table Graph** $G = (V, A)$, where each node $v\_i$ in $V$ represents a table cell with two attributes: $$ b\_i = (b\_i^x, b\_i^y, b\_i^w, b\_i^h) $$ $$ l\_i = (row\_i^{start}, row\_i^{end}, col\_i^{start}, col\_i^{end}) $$ Here $b\_i$ encodes the spatial bounding box (center, width, height) and $l\_i$ encodes the logical location (start-row, end-row, start-column, end-column). The adjacency matrix $A$ is constructed from Euclidean distances between cell centers, with separate row and column components: $$ a\_{i,j}^{row} = \exp\\{ -\left(\frac{b\_i^y - b\_j^y}{H} \cdot \alpha\right)^2 \\} $$ $$ a\_{i,j}^{col} = \exp\\{ -\left(\frac{b\_i^x - b\_j^x}{W} \cdot \alpha\right)^2 \\} $$ The adjustment factor $\alpha$ controls how sharply edge weights decay with distance, effectively limiting message passing to nearby cells. ### Two-Branch Architecture TGRNet uses a ResNet-50 backbone with FPN to extract multi-scale features, then splits into two branches: 1. **Cell Spatial Location Branch:** A segmentation module produces a 3-class pixel map (background, cell, boundary). A split-aggregation module pools features along row and column dimensions to capture tabular structure statistics. Cell bounding boxes are extracted as connected components from the segmentation map. 2. **Cell Logical Location Branch:** Detected cells initialize a graph. Node features combine spatial branch features with RoIAlign features. Parallel GCNs update node representations for row and column prediction separately. An ordinal classifier converts each logical index prediction into $T-1$ binary sub-problems. ### Ordinal Regression with Focal Loss The logical index $r\_i \in \\{0, 1, \ldots, T-1\\}$ is converted to a binary vector $q\_i \in \mathbb{R}^{T-1}$ where $q\_i^t = 1$ if $t < r\_i$. The loss function combines ordinal regression with a focal-style modulation to handle the long-tailed distribution of logical indices: $$ \Psi(x\_i', \Theta) = \sum\_{t=0}^{r\_i-1} (1 - p\_i^t)^{\gamma\_t} \log(p\_i^t) + \sum\_{t=r\_i}^{T-2} (1 - p\_i^t)^{\gamma\_t} \log(1 - p\_i^t) $$ $$ \gamma\_t = \min(2, -(1 - \lambda\_t)^2 \log(\lambda\_t) + 1) $$ where $\lambda\_t$ is the empirical probability of index $t$ in the training set. This adaptive focusing parameter assigns higher loss weight to rare (large) indices. ## What experiments were performed? ### Datasets * **TableGraph-24K:** A 24K-table subset of TableGraph-350K (20K train, 2K val, 2K test). Max row/column indices: 37/21. * **CMDD:** 476 medical laboratory report tables (372 train, 104 test). No spanning cells. Max row/column indices: 24/5. * **ICDAR13-Table:** 156 tables with spanning cells and varied styles. Split 50/50 for train/test. Max row/column indices: 57/12. * **ICDAR19-cTDaR (TrackB1):** 881 tables from archival historical documents (679 train, 202 test). Max row/column indices: 87/43. The largest table contains over 2,000 cells. ### Metrics * **Cell spatial location:** Precision, Recall, and Hmean at IoU threshold 0.5. * **Cell logical location:** Accuracy on four logical indices ($A\_{rowSt}$, $A\_{rowEd}$, $A\_{colSt}$, $A\_{colEd}$) and an overall accuracy $A\_{all}$ requiring all four to be correct. * **Combined:** $F\_{\beta=0.5}$ combining Hmean and $A\_{all}$. * **Adjacency-based:** Weighted average F-Score (WAF) for comparison with adjacency relation methods. ### Key Results On TableGraph-24K, TGRNet achieves 0.906 Hmean for cell detection and 0.832 $A\_{all}$ for logical location, yielding $F\_{\beta=0.5} = 0.890$. On ICDAR13-Table, performance drops to $F\_{\beta=0.5} = 0.519$, which the authors attribute to limited training data (78 tables) and domain shift from the pre-training data. On CMDD, using ground-truth cell boxes, TGRNet reaches 0.992 $A\_{all}$ for end-to-end table graph reconstruction. ### Robustness Analysis Experiments with randomly removed nodes (simulating imperfect cell detection) show TGRNet maintains relatively stable performance. At 80% cell coverage on CMDD, TGRNet achieves 0.857 $A\_{all}$ compared to ReS2TIM's 0.705. ### Ablation Studies On TableGraph-24K, each component contributes measurably: | Component | $A\_{all}$ | $F\_{\beta=0.5}$ | |:---|:---|:---| | Baseline (linear + cross-entropy) | 0.697 | 0.850 | | + GCN | 0.788 | 0.876 | | + Ordinal regression | 0.824 | 0.882 | | + Focal loss | 0.832 | 0.890 | The GCN provides the largest single improvement (+0.091 on $A\_{all}$), followed by ordinal regression (+0.036) and focal loss (+0.008). ### Logical Location vs. Adjacency Relation On ICDAR19-cTDaR, TGRNet achieves 0.267 $A\_{all}$ while ReS2TIM reaches only 0.138. However, ReS2TIM obtains a higher WAF (0.481 vs. 0.283) due to its pairwise relation loss being better aligned with the adjacency metric. The authors use this to illustrate that good adjacency prediction does not necessarily translate to accurate global logical structure recovery. ## What are the outcomes/conclusions? The paper demonstrates that framing table structure recognition as graph reconstruction provides a unified representation from which other formats (adjacency matrices, markup sequences, databases) can be derived. The ordinal classification formulation with focal loss effectively handles the long-tailed distribution of row/column indices. Key limitations include: * Performance degrades on datasets with limited training data and domain shift (ICDAR13-Table). * On challenging historical documents (ICDAR19-cTDaR), both cell detection and logical location prediction remain difficult, with $A\_{all}$ at only 0.267. * The approach relies on connected-component extraction from segmentation maps, which can fail when cells are tightly packed (requiring morphological post-processing). * Input images are resized to a fixed resolution (480 x 480 or 800 x 800), which may lose fine-grained details in large tables. ## Reproducibility ### Models * **Backbone:** ResNet-50 with FPN, producing 4 feature maps at strides 4, 8, 16, 32, each reduced to 256 channels. * **Cell spatial branch:** 1x1 conv to reduce 1024 to 256 channels, split-aggregation module with row/column average pooling, 3-class segmentation output. * **Cell logical branch:** Parallel GCN pair (one for rows, one for columns) with ordinal classifiers. Node features are 1280-dimensional (256 spatial + 1024 RoIAlign with 2x2 output). * **Pretrained checkpoints:** Available for CMDD, ICDAR13-Table, ICDAR19-cTDaR, and TableGraph-24K via the GitHub repository. ### Algorithms * **Pre-training strategy:** The cell spatial branch is pre-trained for 50 epochs with the logical branch frozen before end-to-end training on TableGraph-24K. * **Transfer learning:** For ICDAR13-Table, the model is initialized from TableGraph-24K-trained weights. * **Adjustment factor $\alpha$:** Set to 3 for most experiments; increased to 10 for ICDAR19-cTDaR due to higher cell density. * **Graph construction during training:** Only cells with IoU > 0.5 against ground truth are included as nodes. * **ICDAR19-cTDaR modifications:** Spatial and logical branches trained separately due to GPU memory constraints; morphological opening (3x3 kernel) applied to segmentation maps; graph edges pruned to top $8 \times N$ by weight. ### Data * **TableGraph-350K:** 358,767 tables (343,988 train / 7,420 val / 7,359 test) derived from TABLE2LATEX-450K. Annotations generated by parsing LaTeX source code with color-coded row/column borderlines to extract cell spatial and logical locations. Max row index: 48, max column index: 27. * **TableGraph-24K:** 24,000-table random subset (20K/2K/2K split). Max row index: 37, max column index: 21. * **Source data:** TABLE2LATEX-450K collects tables from arXiv articles with LaTeX source and rendered images. ### Evaluation * **Metrics:** Precision/Recall/Hmean at IoU 0.5 for spatial detection; per-index accuracies and $A\_{all}$ for logical location; $F\_{\beta=0.5}$ for combined performance; WAF for adjacency comparison. * **Baselines:** ReS2TIM is the primary comparison for logical location prediction. Fair comparison with spatial detection methods is noted as difficult due to inconsistent evaluation settings across prior work. * **Limitations acknowledged:** Small training sets, domain shift between datasets, and the challenge of large historical document tables. * **No error bars or multi-run statistics reported.** ### Hardware * The paper does not report specific GPU types, training times, or computational costs. ICDAR19-cTDaR experiments note GPU memory constraints requiring the two branches to be trained separately, and input images are resized to 800x800 (vs. 480x480 for other datasets). ## BibTeX ```bibtex @inproceedings{xue2021tgrnet, title={TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition}, author={Xue, Wenyuan and Yu, Baosheng and Wang, Wen and Tao, Dacheng and Li, Qingyong}, booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, pages={6721--6730}, year={2021} } ```