# GraphTSR: Complicated Table Structure Recognition via Graph Neural Networks > Research notes on GraphTSR, a GNN-based model for PDF table structure recognition that frames cell adjacency prediction as edge classification on a graph, along with the SciTSR dataset it introduces. - Authors: Hunter Heidenreich - Published: 2026-03-18 - Paper: Preprint (arXiv) - Dataset: SciTSR Dataset and Evaluation Scripts - Source: Bevaya Labs, --- ## TL;DR GraphTSR reformulates table structure recognition (TSR) in PDFs as an edge-classification problem on a cell adjacency graph, using alternating edge-to-vertex and vertex-to-edge graph attention blocks to predict horizontal and vertical cell relations. The paper also introduces SciTSR, a 15,000-table dataset of scientific PDF tables with structure labels derived from LaTeX source, split 12,000/3,000 train/test. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$. The headline contribution is a new graph neural network architecture (GraphTSR) for recognizing table structure from PDF cell inputs. The paper's center of gravity is the architectural design, comparison against three baselines, and the per-dataset, per-metric performance tables. **Secondary:** $\Psi\_{\text{Resource}}$. The paper simultaneously releases SciTSR, a large-scale dataset that did not exist before this work and which has since become a standard benchmark in the TSR community. Without SciTSR, the experimental section would have been limited to the tiny ICDAR-2013 set (156 tables, no training split). ## What is the motivation? PDF table structure recognition requires understanding which cells are adjacent to which, including cells that span multiple rows or columns ("spanning cells"). Prior work fell into two camps: rule-based methods that relied on explicit layout heuristics, and a small set of deep learning approaches that treated the problem as image segmentation into row/column regions. Both camps handled simple grid-like tables reasonably well but struggled with tables containing spanning cells, which carry disproportionately important semantic content (they are often headers). The existing benchmark (ICDAR-2013) had only 156 tables and no training split, which severely constrained data-driven research. The authors identify both the methodological gap (no principled way to handle spanning cells) and the resource gap (no adequate training corpus) as jointly limiting progress. ## What is the novelty? GraphTSR treats each cell as a node and the task as predicting whether any two cells share a horizontal or vertical adjacency edge. The pipeline has four stages: 1. **Pre-processing**: extract cell text content and bounding boxes from the PDF using an existing tool (same procedure as Shigarov et al., 2016). 2. **Graph construction**: connect each cell to its $K = 20$ nearest neighbors by spatial distance, yielding an approximately sparse graph with $O(K|V|)$ edges rather than $O(|V|^2)$. 3. **Relation prediction**: run GraphTSR to classify each edge as `vertical`, `horizontal`, or `no relation`. 4. **Post-processing**: convert the labeled graph back into a structured table representation. The core model maintains two representation streams, one over vertices (cells) and one over edges (candidate adjacency pairs), updated by alternating attention blocks. The **edge-to-vertex** block updates each cell's representation by attending over its neighboring edge nodes; the **vertex-to-edge** block updates each edge's representation by attending over its neighboring cell nodes. Graph attention is local (restricted to the $K$-NN neighborhood) rather than global: $$ a\_{iu} = \frac{e^{\mathbf{k}\_i^\top \mathbf{q}\_u / \sqrt{d\_k}}}{\sum\_{j \in N(u)} e^{\mathbf{k}\_j^\top \mathbf{q}\_u / \sqrt{d\_k}}} $$ $$ \text{GraphAtt}(\mathbf{q}\_u, \mathbf{K}, \mathbf{V}) = \sum\_{j \in N(u)} a\_{ju} \mathbf{v}\_j $$ Each attention block wraps the graph attention with residual connections and layer normalization, following the Transformer block structure: $$ \tilde{\mathbf{H}}\_v^{(n)} = \text{Norm}\!\left(\mathbf{H}\_v + \text{GraphAtt}(\mathbf{Q}, \mathbf{K}, \mathbf{V})\right) $$ $$ \mathbf{H}\_v^{(n)} = \text{Norm}\!\left(\tilde{\mathbf{H}}\_v^{(n)} + \text{FFN}(\tilde{\mathbf{H}}\_v^{(n)})\right) $$ **Vertex features** include cell size, absolute bounding-box coordinates, and relative positions. **Edge features** include Euclidean, x-axis, and y-axis distances in both absolute and relative form, plus x-axis and y-axis overlap between cell pairs. These hand-crafted features are fed as initial representations to the two streams. The key design decision is representing the original graph as a bipartite graph (cells and candidate edges as separate node sets), so that both vertex and edge states can be updated with the same graph attention mechanism. ## What experiments were performed? **Datasets:** SciTSR (12,000 train / 3,000 test), ICDAR-2013 (test-only, 156 tables), and SciTSR-COMP (the 716 complicated tables from the SciTSR test set, used as a harder held-out subset). **Metric:** The ICDAR 2013 evaluation protocol, which converts a table to a list of horizontally and vertically adjacent cell pairs and computes precision, recall, and F1. Both macro- and micro-averaged scores are reported. **Baselines:** - Tabby (Shigarov et al., 2016): rule-based PDF table extractor - DeepDeSRT (Schreiber et al., 2017): CNN-based semantic segmentation of row and column regions - Adobe Acrobat DC SDK: commercial table-to-HTML extraction Note that both DeepDeSRT and GraphTSR are trained on SciTSR training data and then evaluated zero-shot on ICDAR-2013, testing cross-domain generalization. **Training details:** Adam optimizer, initial learning rate 0.0005, batch size 1 (one graph per step), 15 epochs, L2 weight decay $\lambda = 0.0001$, dropout $p = 0.4$ on each sub-layer output. Class imbalance is addressed with a manual rescaling weight: 0.2 for `no relation` edges and 1.0 for `vertical` and `horizontal` edges in the cross-entropy loss. ## What are the outcomes/conclusions? On ICDAR-2013, GraphTSR achieves macro-F1 of 0.837 and micro-F1 of 0.872, surpassing Tabby (0.816 / 0.854) and substantially outperforming DeepDeSRT (0.568 / 0.615). The cross-domain gap for DeepDeSRT suggests that image-based models are sensitive to table rendering style, while GraphTSR's text-and-geometry features generalize better. On SciTSR, GraphTSR achieves macro-F1 of 0.934 and micro-F1 of 0.953, ahead of Tabby (0.912 / 0.921) and DeepDeSRT (0.897 / 0.890). The more informative comparison is SciTSR-COMP. All baselines drop at least 4 points relative to full SciTSR; GraphTSR retains macro-F1 0.934 and micro-F1 0.955, while Tabby falls to 0.855 / 0.882. On the most demanding sub-evaluation (spanning cells only, ignoring non-spanning adjacencies), the advantage is sharper: GraphTSR reaches macro-F1 0.703 vs. Adobe's 0.485 and Tabby's 0.379. DeepDeSRT cannot recover any spanning-cell structure at all (its outputs are always grid-like). The authors conclude that framing TSR as graph edge prediction rather than image segmentation or sequence generation naturally handles spanning cells, because each cell-to-cell relation is predicted independently and the model is not forced into a grid assumption. **Limitations (stated and unstated):** - The pre-processing step depends on a rule-based PDF text extractor to supply cell bounding boxes; the model itself does not handle raw PDF pages or tables from scanned images. - The SciTSR tables come entirely from scientific papers on arXiv compiled from LaTeX source; tables from other domains (business reports, government documents, HTML pages) are not represented, and the LaTeX-derived labels may not generalize to noisier real-world PDFs. - Training requires a ground-truth cell-bounding-box-to-LaTeX-cell matching step ("cell matching"), which relies on accurate pre-processing and clean LaTeX source; this step is not evaluated separately. - No ablation is provided over the number of attention blocks, the KNN $K$ value, or the feature engineering choices. - GPU vs. CPU training is not discussed in the context of scalability or inference speed. - DeepDeSRT was reimplemented by the authors rather than using original code (none was available). A reimplemented baseline may not match the original's tuning, which puts the comparison's fairness in question. - SciTSR was designed and constructed to match GraphTSR's input format: pre-extracted cell bounding boxes from clean LaTeX source. Image-based methods like DeepDeSRT are disadvantaged on this benchmark by design, since the benchmark natively favors text-and-geometry approaches. ## Reproducibility ### Models GraphTSR is a 4-block model with hidden dimension $d = 64$, implemented in PyTorch 0.4.1. The bipartite-graph formulation has edge-to-vertex and vertex-to-edge attention blocks, each following the Transformer residual-norm-FFN pattern. Total parameter count is not reported. No pretrained checkpoints are released. The GitHub repository contains the SciTSR dataset (via Google Drive download) and evaluation scripts for computing precision/recall/F1 against ground-truth relation labels; it does not include GraphTSR training code. ### Algorithms - Optimizer: Adam, learning rate 0.0005 - Batch size: 1 graph per step - Epochs: 15 - Dropout: $p = 0.4$ on each sub-layer output - Weight decay: L2 with $\lambda = 0.0001$ - Loss: cross-entropy with class weights (0.2 for `no relation`, 1.0 for adjacency classes) - KNN graph construction: $K = 20$ - No data augmentation or curriculum learning described ### Data SciTSR is collected by crawling LaTeX source files from arXiv, extracting table environments, compiling them to individual PDF files, and parsing `\multirow`/`\multicolumn` commands to produce JSON structure labels. The full dataset has 15,000 tables (12,000 train, 3,000 test). The 716 complicated-table test subset (SciTSR-COMP) is the subset of the test split that contains at least one spanning cell. The dataset is publicly available under the MIT license. A notable caveat: since labels come from LaTeX source, the dataset cannot easily include tables from scanned documents or proprietary PDF producers. It is also restricted to the scientific paper style of arXiv submissions, which tend to be typeset consistently. ### Evaluation The ICDAR 2013 evaluation protocol converts each table to a set of adjacent-cell relation tuples and computes precision, recall, and F1 by set overlap. Both macro (per-table average) and micro (per-relation aggregate) variants are reported. No error bars, significance tests, or multi-run averages are provided. ICDAR-2013 has no training split; results on it test zero-shot generalization from SciTSR training data. ### Hardware Training runs on Intel Xeon CPUs. Each epoch over the 12,000 SciTSR training graphs takes approximately 20 minutes, giving roughly 5 hours of total training time. No GPU is used or reported. Inference hardware and latency are not discussed. ## BibTeX ```bibtex @misc{chi2019complicated, title={Complicated Table Structure Recognition}, author={Chi, Zewen and Huang, Heyan and Xu, Heng-Da and Yu, Houjin and Wanxuan Yin and Xian-Ling Mao}, year={2019}, eprint={1908.04729}, archivePrefix={arXiv}, primaryClass={cs.IR} } ```