# TABLET: Table Structure Recognition using Encoder-only Transformers > A research note on TABLET, an ICDAR 2025 split-merge TSR method from iWudao Tech that reformulates row/column splitting as sequence labeling and cell merging as grid classification, achieving 18 FPS on dense financial tables with competitive TEDS scores. - Authors: Bevaya AI Team - Published: 2025-06-08 - Updated: 2026-03-23 - Paper: Preprint (arXiv) - Paper: Published (ICDAR 2025) - Source: Bevaya Labs, --- ## TL;DR TABLET is a split-merge TSR model that treats row and column splitting as 1D sequence labeling and cell merging as OTSL-based grid cell classification, using only Transformer encoders throughout. Trained on FinTabNet and PubTabNet, it reaches 98.54 TEDS on FinTabNet while processing 18 FPS end-to-end, roughly 2.5 times faster than the prior fastest reported split-merge method. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$ The headline contribution is the TABLET architecture: dual Transformer encoders for split sequence labeling and a third Transformer encoder for merge classification. The paper devotes most pages to the model design, ablations on encoder depth, and benchmark comparisons on FinTabNet and PubTabNet. **Secondary:** None. No dataset, benchmark, or tool is released alongside the method. ## What is the motivation? Table structure recognition in production environments faces two competing pressures: accuracy and throughput. End-to-end autoregressive approaches (e.g., TableMaster, TableFormer, VAST, TFLOP) generate HTML or OTSL sequences token by token, which gives them a natural advantage on TEDS because TEDS itself evaluates cell-level accuracy in a way that mirrors their output format. However, autoregressive decoding is inherently sequential, making it slow on large, densely populated tables. For financial documents (annual reports, earnings announcements), tables often have dozens of rows and columns with small fonts, creating long output sequences that slow autoregressive methods further. The authors also observe that for end-to-end methods, predicted cell bounding boxes frequently misalign with OCR-extracted text positions, causing errors in the final TEDS evaluation even when structure is correct. This TEDS-Struct vs. TEDS gap is visible in the paper's comparison tables (e.g., Zhu et al. show a gap exceeding 2 points; Ly et al. exceeding 3 points). Top-down split-merge methods avoid bounding box prediction and matching entirely, but existing approaches (TSRFormer, SEMv2, SEMv3) rely on Spatial CNN feature extractors or RNN-based sequence models that either introduce complexity or lag behind Transformer-based alternatives. The paper's goal is a split-merge model that uses only Transformer encoders while maintaining a concise, high-resolution feature pipeline suited for dense financial tables. ## What is the novelty? ### Overall Architecture TABLET is a two-stage pipeline. A split model segments the table into an $R \times C$ grid; a merge model then classifies each grid cell to reconstruct spanning cells. The two models are trained separately. Before inference, all table images are rescaled so that the longer side is 960 pixels (preserving aspect ratio) and padded to $960 \times 960$. ### Split Model A modified ResNet-18 (Max Pooling removed, all channel counts halved) combined with an FPN at 128 channels produces a feature map $F\_{1/2}$ of size $(H/2) \times (W/2) \times 128$, half the input resolution. Removing Max Pooling preserves fine-grained spatial detail that matters for small-font, dense tables. Global and local features are extracted along each axis independently: - **Horizontal (row splitting):** Global projection along the horizontal axis yields $F\_{RG}$ of shape $(H/2) \times 128$. Local features apply Average Pooling ($1 \times 2$) then $1 \times 1$ convolution to obtain $F\_{RL}$ of shape $(H/2) \times (W/4)$. Concatenation gives $F\_{RG+L}$ of shape $(H/2) \times (128 + W/4)$. - **Vertical (column splitting):** The same procedure along the vertical axis produces $F\_{CG+L}$ of shape $(W/2) \times (128 + H/4)$. Each feature map is fed into its own Transformer encoder (3 layers, 8 heads, $d\_{ff} = 2048$, dropout 0.1). The fixed input sequence lengths are $H/2 = 480$ and $W/2 = 480$; positional embeddings are 1D, randomly initialized, and learned end-to-end. Each encoder output position is passed through a linear head for binary classification (split vs. non-split line). Training uses Focal Loss: $$ FL\_{\text{split}} = \frac{1}{n\_h} \sum\_{i=1}^{n\_h} \alpha\_i (1-p\_i)^\gamma (-\log p\_i) + \frac{1}{n\_v} \sum\_{j=1}^{n\_v} \alpha\_j (1-p\_j)^\gamma (-\log p\_j) $$ with all class weights $\alpha = 1$ and focusing parameter $\gamma = 2$. Predictions are upsampled 2$\times$ to match input resolution. A post-processing rule reclassifies any non-split region containing no OCR text projection as a split region, handling empty rows and columns. ### Merge Model A standard ResNet-18 + FPN (256 channels) extracts a feature map $F\_{1/4}$ at $(H/4) \times (W/4) \times 256$. Grid cells from the split output are mapped to $F\_{1/4}$ via RoIAlign ($7 \times 7$), yielding a feature tensor $F\_{\text{grids}}$ of shape $R \times C \times (7 \times 7 \times 256) = R \times C \times 12544$. A two-layer MLP (Linear + ReLU, both layers 512-dimensional) compresses each cell to 512 dimensions, forming a sequence $S\_{\text{grids}}$ of length $R \times C$. A Transformer encoder (3 layers, 8 heads, $d\_{ff} = 2048$, dropout 0.1, maximum sequence length 640) with learnable 2D positional embeddings models cross-cell interactions. A linear classification head then assigns each grid cell one of four OTSL labels: - **C:** new cell (may contain content) - **L:** left-looking span (merges with the cell to its left) - **U:** up-looking span (merges with the cell above) - **X:** cross span (merges with both left and upper neighbors) The NL (newline) token from OTSL is unnecessary in a split-merge context and is not used. Merging again uses Focal Loss: $$ FL\_{\text{merge}} = \frac{1}{R \times C} \sum\_{k=1}^{R \times C} \alpha\_k (1-p\_k)^\gamma (-\log p\_k) $$ with $\alpha = 1$, $\gamma = 2$. The OTSL output is converted to HTML, and OCR-extracted text blocks are assigned to cells by position. ## What experiments were performed? ### Datasets - **FinTabNet v1.0.0:** 91,596 train / 10,656 val / 10,635 test tables from financial announcements. A manually refined subset of 9,681 test tables (Hou et al., same authors) with corrected annotations is also used. - **PubTabNet v2:** 500,777 train / 9,115 val tables from scientific articles. Note: PubTabNet does not provide original PDF files, so OCR is required to extract cell text. ### Metrics TEDS (content-inclusive), TEDS-Struct (structure only, ignoring cell content), and Accuracy (proportion of tables with fully correct structure and content). ### Ablations Tables 2 and 3 in the paper sweep the Transformer encoder depth (0, 1, 3, 6 layers) independently for the split and merge models. The 3-layer configuration consistently achieves the best results; deeper encoders (6 layers) degrade performance, suggesting the task does not benefit from additional capacity at this scale. ### Speed Comparison Inference speed is measured end-to-end (image resizing, Split-Merge inference, OCR text matching, HTML post-processing) on a single NVIDIA A100 80GB GPU at FP32. | Method | Adj. FPS | |:---|:---:| | TABLET | 18.01 | | RobusTabNet | ~7.33 | | TSRFormer | ~7.31 | | TRUST | ~4.44 | | SEMv2 | 7.30 (unadjusted; image size not reported by authors) | | VAST | ~0.69 | The authors also benchmark Qwen2.5-VL-7B on FinTabNet: zero-shot TEDS is more than 10 points below TABLET; after fine-tuning, the gap narrows to roughly 2 points, but FPS is over 100$\times$ slower. ## What are the outcomes/conclusions? **FinTabNet (test set):** TEDS Simple/Complex/All = 98.97 / 98.14 / 98.54; TEDS-Struct = 99.10 / 98.35 / 98.71; Accuracy = 88.18%. The authors report this as the highest TEDS among listed methods with image-only input. The near-zero gap between TEDS and TEDS-Struct (0.17 points) confirms that bounding box misalignment is absent for split-merge methods. **FinTabNet (refined test set):** TEDS = 98.87; TEDS-Struct = 98.99; Accuracy = 90.00%. **PubTabNet (val set):** TEDS = 96.79; TEDS-Struct = 97.67. Competitive but below DRCC (97.80 TEDS / 98.90 TEDS-Struct). The authors note that DRCC's semi-autoregressive approach is less sensitive to resolution than pixel-classification split-merge methods, which partly explains the gap. **Error analysis (FinTabNet):** The authors identify five failure modes: adjacent columns with overlapping text projections that prevent correct splitting; misaligned column headers; multi-line single-cell text mistakenly split into multiple rows; empty cell annotation inconsistencies; and cases where the model's output is actually correct but the FinTabNet annotation is wrong. **Acknowledged limitations:** The method assumes tables have already been extracted and dewarped; it does not handle raw document pages, distorted images, or tables in natural scenes. It depends on an OCR engine as preprocessing, so OCR quality directly affects final TEDS scores (especially visible on PubTabNet, where different methods use different OCR tools). The merge model sequence length is capped at 640 grid cells; very large tables that produce more grid cells than this are truncated. **Unacknowledged gaps:** No code or model weights are released. No error bars or multi-run statistics are reported. The Accuracy metric (fully correct tables) is only reported on FinTabNet, not PubTabNet. The comparison table on FinTabNet is incomplete (several methods report only TEDS-Struct, omitting TEDS), making direct comparison across all metrics impossible. ## Reproducibility | Resource | Type | License | Link | |:---|:---|:---|:---| | Preprint (arXiv) | Paper | arXiv-nonexclusive-v1.0 | [arxiv.org/abs/2506.07015](https://arxiv.org/abs/2506.07015) | | FinTabNet v1.0.0 | Dataset | CDLA-Permissive-1.0 | [github.com/ibm-aur-nlp/FinTabNet](https://github.com/ibm-aur-nlp/FinTabNet) | | PubTabNet v2 | Dataset | CDLA-Permissive-1.0 | [github.com/ibm-aur-nlp/PubTabNet](https://github.com/ibm-aur-nlp/PubTabNet) | No code, pretrained weights, or evaluation scripts were released with this paper. ### Models - **Split model:** Modified ResNet-18 (no MaxPool, half channels) + FPN (128 channels); two Transformer encoders (3 layers, 8 heads, $d\_{ff} = 2048$, dropout 0.1); 16.1M parameters total. - **Merge model:** Standard ResNet-18 + FPN (256 channels); RoIAlign $7 \times 7$; 2-layer MLP (512-dim); Transformer encoder (3 layers, 8 heads, $d\_{ff} = 2048$, dropout 0.1); 32.5M parameters total. - Total: approximately 48.6M parameters. - No pretrained weights or code repository released. ### Algorithms - Optimizer: AdamW ($\beta\_1 = 0.9$, $\beta\_2 = 0.999$, $\epsilon = 10^{-8}$, weight decay $5 \times 10^{-4}$). - Learning rate: $3 \times 10^{-4}$ for the split model (constant); polynomial decay schedule (power 0.9) for the merge model. - Gradient clipping: L2 norm with `max_norm` = 0.5. - Batch size: 32. - Training epochs: 16 (split model), 24 (merge model). - Focal Loss: $\alpha = 1$, $\gamma = 2$ for both split and merge. - Input resolution: 960$\times$960 (proportional resize + zero-padding). - Split region annotation preprocessing: regions narrower than 5 pixels are expanded symmetrically to a minimum of 5 pixels. ### Data - FinTabNet v1.0.0: 91,596 train / 10,656 val / 10,635 test. Available under CDLA-Permissive-1.0 from IBM Research. - PubTabNet v2: 500,777 train / 9,115 val. Available under CDLA-Permissive-1.0. PubTabNet does not provide original PDFs; cell text must be obtained with an OCR tool. - Refined FinTabNet test set (9,681 tables): released by the same authors in prior work (Hou et al.); exact availability not specified in this paper. - No new datasets are introduced or released. ### Evaluation - TEDS and TEDS-Struct computed over all tables, split into simple (no spans) and complex (at least one spanning cell) subsets. - Accuracy = percentage of tables with fully correct structure and content. - Baselines vary in what they report; direct comparison on all three metrics is not possible for most methods. - No error bars, significance tests, or multiple training runs reported. ### Hardware - Training: 2$\times$ NVIDIA A100 80GB GPUs. - Inference: 1$\times$ NVIDIA A100 80GB GPU; 18.01 FPS end-to-end (960$\times$960 input, full pipeline including OCR text matching and HTML post-processing). - No training time, total GPU-hours, or memory breakdown reported. ## BibTeX ```bibtex @inproceedings{hou2025tablet, title={TABLET: Table Structure Recognition using Encoder-only Transformers}, author={Hou, Qiyu and Wang, Jun}, booktitle={Proceedings of the International Conference on Document Analysis and Recognition (ICDAR)}, year={2025} } ```