# LORE++: Pre-Training Boosts Logical Location Regression for Table Structure Recognition

> LORE++ extends the LORE table structure recognition framework with two pre-training tasks (MAE and Logical Distance Prediction) on 1.5 million table images, improving accuracy, generalization, and few-shot performance.

- Authors: Hunter Heidenreich
- Published: 2024-01-03
- Updated: 2026-03-26
- Paper: Preprint (arXiv) <https://arxiv.org/abs/2401.01522>
- Source: Bevaya Labs, <https://labs.bevaya.ai/>

---


## TL;DR

LORE++ extends the LORE table structure recognition (TSR) framework by introducing two pre-training tasks: a Masked Autoencoder (MAE) task to learn visual table clues, and a Logical Distance Prediction (LDP) task to learn grid-level logical relationships between text regions. Pre-training on 1.5 million table images consistently boosts accuracy, data efficiency, and generalization compared to the LORE baseline, without changing the underlying logical-location-regression paradigm.

## What kind of paper is this?

**Dominant:** $\Psi\_{\text{Method}}$: the paper extends an existing TSR framework (LORE) with a pre-training approach. The core contributions are the pre-training architecture, the LDP task, and the joint-training masking strategy. The headline contribution is methodological.

**Secondary:** none.

## What is the motivation?

The original LORE model demonstrated that formulating TSR as logical coordinate regression is both simpler and more accurate than adjacency-based or markup-sequence-based paradigms. However, it did not exploit the large pool of table data available from sources like PubTables-1M (over one million tables) and TableBank. In contrast, pre-trained models had driven major gains in both computer vision and NLP.

Two concrete limitations motivated the upgrade. First, ViT architectures, which are more natural fits for MAE pre-training, actually underperform the CNN-based LORE baseline when trained from scratch on limited TSR data (82.1% vs. 82.9% accuracy on WTW), indicating that sufficient pre-training is needed to realize their potential. Second, TSR requires not just visual understanding of cell regions and ruling lines, but also comprehension of logical grid relationships: which cells share a row or column, and by how much they are separated. No prior TSR pre-training work addressed both aspects jointly.

## What is the novelty?

**Logical Distance Prediction (LDP).** The key contribution is a pre-training proxy task tailored to TSR's unique demand for logical structure understanding. For each pair of text regions in a table image (obtained from an off-the-shelf OCR engine), the model is trained to predict the row and column logical distances between them. Unlike labeled TSR datasets, the LDP supervision can be generated automatically from any table image with readable text, enabling pre-training at scale.

Concretely, OCR results give 2D positions of text regions, which are then clustered into grid rows and columns. The logical distance between text regions $i$ and $j$ is then computed as:

$$
\Delta r\_{ij} = |r^{(i)} - r^{(j)}|, \quad \Delta c\_{ij} = |c^{(i)} - c^{(j)}|
$$

The logical decoder is trained with an L1 loss on the predicted distances.

**Joint pre-training with unidirectional self-attention.** LORE++ trains MAE and LDP jointly in a single forward pass. To prevent information leakage from masked patches to unmasked ones (which would make MAE trivial), the encoder uses a unidirectional self-attention scheme: unmasked tokens attend only to each other, while masked tokens attend to all patches. This lets the encoder produce meaningful representations for both tasks without separate passes.

**Architecture.** LORE++'s encoder is a 12-layer ViT with 12-head self-attention, hidden size 384, and feed-forward intermediate size 1536 (comparable in parameter count to ResNet-50 or DLA-34). The spatial decoder follows ViT-MAE design. The logical decoder shares the self-attention-plus-linear structure of the LORE base and stacking regressors but operates on paired text-region features for distance prediction. During fine-tuning, the ViT encoder initializes from the pre-trained encoder, and both the base and stacking regressors initialize from the pre-trained logical decoder.

**Pre-training data.** A corpus of 1.5 million table images is assembled from PubTables-1M, TableBank, and smaller academic datasets. LDP labels are generated automatically via OCR and clustering, requiring no manual annotation.

## What experiments were performed?

**Benchmarks** cover both digital-born and physical tables:

- ICDAR-2013: 238 table images; cross-validation only (no training split).
- SciTSR-comp: complex-table subset of SciTSR.
- PubTabNet: 20,000 training images sampled from the full 339k.
- WTW: wild table photos with distortion and spanning cells.
- TableGraph-24K: digital-born scientific tables.
- ICDAR-2019: scanned document tables.
- TableBank: markup-sequence benchmark.

**Metrics** include three types to allow comparison across paradigms: logical location accuracy (fraction of cells with all four indices exactly correct), adjacency F1 (precision/recall on adjacent cell pairs), and TEDS/BLEU for markup-sequence methods.

**LORE vs. LORE++ comparison (Table III)** is the primary quantitative result. Both models are evaluated on ICDAR-2013, SciTSR-comp, PubTabNet, and WTW using detection F1 (D-F1), relation F1 (R-F1), and logical location accuracy (Acc).

**Pre-training ablation (Table V)** systematically disentangles: (a) CNN baseline (LORE), (b) ViT with no pre-training, (c) ViT pre-trained with MAE on ImageNet, (d) ViT pre-trained with MAE on the curated table dataset, (e) ViT pre-trained with MAE and LDP jointly (LORE++).

**Data efficiency experiments (Figure 9)** train LORE and LORE++ on 20%, 60%, and 100% of WTW and SciTSR-comp training data.

**Generalization experiments (Table IX)** train on a hybrid dataset (WTW + 20k PubTabNet + TableGraph-24K) and evaluate on the held-out ICDAR-2013 and SciTSR-comp test sets.

Training used an Adam optimizer with weight decay 0.05, $(\beta\_1, \beta\_2) = (0.0, 0.95)$, learning rate $1.5 \times 10^{-4}$ with 5% linear warm-up. Fine-tuning follows the same schedule as LORE: initial LR $1 \times 10^{-4}$, decay to $1 \times 10^{-5}$ at epoch 70 and $1 \times 10^{-6}$ at epoch 90 over 100 epochs.

## What are the outcomes/conclusions?

**Accuracy improvements.** LORE++ consistently outperforms LORE across all evaluated datasets (Table III). The largest gains appear on ICDAR-2013: logical accuracy improves from 86.8% to 93.2%, and D-F1 from 97.2% to 98.5%. On WTW (the most challenging benchmark), logical accuracy improves from 82.9% to 84.1% and D-F1 from 96.4% to 97.0%. The pre-training also boosts PubTabNet logical accuracy from 91.0% to 92.7%.

**Pre-training task contributions (Table V).** Using only MAE on the table-domain data (3d) already beats MAE pre-training on ImageNet (3c) at 83.2% vs. 82.1% accuracy on WTW, suggesting that domain-specific pre-training matters even with a smaller dataset. Adding LDP (3e, LORE++) pushes accuracy to 84.1%, with R-F1 improving from 96.4% to 96.9%. Notably, spatial detection also benefits from the LDP task, indicating that logical and spatial understanding are mutually reinforcing.

**Data efficiency.** LORE++ trained on 60% of SciTSR-comp data matches LORE trained on 100% of that data. On WTW, LORE++ maintains a consistent advantage across all data fractions.

**Generalization.** In the cross-dataset setting (Table IX), LORE++ boosts logical accuracy on ICDAR-2013 from 78.6% to 87.5% and on SciTSR-comp from 87.1% to 93.5%.

**Inference efficiency.** Despite switching to a ViT encoder, LORE++ achieves slightly faster inference than LORE (0.43 s vs. 0.45 s per image) and uses 29.7M parameters vs. 24.2M, representing a modest increase (Table VIII, FLOPs: 88.3G vs. 75.2G). Both are far more efficient than the EDD markup-sequence baseline at 14.8 s per image using 339k training samples.

**Limitations acknowledged by the paper.** The ViT architecture performs worse than the CNN-based LORE when trained from scratch on limited data, so the gains are contingent on having access to the pre-trained weights. The paper evaluates on a fixed set of benchmarks and does not include newer, harder datasets such as FinTabNet. LDP supervision requires an OCR system at pre-training time, and OCR errors could introduce noise in the logical-distance labels. One issue the paper does not address is that TableBank appears in both the pre-training corpus and as a downstream evaluation benchmark, which raises a potential contamination concern for those results.

## Reproducibility

### Models

- Encoder: 12-layer ViT, 12-head self-attention, hidden size 384, feed-forward intermediate size 1536, roughly comparable in parameter count to ResNet-50. Total model parameters: 29.7M (FLOPs: 88.3G at 1024x1024 with 32 cells).
- Logical decoder (pre-training) and base/stacking regressors (fine-tuning): 3 self-attention layers each, matching the vanilla LORE configuration.
- The LORE code base is released under Apache-2.0 at the Alibaba AdvancedLiterateMachinery repository for the original LORE; no separate LORE++ release is confirmed as of this writing.
- ViT encoder is initialized with MAE pre-trained weights; logical decoder initializes the base and stacking regressors at fine-tuning time.

### Algorithms

- Pre-training optimizer: Adam with weight decay 0.05, $(\beta\_1, \beta\_2) = (0.0, 0.95)$, learning rate $1.5 \times 10^{-4}$, 5% linear warm-up.
- Fine-tuning: initial LR $1 \times 10^{-4}$, step decay to $1 \times 10^{-5}$ at epoch 70 and $1 \times 10^{-6}$ at epoch 90, over 100 epochs.
- Pre-training batch size: 196. Total pre-training step count is not specified in the paper ("for steps" is stated without a number).
- MAE masking ratio: 50% of patches (lower than the 75% used in the original ViT-MAE, to account for the information-dense nature of table images).
- Unidirectional self-attention: unmasked tokens attend only to each other; masked tokens attend to all.
- L1 loss for logical distance prediction; MSE for pixel reconstruction (MAE).
- Input resolution: 224 during pre-training; max-side 1024 (512 for SciTSR and PubTabNet) during fine-tuning.

### Data

- Pre-training: 1.5 million table images from PubTables-1M, TableBank, and additional small-scale academic datasets.
- LDP labels generated by: off-the-shelf OCR to locate text regions, then clustering horizontal/vertical positions to assign grid row/column indices.
- Fine-tuning datasets: the same benchmarks as LORE (ICDAR-2013, SciTSR-comp, PubTabNet with 20k samples, TableGraph-24K, ICDAR-2019, WTW, TableBank).
- No held-out pre-training validation set is described; potential overlap between pre-training tables and fine-tuning/test tables is not explicitly analyzed. Notably, TableBank is used in both the pre-training corpus and as a fine-tuning/evaluation benchmark, creating a direct contamination risk that the paper does not address.

### Evaluation

- Logical location accuracy: fraction of cells with all four indices $(r\_s, r\_e, c\_s, c\_e)$ exactly correct. Reported separately for column indices (A-C), row indices (A-R), and all four (Acc).
- Adjacency F1: precision/recall on predicted adjacent cell pairs, following the ICDAR 2013 methodology.
- TEDS: tree edit distance similarity between predicted and ground-truth HTML trees; PDF text extracted following Zhong et al. (2020).
- BLEU (4-gram) for TableBank markup evaluation.
- 5 runs averaged for all reported results; no significance tests or confidence intervals provided.
- Ablation studies are conducted on WTW only; it is unclear whether component conclusions generalize across benchmarks.

### Hardware

- 4x NVIDIA Tesla V100 GPUs.
- GPU-hours and memory requirements not reported.
- Inference: 0.43 s/image at 1280x1280 on PubTabNet validation set (LORE++), vs. 0.45 s/image (LORE) and 14.8 s/image (EDD).

## BibTeX

```bibtex
@article{long2024loreplusplus,
  title={{LORE++}: Logical Location Regression Network for Table Structure Recognition with Pre-training},
  author={Long, Rujiao and Xing, Hangdi and Yang, Zhibo and Zheng, Qi and Yu, Zhi and Yao, Cong and Huang, Fei},
  journal={arXiv preprint arXiv:2401.01522},
  year={2024}
}
```