# TabSniper: Table Detection and Structure Recognition for Bank Statements

> TabSniper is an end-to-end pipeline from American Express for table detection, categorization, and structure recognition tailored to bank statements, introducing the BankTabNet dataset and a DETR-based model with CIoU loss and long-table split-merge handling.

- Authors: Hunter Heidenreich
- Published: 2024-12-17
- Updated: 2026-03-26
- Paper: Preprint (arXiv) <https://arxiv.org/abs/2412.12827>
- Source: Bevaya Labs, <https://labs.bevaya.ai/>

---


## TL;DR

TabSniper is an end-to-end bank statement processing pipeline from American Express that chains table detection and categorization (TDC) with table structure recognition (TSR), both built on DETR fine-tuned from PubTables-1M. The accompanying BankTabNet dataset provides 11,607 page-level and 5,165 table-level annotations from real bank statements. The primary contribution is a deployed workflow for extracting transactions from diverse bank templates, with CIoU loss substitution and a long-table split-merge strategy as the key model-level changes.

## What kind of paper is this?

**Dominant:** $\Psi\_{\text{Impact}}$ (translational/operational). The headline contribution is a validated end-to-end pipeline for a specific industrial use case: extracting credit and debit transactions from bank statements for credit underwriting. The paper describes the full workflow from raw PDF pages to a structured JSON of transactions, includes a post-processing checksum (balance reconciliation), reports CPU inference time as a primary metric, and is explicitly motivated by real-time deployment constraints. The "win" is the validated operational outcome, not a new modeling paradigm.

**Secondary:** $\Psi\_{\text{Method}}$ (the CIoU loss substitution and split-merge strategy for long tables are genuine, if modest, algorithmic contributions) and $\Psi\_{\text{Resource}}$ (BankTabNet is a purpose-built annotated dataset for this domain).

## What is the motivation?

Bank statement processing is a core task in credit underwriting: lenders need to assess cash flows, spending patterns, and transaction histories before making lending decisions. Unlike structured financial reports, bank statements vary substantially across issuers in layout, template, and table structure. Scanned and digitally generated PDFs coexist, and tables routinely span multiple pages with densely packed multi-line rows.

Standard TSR datasets (PubTabNet, FinTabNet, PubTables-1M) are derived from scientific articles and corporate financial reports. The authors argue these do not capture the specific characteristics of bank statement tables: long tables (often more than 20 rows) with variable intra-cell text spacing that misleads column boundary detection, multiple table categories on a single page (credit, debit, check, transaction balance, summary), and the need to categorize table types for downstream processing.

No existing public dataset addressed bank statement TSR specifically, and no end-to-end system had demonstrated reliable transaction extraction across multiple bank templates. The paper is motivated by this operational gap.

## What is the novelty?

The novelty is concentrated in three areas:

**1. End-to-end bank statement pipeline.** TabSniper chains TDC and TSR into a single workflow designed for real-time CPU deployment. A one-time OCR call is shared across both stages, and a postprocessing layer sorts tables by page order, matches column headers to synonyms for standard transaction fields (Date, Amount, Debit, Credit, Balance), and computes a balance checksum:

$$
\text{Checksum} = \text{OpenBal} - \sum\_{t \in \text{Debit}} t + \sum\_{t \in \text{Credit}} t - \text{EndBal}
$$

A zero checksum confirms that all transactions were extracted for a given statement.

**2. CIoU loss for TSR.** The baseline DETR uses GIoU loss. The authors identify that GIoU does not account for overlapping area, center distance, or aspect ratio, which causes false positive row bounding boxes in densely packed bank tables. They substitute CIoU, which incorporates a normalized center-distance penalty (DIoU) plus an aspect-ratio consistency term:

$$
L\_{\text{CIoU}} = L\_{\text{DIoU}} + \alpha \cdot \upsilon
$$

$$
\upsilon = \frac{4}{\pi^2} \left(\arctan \frac{w^{gt}}{h^{gt}} - \arctan \frac{w}{h}\right)^2, \quad \alpha = \frac{\upsilon}{(1 - \text{IoU}) + \upsilon}
$$

The full TSR loss becomes:

$$
L\_{\text{TSR}} = \sum\_{n=1}^{N} \lambda\_{ce} \cdot L\_{CE} + \lambda\_{l1} \cdot L\_{L1} + \lambda\_{ciou} \cdot L\_{\text{CIoU}}
$$

**3. Long-table split-merge.** Tables with more than 20 rows are split horizontally into two sub-images before passing to the TSR model (which uses a fixed query count of $N = 125$). Predictions are merged at postprocessing time. This addresses the attention drift and missed detections that arise when the fixed-capacity transformer processes tables that are longer than those in its training distribution.

**4. Multi-modal TDC.** The TDC stage runs a DETR vision model first, then refines Credit/Debit categorization using a text-based Multinomial Naive Bayes classifier trained on table caption and header text. Because caption text is outside the table bounding box (and therefore invisible to the vision model), this two-stage multi-modal approach separates detection from categorization.

**5. BankTabNet dataset.** The authors introduce two new annotated datasets sourced from real, PII-masked bank statements: a TDC dataset (11,607 page images, 10 table categories) and a TSR dataset (5,165 table images, 5 object classes). Inter-annotator agreement is measured with Krippendorff's Alpha ($\kappa \geq 0.955$ at $\text{IoU} > 0.5$ for TDC; $\kappa \geq 0.99$ for TSR).

## What experiments were performed?

**TDC evaluation.** TabSniper-TDC is compared against Faster R-CNN, Mask R-CNN, Cascade R-CNN, DiT-B (Cascade), and Dynamic Head, all evaluated on the BankTabNet TDC test split. Metrics are $AP$, $AP\_{50}$, and $AP\_{75}$ (COCO-style). CPU inference time on Apple M1 Max (32 GB) is also reported.

| Model | AP | $AP\_{50}$ | $AP\_{75}$ | CPU time (s/img) |
|---|---|---|---|---|
| Faster R-CNN | 83.50 | 94.55 | 90.25 | 1.71 |
| Mask R-CNN | 83.84 | 95.14 | 90.13 | 1.72 |
| Cascade R-CNN | 84.08 | 93.07 | 89.18 | 2.34 |
| TabSniper-TDC | 85.25 | 93.91 | 90.69 | **1.25** |
| DiT-B (Cascade) | 87.96 | 95.73 | 92.57 | 8.85 |
| Dynamic Head | 89.04 | 97.39 | 94.43 | N/A (GPU only) |

TabSniper-TDC sits below DiT and Dynamic Head on AP but delivers the lowest CPU inference time among all models that support CPU inference.

**TSR evaluation with ablations.** Three ablations are stacked incrementally on BankTabNet TSR test:

| Model / Ablation | $AP\_{50}$ | $AP\_{75}$ | AP | AR |
|---|---|---|---|---|
| TabStructNet | 79.2 | 71.0 | 65.4 | 73.6 |
| LGPMA | 84.8 | 76.1 | 70.0 | 78.8 |
| Table-Transformer (Base) | 85.3 | 76.5 | 70.4 | 79.3 |
| + Split-Merge Long Tables | 92.6 | 80.8 | 72.5 | 81.2 |
| + Padding Variations | 94.6 | 89.8 | 80.2 | 87.2 |
| + Complete IoU Loss | **94.8** | **91.5** | **83.1** | **90.6** |

Each ablation contributes meaningfully, with padding variations delivering the largest single jump in $AP\_{75}$ (from 80.8 to 89.8).

**TSR on external public datasets.** TabSniper-TSR is trained from scratch on PubTables-1M and FinTabNet respectively and compared to Table Transformer:

| Dataset | Model | $AP\_{50}$ | $AP\_{75}$ | AP | AR |
|---|---|---|---|---|---|
| PubTables-1M | Table-Transformer | 96.3 | 92.3 | 84.4 | 89.3 |
| PubTables-1M | TabSniper-TSR | 96.2 | **93.8** | **89.6** | **93.3** |
| FinTabNet | Table-Transformer | 97.4 | 94.2 | 88.8 | 93.1 |
| FinTabNet | TabSniper-TSR | 96.0 | 93.0 | **89.1** | **93.4** |

TabSniper shows modest gains on AP and AR (localization over a range of IoU thresholds) while being slightly below baseline on $AP\_{50}$ for FinTabNet.

**Text classifier evaluation.** Three Multinomial Naive Bayes classifiers (Header NB, Caption NB, Header\_Caption NB) are evaluated on per-category F1. Header\_Caption NB achieves the best results across all categories.

## What are the outcomes/conclusions?

The system demonstrates that a DETR-based pipeline, adapted with domain-specific techniques (long-table split-merge, CIoU loss, padding augmentation, multi-modal TDC), can reliably extract transactions from diverse bank templates at CPU inference speeds suitable for real-time processing (1.91 minutes for a 20-page statement on Apple M1 Max).

The balance checksum mechanism provides a form of deterministic validation: if the extracted debits and credits reconcile to the opening and closing balances, the extraction is likely complete.

On the internal BankTabNet TSR test set, TabSniper achieves AP 83.1 and AR 90.6, substantially above all baselines. On public datasets, it is competitive with Table Transformer, particularly on AP and AR, suggesting the model modifications generalize modestly beyond the bank statement domain.

**Limitations acknowledged by the authors:**
- Tilted tables (from mobile phone captures) cause bounding box misalignment and partial OCR capture. The authors note this could be addressed with document angle correction, which they plan for future versions.
- The paper does not report end-to-end transaction extraction accuracy (e.g., percentage of statements with checksum zero, false transaction rates), limiting assessment of the pipeline's actual production performance.

**Limitations not acknowledged:**
- BankTabNet is proprietary (AmEx internal, PII-masked). The dataset is described but not released, making the bank-statement-specific results unreproducible for external researchers.
- No code or model weights are released. All components (DETR fine-tune, Naive Bayes text classifier, postprocessing heuristics) are described but not available.
- Evaluation uses AP/AR rather than TEDS or GriTS, making direct comparison to most TSR literature difficult.
- The DOI in the paper (`https://doi.org/XXXXXXX.XXXXXXX`) is a placeholder; the actual ACM DOI was not available at time of arXiv submission.
- The paper relies on an in-house OCR service and an in-house statement summary extraction service. These dependencies are opaque and unavailable to external researchers.

## Reproducibility

### Models

- **TDC:** DETR with ResNet-50 backbone, implemented via Detectron2. Pre-trained weights: Table Transformer (PubTables-1M). Fine-tuned on BankTabNet TDC. No weights released.
- **TSR:** DETR with ResNet-50 backbone, pre-trained on PubTables-1M, fine-tuned on BankTabNet TSR with CIoU loss substitution. Query count $N = 125$. No weights released.
- **Text classifier:** Three Multinomial Naive Bayes models trained on caption/header text using count vectorization (vocabulary sizes: 840 caption words, 860 header words). Lightweight; reproducible from the paper's description if data were available.

### Algorithms

- **TDC training:** LR $1 \times 10^{-5}$; 100 epochs; batch size 16; $\lambda\_{ce} = 1$, $\lambda\_{l1} = 5$, $\lambda\_{giou} = 2$. Scale augmentation: shortest side 400-800 px. Optimizer not stated for TDC (Adam stated only for TSR).
- **TSR training:** LR $5 \times 10^{-5}$ with linear decay every 4 epochs; 100 epochs; batch size 2; $\lambda\_{ce} = 1$, $\lambda\_{l1} = 5$, $\lambda\_{ciou} = 2$. Adam optimizer. Scale augmentation: longest side 1100-1300 px. Inference: $1200 \times 1200$ px.
- Long-table split: tables with more than 20 rows are split into two sub-images horizontally. Merge happens in postprocessing.
- Postprocessing heuristics: NMS, gap filling, addition of missing rows/columns, rule-based row separation from OCR date column.
- Balance checksum: $\text{OpenBal} - \sum\_{\text{Debit}} + \sum\_{\text{Credit}} - \text{EndBal} = 0$.

### Data

- **BankTabNet TDC:** 11,607 page images (7,544 train / 1,741 val / 2,322 test). 10 table categories. PII-masked via in-house service. Not publicly available.
- **BankTabNet TSR:** 5,165 table images from 310 bank statements (9,724 train / 2,000 val / 2,200 test after augmentation). 5 object classes. Not publicly available.
- **Annotation quality:** Krippendorff's Alpha $\geq 0.955$ (TDC, $\text{IoU} > 0.5$), $\geq 0.994$ (TDC, $\text{IoU} > 0.9$); $\geq 0.99$ (TSR, $\text{IoU} > 0.5$), $\geq 0.98$ (TSR, $\text{IoU} > 0.9$).
- External datasets used for generalization tests: PubTables-1M (public, CDLA-Perm-2.0 annotations) and FinTabNet (public, CDLA-Permissive-1.0 annotations).

### Evaluation

- Primary metrics: $AP$, $AP\_{50}$, $AP\_{75}$, $AR$ (COCO-style object detection metrics). Text classifier: per-category F1.
- No TEDS, GriTS, or Adjacency F1 reported. Direct comparison to Im2Seq TSR literature is not straightforward.
- Baselines for TSR: TabStructNet, LGPMA, Table Transformer (all trained on BankTabNet TSR). Comparisons on external datasets: Table Transformer trained on the respective dataset.
- No error bars, significance tests, or multi-seed averaging reported.
- The end-to-end transaction extraction accuracy (checksum pass rate) is mentioned conceptually but not quantified in the results tables.

### Hardware

- Training: NVIDIA A100 Tensor Core GPU, 40 GB. Number of GPUs not stated.
- Inference: Apple M1 Max CPU (32 GB). End-to-end pipeline for a 20-page bank statement: approximately 1.91 minutes. Per-image TDC inference: 1.25 s (TabSniper-TDC) on this hardware.
- No GPU-hour estimates or energy consumption figures are reported.

## BibTeX

```bibtex
@inproceedings{trivedi2024tabsniper,
  title={TabSniper: Towards Accurate Table Detection \& Structure Recognition for Bank Statements},
  author={Trivedi, Abhishek and Mukherjee, Sourajit and Singh, Rajat Kumar and Agarwal, Vani and Ramakrishnan, Sriranjani and Bhatt, Himanshu Sharad},
  booktitle={8th International Conference on Data Science and Management of Data (12th ACM IKDD CODS and 30th COMAD)},
  year={2024},
  address={Jodhpur, India},
  publisher={ACM}
}
```