# SemiTabDETR: Semi-Supervised Table Detection with Dual Query Assignment > SemiTabDETR extends Deformable DETR with a dual one-to-one and one-to-many query assignment strategy in a teacher-student semi-supervised framework for table detection, reducing labeled data requirements while improving detection accuracy. - Authors: Hunter Heidenreich - Published: 2024-05-08 - Updated: 2026-03-25 - Paper: Preprint (arXiv) - Paper: Published (IJDAR 2024) - Source: Bevaya Labs, --- ## TL;DR SemiTabDETR is a teacher-student semi-supervised table detector built on Deformable DETR. The key contribution is a dual query assignment strategy: a one-to-many branch generates high-quality pseudo-labels during early training, while a one-to-one branch eliminates duplicate predictions at inference without requiring NMS. With only 30% labeled data, the approach reports mAP 95.7% on TableBank-Word and 97.9% on PubLayNet. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$: The paper proposes a new semi-supervised detection framework. The headline contribution is the dual query assignment mechanism combined with a teacher-student training loop. Ablations and comparison tables occupy the majority of the experimental sections. **Secondary:** None. No new datasets, benchmarks, or theoretical contributions are introduced. ## What is the motivation? Supervised table detection methods depend on large annotated datasets. Annotating document pages for table bounding boxes is expensive, domain-specific, and does not transfer well across publication types. Semi-supervised approaches address this by generating pseudo-labels for unlabeled images, but existing semi-supervised table detectors had two problems: 1. CNN-based methods (Soft Teacher, STAC) require anchor generation and NMS, which adds complexity and limits end-to-end training. 2. Prior transformer-based semi-supervised detectors (Shehzadi et al., ICDAR 2023) used a one-to-one bipartite matching strategy. When pseudo-labels are noisy, one-to-one matching forces each query to align tightly with a potentially incorrect label, propagating errors into subsequent training iterations. The authors observe that one-to-many matching can generate multiple positive proposals per pseudo-label, improving robustness to label noise during the early stages of training, but introduces duplicate detections that require NMS at inference. The goal is to combine the noise-robustness of one-to-many matching with the NMS-free property of one-to-one matching. ## What is the novelty? SemiTabDETR wraps Deformable DETR in a teacher-student semi-supervised framework with two parallel query sets in the decoder: - A primary set $Q = \\{q\_1, \ldots, q\_N\\}$ of $N = 30$ queries undergoes one-to-one bipartite matching via the Hungarian algorithm. - A secondary set $\hat{Q} = \\{\hat{q}\_1, \ldots, \hat{q}\_T\\}$ of $T = 400$ queries undergoes one-to-many matching, where ground truth is replicated $K = 6$ times to form augmented targets $\hat{y}\_g = \\{y\_g^1, \ldots, y\_g^K\\}$. The per-decoder-layer losses for each strategy are: $$\mathcal{L}\_{o2o} = \sum\_{l=1}^{L} \mathcal{L}\_H(y\_p, y\_g)$$ $$\mathcal{L}\_{o2m} = \sum\_{l=1}^{L} \mathcal{L}\_H(\hat{y}\_p, \hat{y}\_g)$$ where $\mathcal{L}\_H$ is the Hungarian matching cost combining classification and regression terms. Training proceeds in two stages. First, the one-to-many strategy runs for all iterations to produce high-quality pseudo-labels and accelerate convergence. Then the one-to-one strategy takes over to eliminate duplicates, so no NMS step is needed at inference. Both strategies use separate classification and regression losses: $$\mathcal{L}\_{o2m} = \mathcal{L}\_{o2m}^{cls} + \mathcal{L}\_{o2m}^{reg}$$ $$\mathcal{L}\_{o2o} = \mathcal{L}\_{o2o}^{cls} + \mathcal{L}\_{o2o}^{reg}$$ The overall semi-supervised loss weights a supervised term on labeled images and an unsupervised term on unlabeled images: $$\mathcal{L} = \mathcal{L}\_s + \omega \mathcal{L}\_u$$ The teacher module receives weakly augmented unlabeled images and generates pseudo-labels filtered by a confidence threshold of 0.7. The student receives strongly augmented versions of both labeled and unlabeled images. Teacher weights are updated via Exponential Moving Average (EMA) from the student. The one-to-many pseudo-label selection is formulated as: $$p\_{o2m} = \left\\{ \underset{\sigma\_i \in \chi}{\text{argmin}} \sum\_{i=1}^{N} \mathcal{L}\_H\!\left(y\_i^T, y\_{\sigma\_i(k)}^S\right) \right\\}\_{i=1}^{|y^T|}$$ where $\chi$ assigns a subset of student proposals to each teacher pseudo-label $y\_i^T$, and $y\_{\sigma\_i(k)}^S$ is the matched student prediction. ## What experiments were performed? **Datasets:** PubLayNet (table class only), TableBank (Word, LaTeX, and Both splits), and ICDAR-19 cTDaR Modern Track A. Experiments use 10%, 30%, and 50% labeled data with the remainder treated as unlabeled. **Metrics:** mAP (IoU 0.50:0.95), AP$^{50}$, AP$^{75}$, and $AR\_L$ (average recall for large objects). ICDAR-19 is also evaluated on Recall, Precision, and F1 at IoU thresholds of 0.8 and 0.9, following the competition protocol. **Baselines:** - Supervised: Faster R-CNN (Ren et al.) and Deformable DETR (Zhu et al.) - Semi-supervised CNN-based: STAC (Sohn et al.), Unbiased Teacher (Liu et al.), Tang et al., Soft Teacher (Xu et al.) - Semi-supervised transformer-based: Shehzadi et al. (ICDAR 2023, the direct predecessor) - ICDAR-19 supervised: TableRadar, NLPR-PAL, Lenovo Ocean, CDeC-Net, HybridTabNet **Ablations:** Number of one-to-many queries (200, 400, 600 evaluated; 400 is best), confidence threshold (0.5, 0.6, 0.7, 0.8; 0.7 is best), and comparison of o2o-only, o2m-only, and combined strategies with/without NMS and their training times. **Implementation:** ResNet-50 backbone pretrained on ImageNet, 150 training epochs, learning rate decayed by 0.1 at epoch 140, input size 600 pixels (800 for comparisons), strong augmentation (flips, resize, patch removal, cropping, grayscale, Gaussian blur) for the student, weak augmentation (horizontal flip) for the teacher. ## What are the outcomes/conclusions? **TableBank (Table 2):** With 30% labeled data on TableBank-Both, SemiTabDETR achieves mAP 93.3, compared to 86.8 for the previous semi-supervised method (Shehzadi et al.) and 82.6 for the supervised Deformable DETR baseline. With only 10% labeled data, the gain over the prior semi-supervised method is 5.2 mAP points. | Dataset | Labels | mAP | AP$^{50}$ | |---|---|---|---| | TableBank-Word | 30% | 95.7 | 96.9 | | TableBank-LaTeX | 30% | 88.0 | 94.3 | | TableBank-Both | 30% | 93.3 | 96.8 | **PubLayNet (Table 4):** With 30% labeled data, mAP 97.9 vs. 90.3 for the prior semi-supervised method, a 7.6 point improvement. With 50% labels, the model's AP$^{50}$ (98.9) matches or exceeds all listed supervised baselines. **ICDAR-19 (Table 7):** With 50% labeled data, the method achieves recall 95.0 at IoU 0.8 but precision of only 61.4, yielding F1 74.5. This falls well below the best-reported supervised results on this benchmark (TableRadar F1 94.5, CDeC-Net F1 94.4). The authors highlight the recall advantage but do not address the precision gap. The ICDAR-19 results for this semi-supervised approach should be read with caution for applications where false positives are costly. **Ablation (Table 12):** Using o2o-only takes 8.74 hours training time; o2m-only requires NMS and 11.33 hours; the combined approach takes 10.99 hours without NMS. The paper notes that Table 12 is run without augmented GT in the one-to-many branch, which is why the margin over o2o-only is small: the combined strategy achieves mAP 96.1 vs. 96.0 for o2o-only on PubLayNet with 10% labels. The primary advantage of the combined approach is the improved pseudo-label quality during early training rather than a large absolute mAP gain in this controlled ablation setting. **Limitations:** No code or model weights were released. The ICDAR-19 results show high recall but poor precision compared to supervised methods, which is not addressed. Hardware details for training are not reported. Error bars and significance tests are absent throughout. ## Reproducibility ### Models - **Architecture:** Deformable DETR with two parallel query sets (30 one-to-one, 400 one-to-many) fed to the same transformer decoder. ResNet-50 backbone. The number of decoder layers is not explicitly confirmed; Deformable DETR's default of 6 is assumed. - **Parameter count:** Not reported. - **Teacher/student relationship:** Both use the same Deformable DETR architecture; the teacher's weights are an EMA of the student's weights and are not separately parameterized. - **Initialization:** ResNet-50 pretrained on ImageNet. - **Weights:** Not released. No code repository linked. ### Algorithms - **Framework:** Not specified; Deformable DETR implementations exist in PyTorch. - **Optimizer:** Not stated. - **Initial learning rate:** Not stated. Only the decay schedule is reported: multiplied by 0.1 at epoch 140 of 150 total epochs. - **Batch size:** Not reported. - **EMA update:** Used for teacher module; decay factor not reported. - **Pseudo-label threshold:** 0.7 (ablated in Table 10). - **Loss weights:** $\alpha\_1 = 2$ (classification), $\alpha\_2 = 5$ (regression); unsupervised weight $\omega$ not reported numerically. - **Augmentation:** Strong (flips, resize, patch removal, crop, grayscale, Gaussian blur); weak (horizontal flip only). - **Query counts:** 30 for one-to-one, 400 for one-to-many; ground truth replicated $K = 6$ times for augmented matching. ### Data - **Training:** PubLayNet, TableBank (Word, LaTeX, Both), ICDAR-19 cTDaR Modern Track A. Evaluated at 10%, 30%, and 50% labeled splits; remainder treated as unlabeled. - **Preprocessing:** Input resized to 600 pixels (longer side); 800 pixels for direct comparison experiments. - **Public availability:** All three datasets are publicly available (PubLayNet: CDLA-Perm; TableBank: Apache-2.0 code, research-only data; ICDAR-19: competition data, available from the challenge organizers). - **Split construction:** The labeled/unlabeled partition is created by randomly sampling the specified percentage from the training set. How random seeds or stratification were handled is not described. ### Evaluation - **Primary metric:** mAP following COCO protocol (IoU 0.50:0.95) for TableBank and PubLayNet; Recall/Precision/F1 at IoU thresholds 0.8 and 0.9 for ICDAR-19, following the competition protocol. - **Baselines fairness:** All semi-supervised methods use the same labeled/unlabeled splits. Supervised baselines are trained on the same labeled subset only. - **Statistical rigor:** No error bars, confidence intervals, or multi-seed runs reported. Results are single-run point estimates throughout. - **Known limitation:** ICDAR-19 precision (61.4 at IoU 0.8 with 50% labels) is substantially below the best-reported supervised results (~95%), despite high recall (95.0). The paper does not analyze this precision-recall imbalance. - **Scope restriction:** All experiments use a ResNet-50 backbone only; generalization to stronger backbones (ResNet-101, ViT-based) is untested. ### Hardware - **Training hardware:** GPU type and count not reported. - **Wall-clock training time:** Reported in Table 12 for the PubLayNet ablation (10% labeled data): o2o-only 8.74 hours, o2m-only 11.33 hours, combined 10.99 hours. These are wall-clock times on unspecified hardware and should not be taken as reproducible benchmarks. - **Inference speed:** The combined strategy runs at 4.36 FPS (same as o2o-only and o2m-only variants, per Table 12), suggesting inference cost is dominated by the backbone and decoder rather than the matching strategy. - **Memory requirements:** Not reported. - **Deployment feasibility:** No discussion of inference-time requirements or deployment constraints. ## BibTeX ```bibtex @article{ehsan2024semitabdetr, title = {End-to-end semi-supervised approach with modulated object queries for table detection in documents}, author = {Ehsan, Iqraa and Shehzadi, Tahira and Stricker, Didier and Afzal, Muhammad Zeshan}, journal = {International Journal on Document Analysis and Recognition (IJDAR)}, year = {2024}, doi = {10.1007/s10032-024-00471-0} } ```