# Hybrid DLA: Enhanced Query Encoding and Hybrid Matching for Document Layout Analysis > A DINO-based detector augmented with backbone-aware query encoding and a two-phase hybrid matching strategy for document layout analysis. - Authors: Hunter Heidenreich - Published: 2026-03-14 - Paper: Preprint (arXiv) - Paper: Published (ICDAR 2024) - Source: Bevaya Labs, --- ## TL;DR This paper extends the DINO detection transformer for document layout analysis with two additions: (1) a query encoding mechanism that fuses RoI-aligned backbone features with decoder queries via cosine similarity, improving small-object detection, and (2) a hybrid matching strategy that trains with one-to-many matching in the first half of training and switches to one-to-one matching in the second half. The authors report 97.3% mAP on PubLayNet, 81.6% on DocLayNet, and 98.6% on PubTables. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$. The headline contribution is an improved detection transformer architecture and training strategy. The bulk of the paper describes the query encoding mechanism, the hybrid matching scheme, and ablations isolating their effects. **Secondary:** None. No new datasets, benchmarks, or artifacts are released. ## What is the motivation? Transformer-based detectors like DINO have shown strong performance on natural image detection, but the authors argue they underperform on document layout analysis tasks, particularly for small graphical objects (headers, footers, section titles). The core issue: decoder queries in DINO lack direct access to fine-grained backbone features, limiting their ability to detect small or visually subtle layout elements. Additionally, standard one-to-one Hungarian matching produces fewer positive training samples, slowing convergence. ## What is the novelty? ### Query Encoding The method extracts features from the ResNet-50 backbone using RoI-Align on bounding box proposals and passes them through an MLP to produce high-level query features: $$Q\_h = \text{MLP}(\text{RoIAlign}(F\_h, b\_j))$$ These are compared to decoder queries via cosine similarity after self-attention: $$Q\_e = \text{similarity}(Q'\_d, Q'\_h)$$ The enhanced query features are concatenated with the original decoder queries: $$Q\_t = \text{Concat}(Q\_d, Q\_e)$$ This combined representation is fed into the decoder: $$o = \text{Decoder}(Q\_t, E \mid A)$$ where $E$ is the encoder output and $A$ is the denoising attention mask. ### Hybrid Matching The training loss combines one-to-many and one-to-one assignment strategies. The one-to-many classification loss is: $$L\_{cls}^{1\text{-}m} = \sum\_{i=1}^{N\_{obj}} |\hat{g}\_i - p\_i| \cdot \text{BCE}(p\_i, \hat{g}\_i) + \sum\_{j=1}^{N\_{no}} p\_j \cdot \text{BCE}(p\_j, 0)$$ with regression: $$L\_{reg}^{1\text{-}m} = \sum\_{i=1}^{N\_{obj}} \hat{g}\_i \cdot L\_{GIoU}(bx\_i, \hat{bx}\_i) + \sum\_{i=1}^{N\_{obj}} \hat{g}\_i \cdot L\_{L1}(bx\_i, \hat{bx}\_i)$$ During training, the first half uses one-to-many matching (more positive samples, faster convergence), and the second half switches to one-to-one matching (eliminates duplicate predictions, enabling NMS-free inference). ## What experiments were performed? The method is evaluated on three benchmarks: - **PubLayNet** (360K pages, 5 classes): 97.3% mAP, improving over DINO baseline (95.5%) and the prior best (Zhong et al., 96.5%). - **DocLayNet** (80K pages, 11 classes): 81.6% mAP, improving over DINO (74.3%) and Zhong et al. (81.0%). Particularly strong on Page-footer (+37.1 over DINO), Page-header (+14.1), and Title (+3.5). - **PubTables** (table detection): 98.6% mAP (AP50: 99.8%, AP75: 99.1%). Baselines include Faster R-CNN, Mask R-CNN, YOLOv5, DINO, DiT-L, LayoutLMv3, VSR, and others. ### Ablations 1. **Query encoding** ($Q\_d + Q\_e$ vs. $Q\_d$ alone): +1.78 mAP on PubLayNet validation. 2. **Matching strategy**: One-to-many alone reaches 98.4 mAP but generates duplicates. The hybrid scheme (one-to-many then one-to-one) achieves 97.3 mAP while remaining NMS-free. 3. **Number of queries**: 300 queries is optimal. Performance degrades at 400 due to overfitting. ## What are the outcomes/conclusions? The two proposed modifications yield consistent improvements over the DINO baseline across all three benchmarks. The query encoding mechanism is particularly effective for small layout elements. The hybrid matching strategy provides a practical trade-off between detection quality and duplicate elimination without requiring NMS at inference. The gains over DINO are substantial on DocLayNet (+7.3 mAP) where documents are structurally diverse, but more modest on PubLayNet (+1.8 mAP) where layouts are relatively uniform. No code or model weights are released, limiting independent verification. ## Reproducibility ### Models - **Backbone:** ResNet-50 pre-trained on ImageNet, with multi-scale feature maps (1/4, 1/8, 1/16, 1/32, 1/64), each projected to 256 channels via 1$\times$1 convolutions - **Architecture:** DINO transformer with deformable attention, augmented with the proposed query encoding module - **Parameter count:** Not reported - **Queries:** 300 learnable queries (optimal per ablation) - **Initialization:** DINO weight initialization source (pretrained or from scratch) not specified - **Weights:** Not released ### Algorithms - **Optimizer:** AdamW - **Batch size:** 16 - **Epochs:** 12 (PubLayNet, PubTables), 24 (DocLayNet) - **Base learning rate:** Not reported - **Learning rate schedule:** Reduced by factor of 10 at a later stage (exact milestone not specified) - **Training tricks:** Warmup, gradient clipping, and mixed precision not mentioned - **Loss functions:** BCE (classification) + GIoU + L1 (regression), combined in one-to-many and one-to-one branches. Loss weighting coefficients not reported. - **Multi-scale training:** Images resized to various lengths with a max size limit (exact dimensions not specified) - **Other augmentation:** Not described beyond multi-scale resizing - **Test-time:** Shorter side resized to 640 - **Matching switch:** One-to-many for first half of training epochs, one-to-one for second half ### Data - **PubLayNet:** 360K pages, 5 classes (Text, Title, List, Table, Figure). Standard train/val/test splits. CDLA-Permissive-1.0 (annotations); underlying PDFs non-commercial only. - **DocLayNet:** 80K pages, 11 classes. Standard splits. CDLA-Permissive-1.0 (annotations); underlying doc licenses vary by domain. - **PubTables:** Table detection benchmark. Standard splits. CDLA-Permissive-1.0. - All three are established benchmarks with published annotation protocols. - No additional or proprietary training data used. ### Evaluation - **Metric:** COCO-style mAP at IoU thresholds 0.50 to 0.95 (step 0.05), plus AP@50 and AP@75 - **Baselines:** Comparison against CNN-based (Faster R-CNN, Mask R-CNN, YOLOv5) and transformer-based (DINO, DiT-L, LayoutLMv3) methods - **Baseline provenance:** DocLayNet DINO numbers are sourced from Zhong et al. [62], not from the original DINO paper or the authors' own runs. PubLayNet DINO numbers cite the original DINO paper [19]. Provenance of other baselines varies by table. - **Statistical rigor:** No error bars, significance tests, or multi-run statistics reported. Number of runs and seed sensitivity not mentioned. - **Author-acknowledged limitations:** The paper's conclusion does not explicitly discuss limitations. It focuses on summarizing contributions without qualifying scope or failure cases. ### Hardware - **GPU:** NVIDIA "RTXA600" GPUs (plural; likely RTX A6000). Exact GPU count not specified. - **Training time:** Not reported - **Inference speed:** Not reported - **Memory requirements:** Not reported - **Cost estimates:** Not reported - **Deployment considerations:** Not discussed. Given the DINO-based architecture with ResNet-50, CPU-only inference is likely impractical for real-time use. ## BibTeX ```bibtex @inproceedings{shehzadi2024hybrid, title={A Hybrid Approach for Document Layout Analysis in Document Images}, author={Shehzadi, Tahira and Stricker, Didier and Afzal, Muhammad Zeshan}, booktitle={International Conference on Document Analysis and Recognition (ICDAR)}, pages={20--37}, year={2024}, organization={Springer} } ```