# YALTAi: Object Detection for Layout Analysis in Kraken > Research note on YALTAi, which replaces pixel-based region segmentation with YOLOv5 object detection for document layout analysis in the Kraken OCR engine. - Authors: Hunter Heidenreich - Published: 2026-03-14 - Paper: Preprint (arXiv) - Paper: Published (JDMDH 2023) - Code: YALTAi - Dataset: YALTAI-MSS-EPB - Dataset: YALTAi-Tables - Source: Bevaya Labs, --- ## TL;DR YALTAi proposes replacing Kraken's pixel-classification-based region segmentation with YOLOv5 object detection using isothetic bounding boxes. On small historical document datasets (around 1,000 images), YOLOv5 outperforms Kraken's built-in segmenter by roughly $7\times$ on mAP, while also being more GPU-efficient. The authors release two datasets and a tool that injects YOLOv5 into Kraken's segmentation pipeline. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$. The headline contribution is a methodological shift: reframing layout analysis from pixel classification to object detection, and demonstrating that this yields large accuracy gains on small datasets when integrated into an existing OCR pipeline (Kraken). **Secondary:** $\Psi\_{\text{Resource}}$. The paper releases two new annotated datasets (YALTAI-MSS-EPB for manuscripts/early printed books and YALTAi-Tables for tabular documents) and an open-source tool (YALTAi) that plugs YOLOv5 into Kraken. ## What is the motivation? Layout analysis (identifying and classifying regions on a page) is the first step in OCR/HTR pipelines. The open-source Kraken engine, widely used in digital humanities, performs pixel-level classification for region segmentation. This approach has two known weaknesses: 1. **Poor performance on small datasets.** Training Kraken's segmenter on roughly 1,000 images or fewer yields unreliable results, which is a common scenario for historical document projects. 2. **Region merging.** Because Kraken operates at the pixel level, it tends to merge adjacent regions of the same class (e.g., neighboring columns), requiring workarounds like "ColOdd/ColEven" naming hacks. Meanwhile, ICDAR competitions shifted from polygon-based evaluation (2011) to pixel-based evaluation (2017+), which the authors argue biased the field away from object detection approaches that might be better suited for downstream text extraction. Three specific digital humanities research projects are cited as being blocked by Kraken's segmentation limitations on small datasets. ## What is the novelty? The core proposal is to treat document layout analysis as an **object detection** task rather than a pixel classification task. Concretely: - Ground truth polygons are simplified to isothetic (axis-aligned) bounding boxes by taking the min/max X and Y coordinates of each polygon: $$\text{bbox} = \big(\min(x\_i),\; \min(y\_i),\; \max(x\_i),\; \max(y\_i)\big)$$ - YOLOv5 is trained on these bounding boxes to detect and classify document regions. - At inference time, YALTAi runs YOLOv5 first for region detection, then injects those regions into Kraken's pipeline before Kraken's line segmenter dispatches text lines into the detected zones. This is not a novel architecture. The contribution is the reframing of the task and the integration: reusing off-the-shelf YOLOv5 within Kraken's ecosystem, which lets users benefit from YOLO's strong few-shot learning and fast inference without abandoning Kraken's line segmentation and OCR capabilities. The datasets follow the **Segmonto** ontology for region classification labels, providing a standardized vocabulary (MainZone, MarginTextZone, GraphicZone, DropCapitalZone, etc.). ## What experiments were performed? ### Datasets Two datasets were assembled for training and evaluation: 1. **YALTAI-MSS-EPB** (Manuscripts and Early Printed Books): 1,110 pages from the 9th to 17th century, drawn from five existing CREMMA/Gallicorpora datasets plus 593 new annotated pages. 12 zone types following Segmonto. Split: 854 train / 154 dev / 139 test (test is out-of-source). 2. **YALTAi-Tables**: Tabular documents from the Lectaurep dataset for train/dev, with an entirely out-of-domain test set spanning the 17th to early 20th century. 4 zone types (Col, Header, Marginal, Text). ### Models - **Kraken** segmenter trained at two input resolutions (640 and 1200 pixels), 50 epochs, best model selected by mAP on dev set. - **YOLOv5x** (extra-large, 175 MiB) and **YOLOv5n** (nano, 3.9 MiB), both trained for 50 epochs on the same bounding-box data. ### Metrics Evaluation uses **mAP@0.5** (mean average precision at IoU threshold 0.5) and per-class **AP**, computed with `mean-average-precision==2021.4.26.0`. Kraken's polygon outputs were converted to bounding boxes for fair comparison. ### Key Results **YALTAI-MSS-EPB:** | Model | mAP | MainZone AP | DropCapital AP | | :--- | :--- | :--- | :--- | | Kraken | 6.98 | 43.5 | 23.3 | | YOLOv5n | 34.63 | 87.0 | 54.6 | | YOLOv5x | 47.75 | 91.7 | 69.2 | YOLOv5x achieves roughly $7\times$ the mAP of Kraken. On MainZone (the most important class for text extraction), YOLOv5x more than doubles Kraken's AP. Kraken scores 0% AP on several minority classes (MarginText, Numbering, QuireMarks, RunningTitle), while YOLOv5x achieves 45-76% AP on these. **YALTAi-Tables:** | Model | mAP | Col AP | Header AP | | :--- | :--- | :--- | :--- | | Kraken | 0.09 | 0.1 | 0.1 | | YOLOv5x | 4.77 | 12.9 | 1.4 | The tabular results are low overall because the test set is entirely out-of-domain, but they highlight Kraken's inability to separate adjacent same-class columns (a known pixel-classification failure mode) versus YOLOv5's ability to detect individual column instances. ### Efficiency All training on a single NVIDIA RTX 2080 Ti (11 GB). YOLOv5 generally uses less peak GPU power and trains in roughly half the time compared to Kraken (batch size 2 vs. 1). YOLOv5x inference is 0.025s per image; YOLOv5n is 0.004s per image. ## What are the outcomes/conclusions? The results suggest that object detection (specifically YOLOv5) is a practical replacement for pixel-based segmentation in Kraken's pipeline, especially for small dataset scenarios common in historical document digitization projects. **Limitations acknowledged by the authors:** - Isothetic bounding boxes cannot represent complex polygon shapes (P-shaped, C-shaped, or rotated regions). This limits applicability to documents with relatively rectangular, unskewed regions. - Bounding boxes can overlap, losing the precision of non-overlapping polygon segmentation. - The tabular test set is deliberately adversarial (fully out-of-domain), making those numbers hard to interpret as general performance indicators. **Limitations not acknowledged:** - The comparison is somewhat asymmetric: YOLOv5 benefits from ImageNet pre-training (via COCO-pretrained weights), while Kraken's segmenter does not leverage comparable pre-training. The performance gap may partly reflect transfer learning advantages rather than a fundamental architectural superiority. - mAP@0.5 is a relatively lenient threshold. Results at higher IoU thresholds (e.g., mAP@0.75 or mAP@[0.5:0.95]) are not reported, which matters because bounding boxes inherently have looser fits than polygons. - No downstream OCR/HTR accuracy is reported. The paper assumes better region detection leads to better text extraction, but this is not validated end-to-end. - No ablation over training set size is provided, despite the paper's central claim being about small-data regimes. It remains unclear at what dataset size the advantage over Kraken's segmenter would diminish or disappear. ## Reproducibility ### Models - No novel architecture; uses off-the-shelf YOLOv5 (commit 29d79a6, July 2, 2022) and Kraken v4.1.2. - YOLOv5x (~86.7M params, 175 MiB) and YOLOv5n (~1.9M params, 3.9 MiB), both initialized from COCO-pretrained weights. Kraken segmenter model is 5 MiB (no comparable pre-training). - No trained weights are released. The YALTAi package provides the integration pipeline but does not bundle domain-specific checkpoints. ### Algorithms - 50 epochs for all models. - YOLOv5: batch size 2, input size 640px, best model selected by YOLOv5's internal metric. The paper does not specify optimizer, learning rate, or augmentation settings; presumably YOLOv5 defaults apply (SGD with momentum 0.937, initial LR 0.01, cosine annealing, mosaic/mixup augmentation). The default augmentation pipeline is notable given the paper's small-data thesis. - Kraken: batch size 1, input resize 640 and 1200. Best model selected by mAP on dev set (computed externally since Kraken does not support this natively). - Loss functions are not described. YOLOv5 uses a composite loss (objectness + classification + CIoU box regression); Kraken uses pixel-level cross-entropy. Neither is discussed in the paper. ### Data - Both datasets are publicly available on Zenodo under CC-BY-4.0. - YALTAI-MSS-EPB: 1,110 pages, fixed train/dev/test splits included. Assembled from five source datasets: CREMMA Medieval, CREMMA Medieval LAT, Eutyches, and two Gallicorpora sets (HTR-MSS-15e-Siecle, HTR-imprime-gothique-16e-siecle), plus 593 original pages annotated for this paper. - YALTAi-Tables: derived from Lectaurep Repertoires with an entirely out-of-domain test set. - Annotation follows Segmonto ontology. Some annotations (the 593 original pages) were produced with early model predictions and corrected by hand on Roboflow. No inter-annotator agreement statistics are reported, and this semi-automated workflow may introduce annotation bias toward the model's error patterns. ### Evaluation - mAP@0.5 using `mean-average-precision==2021.4.26.0` Python package. The AP interpolation method (11-point vs. all-point) is not specified. - IoU threshold 0.5 only; no results at stricter thresholds. - Kraken polygons converted to bounding boxes for fair comparison. The paper reports Kraken results at input resize 1200 (the better setting); results at 640 are not shown in the main tables. - No error bars, multiple runs, or significance tests reported. ### Hardware - Single NVIDIA RTX 2080 Ti (11 GB VRAM, 250W TDP). - Ubuntu with CUDA 10.2, drivers 440.118.02. - Training times: Kraken Segmonto ~6.5 hours; YOLOv5x Segmonto ~3 hours; YOLOv5n ~3.2 hours. - YOLOv5x inference: 0.025s/image. YOLOv5n inference: 0.004s/image. ## BibTeX ```bibtex @article{clerice2023yaltai, title={You Actually Look Twice At it (YALTAi): using an object detection approach instead of region segmentation within the Kraken engine}, author={Cl{\'e}rice, Thibault}, journal={Journal of Data Mining and Digital Humanities}, year={2023}, doi={10.46298/jdmdh.9806} } ```