# TextBite: A Historical Czech Document Dataset for Logical Page Segmentation > Research notes on TextBite, a dataset of 8,449 historical Czech document pages with 78,863 annotated segments for logical page segmentation, evaluated using a pixel-clustering metric based on the Rand index. - Authors: Hunter Heidenreich - Published: 2026-03-10 - Updated: 2026-03-12 - Paper: Preprint (arXiv) - Paper: Published (ICDAR Workshops 2025) - Dataset: TextBite Dataset (Zenodo) - Code: TextBite (GitHub) - Source: Bevaya Labs, --- ## TL;DR TextBite is a dataset of 8,449 manually annotated historical Czech document pages (18th-20th century) for **logical page segmentation**, the task of grouping document content into semantically coherent units. The dataset covers printed newspapers, dictionaries, and handwritten records, with 78,863 annotated segments. The authors propose evaluating segmentation as a **pixel clustering** problem using the Rand index over foreground text pixels, removing dependence on OCR accuracy or precise bounding-box geometry. Best baseline: YOLOv11 detection + GNN merging achieves 92.5% Rand index. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Resource}}$. The headline contribution is the TextBite dataset itself: 8,449 pages with 78,863 annotated logical segments, publicly released on Zenodo with MIT-licensed code. **Secondary:** $\Psi\_{\text{Evaluation}}$ (the pixel-clustering evaluation framework and Rand index metric that decouples evaluation from OCR and bounding-box geometry), $\Psi\_{\text{Method}}$ (three baseline methods combining detection with relation prediction). ## What is the motivation? Logical page segmentation divides a document page into semantically coherent units, enabling finer text embeddings and better retrieval. Prior approaches fall into two camps, each with limitations: 1. **Text-based segmentation** (e.g., TextTiling): operates on OCR output, so errors in recognition and reading order propagate directly into segmentation quality. 2. **Object-detection-based methods** (e.g., DLAFormer): detect bounding boxes for paragraphs and group them, but evaluation penalizes minor geometric deviations that have no impact on the actual text grouping. No existing benchmark evaluates logical segmentation in a way that is both OCR-independent and insensitive to irrelevant geometric variation. TextBite fills this gap with a dataset and evaluation framework designed for historical documents, which present additional challenges: mixed printed/handwritten content, multi-column newspaper layouts, ornamental elements, and inconsistent formatting across centuries. ## What is the novelty? ### Pixel-Clustering Evaluation Rather than evaluating segmentation through text-level metrics or detection-based mAP, TextBite treats logical segments as **clusters of foreground text pixels**. The pipeline: 1. Intersect human-annotated bounding boxes with OCR-detected textlines (from PERO-OCR) 2. Apply adaptive thresholding (301$\times$301 pixel patches) to isolate foreground ink pixels 3. Evaluate using the **Rand index** over foreground pixel pairs only The Rand index measures agreement between predicted and ground-truth clusterings over all pixel pairs. For each pair of foreground pixels, the metric checks whether the two assignments agree (both same-segment or both different-segment): $$\text{RI} = \frac{\text{number of agreeing pixel pairs}}{\text{total number of pixel pairs}}$$ Values range from 0 (no agreement) to 1 (perfect agreement). In practice, when roughly 10% of foreground pixels are misassigned, the Rand index is typically around 0.83. Background pixels are excluded entirely. This means: - Systems are not penalized for bounding-box shape differences that do not affect text grouping - OCR quality does not affect the evaluation metric - Page metadata (headers, footers, edition info) can be freely marked as additional segments without penalty ### The TextBite Dataset - **8,449 pages** (7,346 printed + 1,103 handwritten) - **78,863 annotated segments** of logically coherent text - Historical Czech documents spanning the 18th to 20th centuries - Domains: newspapers, periodicals, books, dictionaries, handwritten school/organizational records - **Annotation categories**: Text region (80,458), Title (48,788), Page number (6,213) - **Relations**: directed links between bounding boxes encoding reading order within segments - **Format**: extended COCO with a `relations` key; OCR provided in PAGE-XML and ALTO formats - **Test split**: 964 pages (779 printed + 185 handwritten) Annotations were created using Label Studio by a team of students, librarians, and researchers. The test portion was manually verified by researchers; the development portion was reviewed by students. ### Baseline Methods Three approaches, all starting from YOLOv11 detection: 1. **YOLOv11 alone**: each detected bounding box is an independent segment 2. **YOLOv11 + GNN**: a graph neural network performs binary edge classification to merge detected regions; uses 29 geometric node features and 12 edge features (including CZERT text embeddings) 3. **YOLOv11 + Transformer**: relation prediction via exclusive relation scoring with optional image backbone (ResNet-18/50). Input queries are detected bounding boxes represented by 2D sinusoidal positional embeddings. Relation scores between output feature vectors $o\_i$ and $o\_j$ are computed as: $$s\_{i,j} = FC\_a^D(o\_i) \cdot FC\_b^D(o\_j), \quad i \in \\{1..N\\}, \; j \in \\{1..N\\}$$ where $FC^D$ is a fully connected layer with dimensionality $D$. The highest-scoring query $q\_j$ is chosen as the relation target for query $q\_i$, forming chains that are grouped into final segments. ## What experiments were performed? ### Detection Stage YOLOv11 was tested in three sizes (nano, small, medium) at five resolutions (640-1400 pixels). The small variant at 1200px resolution was selected (mAP50: 92.0, mAP50-95: 72.2). Training used batch size 16, learning rate $2 \times 10^{-4}$, Adam optimizer, with early stopping after 20 epochs of no validation improvement. ### Merging Stage | Method | Rand Index | 95% CI | | :--- | :--- | :--- | | YOLOv11 (detection only) | 83.9 | [82.6, 85.1] | | YOLOv11 + Transformer | 86.9 | [85.9, 88.0] | | **YOLOv11 + GNN** | **92.5** | **[91.6, 93.3]** | ### Transformer Ablations Adding image features (via ResNet cross-attention) improved the transformer on validation, with the no-image baseline at 86.5 and image-augmented variants ranging from 87.5 to 88.9 Rand index. Freezing the image backbone had no negative effect, and ResNet-18 performed comparably to ResNet-50. The authors selected the frozen ResNet-18 variant (88.5 validation Rand index) as the final model. ### Key Findings - Pure detection treats each box as independent, missing multi-box segments (columns, title-body associations) - The GNN's geometric + text features (via CZERT embeddings) provide strong merging signals - The transformer underperforms the GNN despite image features, suggesting that explicit pairwise edge classification is more effective than sequence-based relation prediction for this task - Confidence intervals were computed via bootstrapping ## What are the outcomes/conclusions? 1. **Pixel-clustering evaluation works.** By focusing on foreground text pixels and using the Rand index, evaluation becomes independent of OCR quality and robust to inconsequential geometric variation. 2. **Two-stage detection + merging is effective.** The GNN-based merging approach substantially improves over detection-only (83.9 to 92.5 Rand index), demonstrating that relation prediction is critical for logical segmentation. 3. **Historical documents remain challenging.** The dataset includes newspapers with complex multi-column layouts, dictionaries with dense entry structures, and handwritten records with loose formatting. 4. **The dataset supports multiple tasks.** Beyond logical segmentation, TextBite can be used for reading order prediction, title detection and association, and general layout analysis. Limitations: - Czech-only (with occasional German/French traces), limiting cross-lingual generalization claims - Primarily sourced from Czech libraries, so historical printing and layout conventions may not transfer to other traditions - The pixel-level ground truth is derived semi-automatically (OCR textline intersection + thresholding), not manually drawn at the pixel level - No comparison to existing segmentation methods beyond the authors' baselines ## Reproducibility ### Models - **Detection**: YOLOv11s at 1200px resolution, implemented via Ultralytics - **GNN**: Residual Gated Graph Convolution (3 layers, 512 dimensions), with Z-score normalized features - **Transformer**: 6-layer encoder, 128 hidden dim, 2 attention heads, optional frozen ResNet-18 image backbone - **Checkpoints**: baseline model weights available on Zenodo (77.3 MB) - **Code**: evaluation scripts and pre-trained weights publicly available at [GitHub](https://github.com/DCGM/textbite-dataset); full training pipelines for baselines are not included ### Algorithms - **YOLOv11**: batch size 16, LR $2 \times 10^{-4}$, Adam, early stopping (20 epochs patience) - **GNN**: batch size 16, LR $1 \times 10^{-3}$, AdamW, Residual Gated Graph Convolution operator, cosine similarity threshold for edge classification - **Transformer**: batch size 4, LR $2 \times 10^{-4}$, cross-entropy loss on relation scores, inputs padded to 80 tokens - **Text features**: CZERT (Czech BERT) masked language model for text embeddings in GNN edge features - **OCR**: PERO-OCR for textline detection and transcription (provided with dataset) ### Data - **Source**: historical Czech documents from multiple Czech libraries - **Size**: 8,449 pages total; 6,400 train / 1,000 validation / 964 test (185 handwritten in test) - **Annotation**: Label Studio; 10+ annotators (students, librarians, researchers); test set verified by researchers - **Format**: extended COCO JSON (bounding boxes + relations), PAGE-XML, ALTO - **License**: MIT (code), CC-BY-4.0 (dataset) - **Download**: Zenodo (11.7 GB dataset + 218.3 MB test labels + 77.3 MB baseline models) - **Language**: Czech (primary), with occasional German and French ### Evaluation - **Metric**: Rand index over foreground text pixels - **Ground truth construction**: annotated bounding boxes intersected with PERO-OCR textlines, then adaptive thresholding (301$\times$301 patches) - **Confidence intervals**: 95% CI via bootstrapping - **Baselines**: three methods (YOLOv11, YOLOv11+GNN, YOLOv11+Transformer) - **Limitations acknowledged**: semi-automatic pixel ground truth, Czech-only, no external method comparisons - **Statistical rigor**: bootstrapped confidence intervals reported for all results ### Hardware - Not explicitly reported in the paper - YOLOv11s is lightweight; GNN and transformer baselines use modest architectures, likely trainable on a single GPU ## BibTeX ```bibtex @inproceedings{kostelnik2025textbite, title={TextBite: A Historical Czech Document Dataset for Logical Page Segmentation}, author={Kosteln{\'\i}k, Martin and Bene{\v{s}}, Karel and Hradi{\v{s}}, Michal}, booktitle={International Conference on Document Analysis and Recognition Workshops (ICDAR Workshops)}, year={2025} } ```