# AnnoPage Dataset: Fine-Grained Non-Textual Element Detection in Historical Documents > We analyze the AnnoPage Dataset, a collection of 7,550 historical document pages annotated with 25 categories of non-textual elements by expert librarians, with YOLO and DETR baselines. - Authors: Hunter Heidenreich - Published: 2026-03-12 - Updated: 2026-03-15 - Paper: Preprint (arXiv) - Dataset: AnnoPage Dataset (Zenodo) - Source: Bevaya Labs, --- ## TL;DR The AnnoPage Dataset provides 7,550 pages from mostly historical documents (predominantly Czech and German, spanning 1485 to the present) annotated with axis-aligned bounding boxes across 25 fine-grained categories of non-textual elements. Annotations were created by expert librarians following the Czech Methodology of image document processing. Baseline experiments with YOLO11 and DETR models reach up to 0.658 mAP@50, suggesting the task remains challenging. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Resource}}$ The headline contribution is the dataset itself: its curation pipeline, annotation methodology, category taxonomy, and public release on Zenodo. The majority of the paper describes data acquisition, the 25-category schema, annotation iterations, dataset statistics, and limitations. Baseline model experiments are secondary and serve mainly to establish reference performance. **Secondary:** $\Psi\_{\text{Evaluation}}$ The paper defines a standard evaluation protocol (mAP@50 and mAP@50-95) and provides baseline results from five model configurations, including confusion matrices that highlight systematic failure modes (e.g., photograph vs. image confusion). This gives future researchers a clear measurement starting point. ## What is the motivation? Existing document layout analysis datasets tend to have one or more of the following limitations: - **Few non-textual categories.** Most datasets focus on textual structure (titles, headers, footers, captions) and lump visual elements into broad classes like "image" or "figure." - **Modern documents only.** Datasets like DocLayNet and PubLayNet focus on contemporary documents and miss visual elements unique to historical materials (friezes, vignettes, signets, exlibris). - **Synthetic data.** Some datasets rely on synthetic generation rather than real-world scanned pages. - **Single-element focus.** Several datasets target only one element type (tables, text lines). Historical documents preserved by libraries and archives contain distinctive visual elements (decorative borders, printer's marks, sheet music, hand-drawn maps) that are important for cultural heritage research but are absent from standard benchmarks. The authors argue that detecting and categorizing these elements is a prerequisite for downstream processing: images can be indexed for search, musical notations can be processed by OMR systems, and mathematical formulas can be parsed by recognition engines. ## What is the novelty? The core novelty is the dataset itself and its fine-grained categorization scheme. Key distinguishing features include: 1. **25 non-textual element categories.** This is considerably more granular than the typical handful of classes in existing datasets. Categories range from common elements (photographs, tables, charts, mathematical formulas) to domain-specific ones (exlibris, signets, vignettes, friezes, sheet music, chemical formulas, decorative inscriptions). The taxonomy follows the Czech Methodology of image document processing, a formally defined standard. 2. **Expert librarian annotations.** Librarians from Czech national institutions (Moravian Library, National Library of the Czech Republic, Library of the Czech Academy of Sciences) annotated the pages using Label Studio. The process involved four iterations: initial manual annotation, two rounds of model-assisted annotation (using YOLO predictions as proposals), and a final consistency verification pass where two YOLO models trained on complementary halves of the data flagged disagreements for review. 3. **Historical document coverage.** The 5,690 Czech Digital Library pages span from 1485 to the present, with the bulk from the late 19th and early 20th centuries. An additional 1,860 pages from six existing datasets (IlluHisDoc, PRImA, PRImA RDCL2019, PRImA Europeana Newspapers, ICDAR2019 cBAD, TexBiG) are re-annotated under the same schema to increase variability and maintain cross-dataset continuity. 4. **Transparent limitations.** The authors explicitly discuss category ambiguity (photograph vs. image, single vs. multiple tables, formula segmentation) and how borderline cases were resolved through annotator discussion. ## What experiments were performed? The authors train object detection baselines on the development set (6,950 pages) and evaluate on a held-out test set of 600 pages selected to match the overall category distribution. ### Models - **YOLO11** variants: YOLO11n, YOLO11s, YOLO11m, YOLO11l (via the Ultralytics package), initialized from publicly available pretrained weights - **DETR** (facebook/detr-resnet-50 from HuggingFace), also initialized from pretrained weights ### Training setup All models were trained for 250 epochs at a fixed resolution of $1024 \times 1024$ pixels. YOLO models used Adam with a learning rate of $2 \times 10^{-4}$. DETR used a learning rate of $5 \times 10^{-5}$. ### Evaluation metrics The primary metrics are mAP@50 and mAP@50-95, following standard COCO-style object detection evaluation. The IoU threshold for mAP@50 is 0.5; mAP@50-95 averages over thresholds from 0.5 to 0.95 in steps of 0.05. ### Key comparisons - **Czech subset vs. full dataset:** Models trained on the full dataset (including re-annotated pages from other datasets) consistently outperform those trained only on the Czech subset, confirming that the extra pages improve generalization. - **Resolution ablation:** YOLO11m was additionally trained at $800 \times 800$ and $1280 \times 1280$ pixel resolutions. The $1024 \times 1024$ setting performed best (0.658 mAP@50 vs. 0.622 at 800 px and 0.651 at 1280 px). ## What are the outcomes/conclusions? ### Results summary | Model | Czech mAP@50 | Czech mAP@50-95 | Full mAP@50 | Full mAP@50-95 | |:---|:---:|:---:|:---:|:---:| | YOLO11n | 0.595 | 0.531 | 0.616 | 0.558 | | YOLO11s | 0.567 | 0.513 | 0.641 | 0.585 | | YOLO11m | 0.592 | 0.541 | **0.658** | 0.598 | | YOLO11l | **0.607** | **0.547** | 0.657 | **0.602** | | DETR | 0.438 | 0.356 | 0.458 | 0.377 | - YOLO11m and YOLO11l achieve the best results, reaching approximately 0.66 mAP@50 and 0.60 mAP@50-95 on the full dataset. These numbers suggest significant room for improvement. - DETR underperforms substantially (0.458 mAP@50), which the authors attribute to the relatively small dataset size, consistent with the known data-hunger of transformer-based detectors. - Confusion matrix analysis reveals that the model most frequently confuses photographs and cartoons with the generic "image" category, and other technical drawings with schemas. Chemical formulas, exlibris, and handwritten inscriptions are frequently missed entirely (high false negative rates). - The extra dataset pages provide a meaningful boost (roughly 2 to 7 points in mAP@50 depending on the model), validating the decision to incorporate re-annotated pages from existing datasets. ### Limitations - The 25-category scheme introduces inherent ambiguity between visually similar classes (photograph vs. image, different drawing types). - Annotation granularity is not always consistent: whether closely adjacent tables or formula fragments should be treated as one element or several depends on context and interpretation. - The dataset is heavily skewed toward certain categories (images, photographs, tables are common; exlibris, chemical formulas, barcodes are rare), which likely contributes to poor detection of tail categories. - Only axis-aligned bounding boxes are provided; no polygon or pixel-level annotations. ## Mapping to Matter vs. Meaning Framework How does AnnoPage's 25-category non-textual element taxonomy map to our [Matter vs. Meaning framework](../matter-vs-meaning/)? Note that AnnoPage annotates only **non-textual elements**; text regions are not labeled. | # | AnnoPage Class | Visual Primitive | Logical Role | Notes | | :---: | :--- | :--- | :--- | :--- | | 1 | **Chemical formula** | `Formula` | `ChemScheme` | Chemical structures and reaction equations. | | 2 | **Symbol** | `Image` | `Logo` | Icons, pictograms, logos, heraldic emblems. Broad category with cultural significance. | | 3 | **Exlibris** | `Image` | `Stamp` | Bookplate ownership marks. | | 4 | **Photograph** | `Image` | `Figure` | Photographic content. Frequently confused with Image class. | | 5 | **Geometric drawing** | `Image` | `Diagram` | Points, lines, triangles, spatial forms with simple drawn lines. | | 6 | **Chart** | `Image` | `Chart` | Data visualizations with coordinate systems. | | 7 | **Initial** | `Image` | `Body` | Decorated drop-cap letters. Treated as Image because standard font properties cannot describe the appearance. | | 8 | **Cartoon** | `Image` | `Figure` | Cartoon illustrations, caricatures, satire. | | 9 | **Map** | `Image` | `Diagram` | Geographic maps. | | 10 | **Mathematical formula** | `Formula` | `DisplayEquation` | Mathematical notation on separate lines. | | 11 | **Sheet music** | `Music` | *(primitive only)* | Staves with musical symbols. | | 12 | **Image** | `Image` | `Figure` | Generic non-photographic visual content (illustrations, engravings). Also serves as fallback. | | 13 | **Other decoration** | `Structure` | *(decorative)* | Ornamental borders, frames, decorative strips. | | 14 | **Other technical drawing** | `Image` | `Diagram` | Drawings, outlines, cross-sections not fitting other technical categories. | | 15 | **Decorative inscription** | `Image` | `Icon` | Ornamental text rendered as artwork (not OCR-readable). | | 16 | **Technical drawing** | `Image` | `Diagram` | Floor plans, top-down building projections. | | 17 | **Barcode** | `OpticalCode` | `Barcode1D` / `Barcode2D` | Machine-readable codes for library collection management. | | 18 | **Stamp** | `Image` | `Stamp` | Provenance marks indicating ownership. Can be textual or graphic. | | 19 | **Advertisement** | `Image` | `Figure` | Visual promotions framed as separate graphic units. | | 20 | **Handwritten inscription** | `Text` | `Annotation` | Marginalia, glosses, provenance notes. `is_handwritten: True`. | | 21 | **Schema** | `Image` | `Diagram` | Simplified representations of systems or processes. | | 22 | **Signet** | `Image` | `Stamp` | Printer's marks, publisher's devices. | | 23 | **Table** | `Table` | *(primitive only)* | Structured data in rows and columns. | | 24 | **Vignette** | `Image` | `Icon` | Ornamental book decoration, commonly on title pages. | | 25 | **Frieze** | `Structure` | *(decorative)* | Elongated horizontal decoration at page tops. | **Coverage strengths:** AnnoPage is unusually rich in historical-document-specific categories (exlibris, signets, friezes, vignettes, coat of arms) that no other DLA dataset covers. It also distinguishes chemical formulas from mathematical formulas and includes music notation. **Coverage gaps:** No text-region annotations at all (the dataset is non-textual elements only). No `SectionHeader`, `Caption`, `PageHeader`, `PageFooter`, `ListItem`, or form primitives. By design, this dataset complements rather than replaces general-purpose DLA datasets. ## Reproducibility ### Models - YOLO11 variants (n/s/m/l) from the Ultralytics package, initialized from publicly available pretrained checkpoints. No custom architectural modifications are described. - DETR uses the `facebook/detr-resnet-50` pretrained model from HuggingFace. No modifications reported. - No model weights from the baseline experiments appear to be released. ### Algorithms - **Optimizer:** Adam for all YOLO variants; the optimizer for DETR is not explicitly named, only the learning rate ($5 \times 10^{-5}$) is given. - **Learning rate:** $2 \times 10^{-4}$ for YOLO, $5 \times 10^{-5}$ for DETR. No schedule (warmup, decay, cosine annealing) is described for either framework. - **Batch size:** Not reported for any model. - **Epochs:** 250 for all models. Model selection criterion (best validation mAP checkpoint vs. final epoch) is not specified beyond "evaluated the best-performing models on the test set." - **Input resolution:** $1024 \times 1024$ pixels (with an ablation at 800 and 1280 for YOLO11m). - **Data augmentation:** Not described beyond what Ultralytics and HuggingFace DETR provide by default. - **Loss functions:** Not specified; presumably the default losses for each framework (YOLO composite box/class/DFL loss; DETR bipartite matching with Hungarian algorithm). ### Data - **Source:** 5,690 pages from the Czech Digital Library (publicly accessible), plus 1,860 pages from six existing datasets (IlluHisDoc, PRImA, PRImA RDCL2019, PRImA Europeana Newspapers, ICDAR2019 cBAD, TexBiG). - **Total annotations:** 27,904 elements across 6,726 pages with at least one element, plus 824 empty pages. - **Split:** Development set (6,950 pages) and test set (600 pages from the Czech subset, containing 1,938 elements). The authors further split the development set into training (6,441 pages) and validation (509 pages, 1,926 elements), selecting validation pages to match the overall category distribution. Test set category distribution was similarly selected to match the overall dataset distribution. - **Annotation process:** Expert librarians using Label Studio across four iterations (manual, two model-assisted rounds, one consistency-verification round). Annotations follow the Czech Methodology of image document processing. Inter-annotator agreement metrics are not reported. - **Public availability:** The dataset is publicly available on Zenodo under CC-BY-4.0. Ground truth is provided in YOLO format. Pages from external datasets must be obtained separately from their original sources. ### Evaluation - **Metrics:** mAP@50 and mAP@50-95 (COCO-style). mAP@50 uses a single IoU threshold of 0.5; mAP@50-95 averages over IoU thresholds from 0.5 to 0.95 in steps of 0.05. Column-wise normalized confusion matrices are also provided for YOLO11l, offering per-category diagnostic detail. - **Baselines:** Five detector configurations (four YOLO11 variants, one DETR). Comparisons are internal (Czech subset vs. full dataset); no cross-dataset comparisons with other layout analysis benchmarks are reported. - **Limitations acknowledged:** Category ambiguity (photograph vs. image, table granularity, formula segmentation), class imbalance (tail categories have very few examples), and axis-aligned bounding boxes only (no polygons or pixel masks). - **Statistical rigor:** No error bars, multiple-run statistics, or seed sensitivity analysis are reported. Results appear to be from single training runs. No significance tests are provided. ### Hardware - Training and inference hardware are not specified in the paper. - No GPU-hour estimates, memory requirements, or latency measurements are reported. - No cost estimates or deployment considerations are discussed. Given the use of standard YOLO11 and DETR architectures, reproducing baselines should be feasible on a single modern GPU, but specifics are left to the reader. ## BibTeX ```bibtex @misc{kiss2025annopage, title={AnnoPage Dataset: Dataset of Non-Textual Elements in Documents with Fine-Grained Categorization}, author={Martin Kišš and Michal Hradiš and Martina Dvořáková and Václav Jiroušek and Filip Kersch}, year={2025}, eprint={2503.22526}, archivePrefix={arXiv}, primaryClass={cs.CV} } ```