# ENP Image and Ground Truth Dataset of Historical Newspapers > The Europeana Newspapers Project (ENP) dataset provides 528 annotated historical newspaper page images from 12 European libraries, with comprehensive PAGE-format ground truth covering layout, text, and reading order. - Authors: Hunter Heidenreich - Published: 2026-03-12 - Updated: 2026-03-13 - Paper: Paper (IEEE) - Dataset: ENP Dataset - Source: Bevaya Labs, --- ## TL;DR The ENP dataset contains **528 historical newspaper page images** from digitization projects of 12 national and major European libraries, spanning the 17th to mid-20th century across 13 languages. Each page is accompanied by comprehensive ground truth in PAGE format, including precise region outlines, type labels, Unicode-encoded full text, and reading order. The authors also provide baseline OCR evaluations using ABBYY FineReader Engine 11 and Tesseract 3.03. --- ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Resource}}$ The headline contribution is the dataset itself: a large, representative, publicly available collection of historical newspaper images with detailed ground truth. The paper devotes most of its space to describing how the dataset was created, what it contains, and how it is hosted and accessed. **Secondary:** $\Psi\_{\text{Evaluation}}$ The second part of the paper establishes a baseline evaluation of two OCR systems (ABBYY FineReader 11 and Tesseract 3.03) on the dataset, using both text-based and scenario-based layout analysis performance measures. This evaluation component, while not the primary contribution, demonstrates the dataset's utility and provides reference performance numbers for future comparisons. ## What is the motivation? Meaningful progress in document analysis for historical materials requires representative datasets with detailed ground truth. Historical newspapers present particular challenges: low print quality, large image sizes, complex multi-column layouts, mixed typefaces (including Gothic/Fraktur), and diverse languages. At the time of publication, no comparable dataset existed. The closest prior work, the IMPACT dataset, focused primarily on books rather than newspapers. The Europeana Newspapers Project (ENP), which processed over 10 million newspaper pages for The European Library and Europeana, provided both the resources and the institutional collaboration needed to assemble a representative collection from 12 European libraries. The authors aimed to build a dataset that would be: - **Realistic:** Reflecting the actual distribution of documents in library collections. - **Comprehensive:** Including metadata, detailed ground truth (layout, text, reading order). - **Flexibly structured:** Supporting searching, browsing, grouping, and direct access from external systems. ## What is the novelty? **1. Scale and representativeness for historical newspapers:** The dataset includes 528 pages from 12 European libraries, covering 13 languages (Dutch, English, Estonian, Finnish, French, German, Latvian, Polish, Russian, Serbian, Swedish, Ukrainian, Yiddish) and publication dates from the 17th century through 1950. The distribution includes grayscale (47%), bitonal (29%), and colour (24%) images, all stored as lossless TIFF at 300 or 400 dpi. Pages average over 383 text lines each. **2. Comprehensive ground truth in PAGE format:** Every page has ground truth in PAGE (Page Analysis and Ground Truth Elements) format, including: - Precise region outlines (polygon boundaries) - Region type labels - Full text in Unicode (including special characters and ligatures) - Reading order The ground truth covers 61,619 regions total, of which 46,889 are text regions containing 202,524 text lines, along with 1,497 images/graphics and 208 tables. **3. Searchable online repository with web services:** The dataset is hosted through a web-based repository with metadata search, random subset generation, an interactive ground truth viewer/editor rendered in the browser, and a direct-access API for integration with external OCR workflows and evaluation tools. **4. Scenario-based OCR baseline:** The authors evaluate two OCR systems using scenario-based profiles that weight different error types according to intended use cases (keyword search, phrase search, content structure access, print/ebook on demand, content-based image retrieval). ## What experiments were performed? The authors evaluated ABBYY FineReader Engine 11 and Tesseract 3.03 in two settings: ### Text-based evaluation A Bag of Words analysis was used rather than word accuracy, since word accuracy becomes unreliable for complex layouts with non-linear reading order. Results for FineReader across font types: - **Normal (Antiqua) fonts:** 81.4% success rate - **Gothic (Fraktur) fonts:** 67.3% success rate - **Mixed fonts:** 64.0% success rate The authors note that the Gothic performance was substantially better than prior expectations, attributable to ABBYY's Fraktur module developed during the IMPACT project. ### Scenario-based layout analysis evaluation Five use scenarios were defined based on input from content holders and users: 1. Keyword search in full text 2. Phrase search in full text 3. Access via content structure 4. Print/eBook on demand 5. Content-based image retrieval Each scenario defines an evaluation strategy with specific weights for segmentation errors (merge, split, miss, false detection), misclassification errors, and reading order errors. Results varied considerably across scenarios, and neither system was a clear winner across all use cases, suggesting that Tesseract could outperform FineReader in certain scenarios. Error breakdowns for FineReader showed different compositions of merge, split, partial miss, miss, false detection, and misclassification errors depending on the scenario. ## What are the outcomes/conclusions? The ENP dataset fills a clear gap in available resources for historical newspaper analysis. The key outcomes are: 1. **A publicly available, representative dataset:** 528 pages with comprehensive ground truth, free of charge for researchers. 2. **Practical cost insights:** Ground truth creation required several hundred person-hours across multiple organizations. The manual correction cost alone was approximately 15,000 euros, underscoring the expense of high-quality annotation (targeting 99.95% accuracy). 3. **OCR baselines reveal remaining challenges:** Gothic and mixed-font documents remain substantially harder than normal text for commercial OCR. Scenario-based evaluation reveals that no single system dominates across all use cases. 4. **Infrastructure for reproducible evaluation:** The online repository, with searchable metadata, random subset generation, and API access, supports reproducible experiments and integration with external tools. ### Limitations - The dataset is limited to 528 pages, constrained by the available budget for ground truth production. - Copyright restrictions reduced the original 600-page collection. - The evaluation covers only two OCR systems available at the time (2015); more recent systems are not benchmarked. - The paper does not report inter-annotator agreement metrics, though a three-stage quality assurance process is described. ## Mapping to Matter vs. Meaning Framework ENP uses the PAGE-XML format (see [PRImA notes](2009-07-prima-dataset.md) for the full PAGE-XML region mapping). The dataset's 61,619 annotated regions break down into three broad types that map to the [Matter vs. Meaning framework](../matter-vs-meaning/) as follows: | ENP Region Type | Count | Visual Primitive | Logical Role | Notes | | :--- | :--- | :--- | :--- | :--- | | **TextRegion** | 46,889 | `Text` | `Body` / `SectionHeader` / `Caption` / etc. | Role depends on PAGE-XML `type` attribute (paragraph, heading, caption, etc.). See PRImA mapping for full subtype breakdown. | | **ImageRegion / GraphicRegion** | 1,497 | `Image` | `Figure` | Photographs, illustrations, advertisements in newspaper pages. | | **TableRegion** | 208 | `Table` | *(primitive only)* | Tables in newspaper content (rare). | | **SeparatorRegion** | *(included in total)* | `Structure` | *(none)* | Column separators, rules between articles. Particularly important for newspaper layouts. | **Structural notes:** ENP provides **reading order** annotations via PAGE-XML's `OrderedGroup`/`UnorderedGroup` elements, which is critical for multi-column newspaper layouts where the visual reading path is non-trivial. The dataset also includes full Unicode text content at the text-line level (202,524 lines total), making it useful for OCR evaluation beyond layout detection. **Coverage strengths:** The multilingual coverage (13 languages) and diachronic range (17th-20th century) give ENP broad coverage that is uncommon among historical newspaper datasets. The PAGE-XML format preserves the logical vs. physical distinction that the M-vs-M framework formalizes. **Coverage gaps:** No explicit classes for `Formula`, `Music`, `OpticalCode`, or any form primitives. Newspaper-specific elements like `JumpLine`, `Flag`, or `Advertisement` are not separated from generic TextRegion or ImageRegion. ## Reproducibility ### Models Not applicable. This is a dataset paper; no models are introduced. ### Algorithms No new algorithms are proposed. The paper evaluates ABBYY FineReader Engine 11 and Tesseract 3.03 using their existing capabilities: - **FineReader configuration:** Run in production workflow mode with three font profiles (Gothic/Fraktur, Normal/Antiqua, Mixed). The Fraktur module was developed as part of the IMPACT project. - **Tesseract configuration:** Version 3.03 was used, but the paper does not specify which language models or configuration parameters were applied. - **Ground truth pre-processing:** Preliminary OCR output in PAGE format was generated using ABBYY FineReader Engine 10 (not 11) and provided to service providers as a starting point for ground truth correction. - **Scenario-based evaluation profiles:** The methodology for weighting segmentation, classification, and reading order errors per scenario follows Clausner et al. (ICDAR 2011), but the exact weight values used in this paper are not reproduced in the text. ### Data - **Source:** 12 national and major European libraries participating in the Europeana Newspapers Project. - **Size:** 528 page images (reduced from an initial collection of 600 due to copyright restrictions). Originally, up to 50 images per institution were targeted. - **Format:** Lossless TIFF images at 300 or 400 dpi. Ground truth in PAGE XML format. - **Languages:** 13 (Dutch, English, Estonian, Finnish, French, German, Latvian, Polish, Russian, Serbian, Swedish, Ukrainian, Yiddish). German pages (169) are the most represented. - **Time period:** 17th century through 1950, with the majority from the 19th century (187 pages) and early-to-mid 20th century (323 pages). - **Ground truth contents:** 61,619 regions, 46,889 text regions, 202,524 text lines, 1,497 images/graphics, 208 tables. - **Quality target:** 99.95% accuracy, enforced through a three-stage quality assurance process (automated validation, manual layout inspection, text verification by contributing libraries). - **Annotation process:** Semi-automated. Service providers received preliminary OCR output in PAGE format (generated using ABBYY FineReader Engine 10) and a customized version of the Aletheia ground truth editor. They could correct or discard the preliminary output. - **Image characteristics:** Grayscale (47%), bitonal (29%), and colour (24%). Common page characteristics include multi-column layout (487 pages), text/image overlap, skew, faint print, and bleed-through. - **Ground truth production cost:** Approximately 15,000 euros for manual correction alone, with several hundred person-hours across multiple organizations for the full pipeline (selection, pre-processing, correction, verification, ingestion, categorization). - **Public availability:** Free of charge for researchers at [primaresearch.org/datasets/ENP](https://www.primaresearch.org/datasets/ENP). Access requires authentication through a permissions management system; some materials may have institution-specific access agreements imposed by the contributing libraries. - **License:** Not explicitly stated in the paper. Access is described as "free-of-charge for researchers" but specific licensing terms are not provided. ### Evaluation - **Text metric:** Bag of Words success rate (chosen over word accuracy to avoid ambiguity from non-linear reading order in complex newspaper layouts). Only FineReader results are broken down by font type (Gothic 67.3%, Normal 81.4%, Mixed 64.0%); per-font Tesseract text accuracy numbers are not reported. - **Layout metric:** Scenario-based evaluation using weighted error types (merge, split, miss, false detection, misclassification, reading order) with five predefined use-case profiles (keyword search, phrase search, content structure access, print/ebook on demand, content-based image retrieval). The evaluation methodology and weighting scheme follow Clausner et al. (ICDAR 2011), but the specific weight values applied are not reproduced in this paper. - **Baselines:** ABBYY FineReader Engine 11 and Tesseract 3.03. The paper does not specify whether the full dataset (all 528 pages) or a subset was used for evaluation. - **Limitations acknowledged:** The dataset is constrained by budget (528 pages from an initial 600); copyright restrictions forced removal of some pages; only two OCR systems from 2015 are benchmarked; no inter-annotator agreement metrics are reported (though a three-stage QA process was used). - **Statistical rigor:** No error bars, confidence intervals, significance tests, or multi-run statistics are reported. Results are presented as single aggregate success rates per font type or scenario. ### Hardware Not reported. The paper does not discuss computational requirements for the OCR evaluations. ## BibTeX ```bibtex @inproceedings{clausner2015enp, title={The ENP Image and Ground Truth Dataset of Historical Newspapers}, author={Clausner, Christian and Papadopoulos, Christos and Pletschacher, Stefan and Antonacopoulos, Apostolos}, booktitle={2015 13th International Conference on Document Analysis and Recognition (ICDAR)}, pages={931--935}, year={2015}, organization={IEEE}, doi={10.1109/ICDAR.2015.7333898} } ```