# PAL Database: Document Layout Annotation for Public Affairs > Research note on the PAL Database, a large-scale layout-annotated dataset of Spanish government gazettes with 441K pages and 8M+ labels. - Authors: Hunter Heidenreich - Published: 2026-03-15 - Paper: Preprint (arXiv) - Paper: Published (ICDAR 2023 Workshops) - Dataset: PAL Database - Source: Bevaya Labs, --- ## TL;DR The PAL Database is a large-scale document layout analysis dataset covering 37,910 Spanish government gazettes (441,300 pages) with over 8 million layout labels across 8 categories. Annotations were generated through a semi-automatic pipeline combining PDF mining, heuristic rules, and human-in-the-loop Random Forest classifiers. The dataset is gated behind a signed license agreement. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Resource}}$. The headline contribution is the PAL Database itself: a new domain-specific dataset for document layout analysis in the public affairs domain. The paper devotes most of its space to describing the data collection pipeline, annotation process, and dataset statistics. **Secondary:** $\Psi\_{\text{Method}}$. The semi-automatic annotation pipeline (heuristic rules followed by iterative human-in-the-loop Random Forest classifiers) is a reusable methodological contribution beyond the dataset itself. ## What is the motivation? Existing DLA datasets (PubLayNet, DocBank, DocLayNet) focus on scientific articles, financial reports, and similar domains. Government and legal documents have distinct visual layouts (multi-column gazettes, structured metadata blocks, legal identifiers) that are underrepresented in current benchmarks. The authors argue that public affairs documents require their own annotated corpus because: - The layout structure differs from academic and business documents - Existing models may not generalize well to this domain - Government document processing is a practical need for legal and administrative automation Beyond layout analysis, the authors also frame the corpus as a large-scale NLP pre-training resource for the legal domain in Spanish and four co-official languages (Basque, Catalan/Valencian, Galician). ## What is the novelty? The core novelty is the dataset and its semi-automatic annotation pipeline: 1. **Data Collection:** Web scraping of 24 Spanish administration sources (3 national, 19 regional, 2 local) using Scrapy. Only born-digital PDFs from 2014 onward are included. 2. **Feature Extraction:** PyMuPDF extracts images, links, and text blocks with 18 features per block (bounding box, font size, bold/italic proportions, capitalization ratios, etc.). Camelot extracts tables. 3. **Two-step Annotation:** - Step 0 (Heuristic): Rule-based labeling using font features and keyword matching, followed by human validation through a custom annotation application - Step 1 (ML-assisted): Random Forest classifiers trained on validated documents (minimum 50 validated pages per source), with iterative human correction 4. **Label Taxonomy:** 8 categories split into 4 basic layout blocks (Image, Table, Link, Text) and 4 text semantic categories (Identifier, Title, Summary, Body). The per-source Random Forest approach acknowledges that government gazette layouts vary significantly across administrative regions, each requiring its own classifier. ## What experiments were performed? The evaluation focuses on validating the text-block classification quality: - **Model:** Random Forest classifiers (max depth 1000), one per data source (24 total) - **Features:** 17 of 18 extracted features (raw text excluded). Positional features normalized by document dimensions. Font type encoded via dictionary mapping. - **Protocol:** 10-fold cross-validation with 80/20 document-level splits on the validation set (1,444 documents, 19,276 pages) - **Metric:** Accuracy on 4-class text block classification (Identifier, Title, Summary, Body) ## What are the outcomes/conclusions? **Overall accuracy** ranges from 93.37% (Source 16, Ceuta) to 99.93% (Source 20), with 20 of 24 sources exceeding 96%. **Per-class performance:** - **Identifier:** Highest accuracy; mean above 99% for 22 of 24 sources. These blocks have consistent position, token count, and font characteristics across documents. - **Body:** All sources above 95%. The dominant class, easily distinguished by font regularity. - **Title:** Hardest category; all sources above 84%, 19 above 90%. The authors note titles are the hardest to label consistently due to varying editorial conventions across sources. - **Summary:** Generally high, but Source 16 (Ceuta) is an outlier at 61.08% ($\pm$12.35 std), indicating high instability for that source. **Limitations:** - Restricted to born-digital PDFs only; scanned/image-based documents are excluded - Feature extraction depends on internal PDF structure, which varies by editing software - Single human supervisor validated all documents (no inter-annotator agreement metrics) - The training set (422K pages) uses ML-generated labels that were only spot-checked ("visual inspection of an arbitrary selection of documents"), not systematically validated - No cross-source generalization experiments; each source gets its own classifier, so it is unknown whether the dataset supports training a single unified model - Only Random Forest classifiers evaluated; no comparison with deep learning approaches - The dataset is not freely available (requires signed license agreement) ## Reproducibility ### Models - Random Forest classifiers with max depth 1000, one per source - No deep learning models trained - No pretrained weights released - No source code released; the GitHub repository contains only a README and license agreement ### Algorithms - Feature extraction via PyMuPDF (text blocks, images, links, fonts) and Camelot (tables) - 18 features per text block (Table 2 in the paper): page number ($f\_1$), bounding box ($f\_{2\text{-}5}$: $x\_0, y\_0, x\_1, y\_1$), center coordinates ($f\_{6\text{-}7}$: $x\_c, y\_c$), distances to page edges ($f\_{8\text{-}11}$), raw text/data ($f\_{12}$), bold proportion ($f\_{13}$), italic proportion ($f\_{14}$), average font size ($f\_{15}$), font types tuple ($f\_{16}$), capital letter proportion ($f\_{17}$), word count ($f\_{18}$) - For classification, 17 of 18 features used ($f\_{12}$ excluded). Positional features ($f\_2$-$f\_{11}$) normalized by page dimensions. - Font type ($f\_{16}$) encoded via dictionary mapping (details not provided) - Full heuristic rule set for Step 0 annotation not disclosed; the paper provides only one example rule (bold proportion $> 0.5$ for titles) - 10-fold cross-validation with 80/20 document-level splits ### Data - **Sources:** 24 Spanish administration websites (official gazettes) - **Scale:** 37,910 documents, 441,300 pages, 138.1M tokens, 8M+ labels - **Split:** Train: 36,466 docs (422K pages); Validation: 1,444 docs (19,276 pages) - **Languages:** Spanish (primary), Basque, Catalan/Valencian, Galician - **Availability:** Gated. Requires downloading and signing a license agreement from the [BiDAlab GitHub repo](https://github.com/BiDAlab/PALdb), then emailing credentials to atvs@uam.es - **Annotation:** Semi-automatic (heuristic + ML-assisted) with single-annotator human validation ### Evaluation - **Metric:** Accuracy (4-class text block classification) - **Protocol:** 10-fold cross-validation, document-level splits - **No cross-dataset evaluation** against established DLA benchmarks (PubLayNet, DocLayNet) - Standard deviations reported across 10-fold CV, but no formal significance tests (e.g., paired t-tests between sources or against baselines) - **No comparison with deep learning baselines** (Faster R-CNN, YOLO, LayoutLM, etc.) ### Hardware - Not reported. Random Forest classifiers are lightweight and likely trained on CPU. - No training time, memory, or compute cost information provided. ## BibTeX ```bibtex @inproceedings{pena2023document, title={Document Layout Annotation: Database and Benchmark in the Domain of Public Affairs}, author={Pe{\~n}a, Alejandro and Morales, Aythami and Fierrez, Julian and Ortega-Garcia, Javier and Grande, Marcos and Puente, {\'I}{\~n}igo and C{\'o}rdova, Jorge and C{\'o}rdova, Gonzalo}, booktitle={Document Analysis and Recognition -- ICDAR 2023 Workshops}, series={Lecture Notes in Computer Science}, volume={14194}, pages={126--141}, year={2023}, publisher={Springer}, doi={10.1007/978-3-031-41501-2_9} } ```