# SciBank: Deep Learning in Time-Frequency Domain for Document Layout Analysis > Research notes on SciBank, a 74K-page dataset of annotated scientific paper regions with 12 classes including inline equations, and the spectrogram-based DLA method introduced alongside it. - Authors: Bevaya AI Team - Published: 2026-03-10 - Updated: 2026-03-15 - Paper: Published (IEEE Access 2021) - Dataset: SciBank (IEEE DataPort) - Source: Bevaya Labs, --- ## TL;DR Grijalva et al. introduce a three-stage pipeline for scientific document layout analysis that converts page regions into spectrograms of intensity histograms and classifies them with a CNN. Alongside the method, they release **SciBank**, a 74,435-page dataset of scientific articles annotated with 12 region classes, notably including inline equations, a label absent from most competing datasets. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$. The headline contribution is a DLA pipeline that operates in the time-frequency domain (spectrograms of intensity histograms) rather than directly on pixel data. **Secondary:** $\Psi\_{\text{Resource}}$. The paper also releases SciBank, a large-scale annotated dataset for scientific document layout analysis, available on IEEE DataPort under CC-BY-4.0. ## What is the motivation? Existing document layout analysis approaches typically operate directly on spatial pixel features. The authors argue that transforming page regions into the frequency domain (via spectrograms of horizontal and vertical intensity histograms) can capture structural patterns more compactly, enabling a lighter-weight classifier. They also note that prior datasets like PubLayNet, DocBank, and TableBank lack annotations for inline equations, a common and structurally important element in scientific papers. ## What is the novelty? The core idea is a three-stage pipeline: 1. **Segmentation and spectrogram extraction:** Each page is segmented into regions of interest. Horizontal and vertical intensity histograms are computed for each region, then converted into spectrograms via short-time Fourier transform. 2. **CNN classification:** A deep CNN classifies each spectrogram-represented region into one of the 12 layout categories. 3. **Inline equation detection:** A Bag of Visual Words (BOVW) approach with Zernike moments identifies isolated equations within text regions. The central idea is that projecting spatial structure into frequency space can reduce computational cost while preserving discriminative layout features. ## What experiments were performed? - The method was evaluated on SciBank (74,435 pages from 11,007 scientific papers). - The 12 classes are: abstract, text block, caption, keywords, reference, section heading, subsection heading, title, table, figure, isolated equation, and inline equation. - Annotations were generated automatically and validated by human curators. - The primary metrics reported are the Adjusted Rand Index (ARI) and Variation of Information (VI), along with overall classification accuracy. - The authors report comparisons against prior approaches. We were unable to access the full text, so the specific baselines are not enumerated here. ## What are the outcomes/conclusions? The system achieved an overall accuracy of 96.27%, which the authors report as competitive with or exceeding prior methods at lower computational cost. The spectrogram-based representation appears to capture sufficient structural information for region classification without requiring multimodal (text + vision) input. ### Limitations - The dataset is limited to scientific articles, so generalization to other document types (forms, reports, newspapers) is untested. - The annotation pipeline is automatic with human validation rather than fully manual, so annotation quality may vary, particularly for ambiguous regions. - The evaluation uses ARI and VI rather than the more standard COCO-style mAP metrics, making direct comparison with other DLA benchmarks difficult. - The 12-class label set is moderately fine-grained but does not include structural relations (e.g., caption-to-figure links) or reading order. - The paper does not report per-class performance, so it is unclear how well the method handles rare or visually similar classes. ## SciBank Dataset Details | Property | Value | | :--- | :--- | | **Pages** | 74,435 | | **Papers** | 11,007 | | **Domain** | Scientific articles | | **Annotation** | Automatic + human validation | | **Classes** | 12 | | **Eval split** | Unknown | | **Formats** | PNG, PDF, CSV | | **Size** | ~48 GB | | **License** | CC-BY-4.0 | | **Access** | IEEE DataPort (free account required) | **Label set:** Abstract, Text Block, Caption, Keywords, Reference, Section, Subsection, Title, Table, Figure, Isolated Equation, Inline Equation. The inclusion of **inline equations** as a distinct class is SciBank's most notable differentiator from PubLayNet (5 classes), DocBank (12 classes, token-level), and TableBank (table-only). **Note on authorship:** The IEEE DataPort entry lists seven authors (adding Carla Parra and Marco Gallardo to the five on the IEEE Access paper), suggesting additional contributors were involved in the dataset curation and release. ## Reproducibility ### Models - The classification model is a deep CNN operating on spectrogram inputs. Specific architecture details (layer count, parameter count) are described in the paper but were not available without full-text access at the time of this review. - No pretrained weights appear to be publicly released. ### Algorithms - Spectrograms are derived from horizontal and vertical intensity histograms via short-time Fourier transform. - Inline equation detection uses Bag of Visual Words with Zernike moments. - Training details (optimizer, learning rate, batch size) are in the paper. ### Data - Source: 11,007 scientific papers (source repositories not specified in available metadata). - Annotations generated automatically, then validated by human curators. - Available on IEEE DataPort under CC-BY-4.0. - Evaluation split structure is unclear from available metadata. ### Evaluation - **Metrics:** Adjusted Rand Index (ARI), Variation of Information (VI), overall accuracy. - These are not the standard COCO-style mAP metrics used by most modern DLA benchmarks, which limits comparability. - Per-class breakdowns were not available in the metadata reviewed. ### Hardware - Not specified in available metadata. ## BibTeX ```bibtex @article{grijalva2021deep, author={Grijalva, Felipe and Santos, Erick and Acu\~{n}a, Byron and Rodr\'{i}guez, Juan Carlos and Larco, Julio C\'{e}sar}, journal={IEEE Access}, title={Deep Learning in Time-Frequency Domain for Document Layout Analysis}, year={2021}, volume={9}, pages={151254--151265}, doi={10.1109/ACCESS.2021.3125913} } ```