# MinerU2.5: Coarse-to-Fine Document Parsing with Decoupled Layout > Analysis of MinerU2.5, a 1.2B VLM for document parsing using two-stage decoupled layout analysis with PageIoU metric and IMIC hard-case mining - Authors: Hunter Heidenreich - Published: 2025-09-01 - Updated: 2026-03-04 - Paper: MinerU2.5 Paper - Code: MinerU Code - Model: MinerU2.5-2509-1.2B Model - Source: Bevaya Labs, --- ## TL;DR MinerU2.5 is a 1.2B-parameter vision language model for document parsing that decouples global layout analysis from local content recognition using a coarse-to-fine, two-stage inference pipeline (thumbnail layout first, then native-resolution crops). With a NaViT vision encoder (675M params), a Qwen2-0.5B decoder with M-RoPE, and a pixel-unshuffle patch merger, it achieves strong results on OmniDocBench (Overall 90.67) while maintaining practical throughput ($\\approx$ 2.12 pages/s on A100-80G). The paper introduces PageIoU (layout metric), ADR (atomic decomposition for long formulas), OTSL (compact table target), and IMIC (consistency-based hard-case mining). * * * ## What kind of paper is this? Primarily $\\Psi_{\\text{Method}}$ with strong $\\Psi_{\\text{Resource}}$ and $\\Psi\_{\\text{Evaluation}}$ components. The headline novelty is the two-stage decoupled parsing architecture: thumbnail-based layout detection followed by native-resolution crop recognition. Secondary contributions include new evaluation methodology (PageIoU metric) and a data flywheel (IMIC mining strategy). The model weights and code are publicly released. **Rough superposition:** $\\Psi_{\\text{Method}}(0.65) + \\Psi_{\\text{Resource}}(0.20) + \\Psi\_{\\text{Evaluation}}(0.15)$ ## What is the motivation? High-resolution document parsing faces a fundamental tension: native-resolution processing yields better recognition quality but incurs $O(N^2)$ attention costs over massive token sequences. Existing solutions have critical gaps: - **Pipeline systems** are modular but suffer from error propagation and cumbersome integration - **End-to-end VLMs** are bottlenecked by hallucinations on long documents and token redundancy from blank or low-information regions - Prior systems either sacrifice resolution (hurting small text/formulas) or sacrifice throughput (impractical for production) MinerU2.5 addresses this by decoupling global layout from local content: make layout explicit in Stage I on a cheap thumbnail, then "spend" tokens only on content-bearing crops in Stage II at native resolution. ## What is the novelty? The core contribution is a **coarse-to-fine, two-stage decoupled parsing architecture**: - **Stage I (global)**: Predict layout/rotation/reading-order on a fixed 1036$\\times$1036 thumbnail, avoiding native-resolution $O(N^2)$ token blowup - **Stage II (local)**: Process native-resolution crops independently, enabling parallelism and bounded token counts Supporting innovations: - **PageIoU**: A page-level coverage metric for layout using coverage maps and pixel-wise min/max aggregation, better matching human perception than IoU@0.5 for text blocks (Figure 4, p. 15) - **ADR (Atomic Decomposition & Recombination)**: For multi-line formulas, decompose into atomic lines, recognize each, then recombine with LaTeX alignment (Figure 5, p. 16) - **OTSL**: A compact table representation reducing structural tokens from $\\sim$28 to 5, shrinking sequences by $\\sim$50% (Figure 6, p. 17) - **IMIC (Iterative Mining via Inference Consistency)**: Run stochastic decoding multiple times; low consistency (measured by PageIoU/TEDS/CDM) flags hard cases for human annotation (Figure 7, p. 18) - **M-RoPE**: Replace 1D-RoPE with multi-dimensional RoPE to generalize across varying crop resolutions/aspect ratios ## What experiments were performed? Evaluation spans six benchmark categories with detailed ablations: - **Full-page parsing**: OmniDocBench (1,355 pages, EN/ZH) measuring text edit distance, formula CDM, table TEDS/TEDS-S, reading order edit distance - **Dense text OCR**: Ocean-OCR (EN/ZH splits) with edit distance, F1, BLEU, METEOR metrics - **Unit tests**: olmOCR-bench (math, scans, tiny text) using ExpRate (render-based) for math splits instead of AST-string CDM - **Layout analysis**: OmniDocBench, D4LA, DocLayNet using PageIoU metric with unified tag set (headers, footers, page numbers, code, algorithms, references, lists, caption sub-types) - **Table recognition**: PubTabNet, FinTabNet, CC-OCR, OCRBench v2, in-house TR dataset - **Formula recognition**: CPE, HWE, SCE, SPE, LaTeX-80MM, plus in-house Chinese/Fuzzy/Complex sets Baselines include Qwen2.5-VL (7B/72B), InternVL3, Gemini-2.5-Pro, and specialized systems (GOT, Docling, MinerU2). The paper also provides throughput benchmarks across A100-80G, RTX 4090-48G, and H200-141G. ## What are the outcomes/limitations? **Key results** (Tables 5--11, pp. 19--24): - **OmniDocBench Overall: 90.67** (best reported), with Text-Edit 0.047, TEDS 88.22, CDM 88.46, RO-ED 0.044 - Overall score defined as: $\\text{Overall} = \\frac{(1-\\text{TextEdit}) \\times 100 + \\text{TableTEDS} + \\text{FormulaCDM}}{3}$ - **FinTabNet**: TEDS 95.97 / TEDS-S 97.61 (leading by margin) - **olmOCR-bench**: 75.2 overall (best among listed); AR (math) 76.6, Old Scans Math 54.6, Long Tiny Text 83.5 - **Ocean-OCR English**: ED 0.033, F1 0.945, BLEU 0.909, METEOR 0.950 - **Throughput**: 2.12 pages/s (A100-80G), 4.47 pages/s (H200-141G), 1.70 pages/s (RTX 4090-48G) via vLLM **Limitations and open questions**: - **Stage I is a recall bottleneck**: Missed or merged elements in layout propagate to Stage II; the system cannot recover from layout errors - **Cross-domain robustness depends on the data engine**: Rare layouts (multi-page wraps, exotic scripts) may require targeted IMIC mining - **Heuristic tuning**: Crop scheduling and repetition penalties are manually tuned; learned scheduling policies could improve batch efficiency - **Dependency on frontier models**: The data curation pipeline uses Qwen2.5-VL-72B, Gemini-2.5-Pro for pre-annotation, which may limit reproducibility * * * ## Model ### Architecture (Figure 2, p. 7) | Component | Details | | -------------- | ---------------------------------------------------------------------------- | | Vision encoder | NaViT (Native-Res ViT), $\\sim$675M params, 2D-RoPE, arbitrary aspect ratios | | Patch merger | Pixel-unshuffle $2\\times 2$, reduces vision tokens before LM | | LM decoder | Qwen2-Instruct 0.5B with M-RoPE | | **Total** | $\\sim$1.2B params | **Key architectural choices**: - **NaViT with 2D-RoPE**: Chosen over Qwen2.5-VL's window attention, which the authors note can degrade document parsing performance - **M-RoPE** (multi-dimensional RoPE): Replaces 1D-RoPE in the LM decoder to generalize across varying crop resolutions and aspect ratios - **Pixel-unshuffle patch merger**: Merges adjacent $2 \\times 2$ vision tokens before feeding to the LM, trading off efficiency and performance ### Two-stage pipeline - **Stage I (global layout)**: - Input: Uniformly resized to 1036$\\times$1036 pixels - Output: Box positions, classes, rotation angles, and reading order in a single pass - Benefit: Fixed thumbnail improves box stability and training efficiency - **Stage II (local content)**: - Input: Crops from original high-res image using Stage I boxes - Resolution: Native resolution with upper bound of 2048$\\times$28$\\times$28 pixels - Output: Text, table (OTSL format), or formula (LaTeX) per crop - Benefit: Crops process independently, enabling batching and parallelism ### Task reformulations **Layout analysis as multi-task**: Predict Position, Class, Rotation Angle, and Reading Order in one pass (instead of pushing rotation/order downstream to separate stages). **Formulas (ADR)**: "Whole-part" approach classifies formulas as atomic vs compound, decomposes compound formulas into lines, recognizes each line, then recombines using layout info. **Tables (OTSL)**: Generate OTSL instead of HTML to reduce structural token redundancy (structural tokens reduced from "over 28" down to "5", $\\sim$50% shorter sequences), then convert to HTML post-processing. ### Prompts (Appendix B) Task-specific prompts switch output format: ```text Layout Detection: ``` Output: box positions, classes, rotation, reading order tokens ```text Text Recognition: ``` Output: `OCR Results:{text}` ```text Formula Recognition: ``` Output: `LaTeX:{latex}` ```text Table Recognition: ``` Output: `OTSL:{otsl}` > **Contrast to end-to-end OCR VLMs**: Instead of attending over massive native-res token sequences for the whole page, MinerU2.5 filters with Stage I and only "spends" tokens on content-bearing crops. This cuts $O(N^2)$ token growth, reduces hallucinations, and keeps reading order explicit. * * * ## Data ### Data Engine (Figure 3, p. 12) The closed-loop curation pipeline balances: - **Layout variety**: Document types including papers, textbooks, reports, slides - **Element mix**: Titles, paragraphs, tables, formulas, figures - **Language distribution**: EN/ZH Methods: page-level image clustering, metadata sampling, element-balance detector to ensure representative coverage. ### Pre-training label refinement Starting from MinerU2-pipeline outputs, refined with specialist models: - **Text crops**: Qwen2.5-VL-72B-Instruct - **Formulas**: UniMERNet (retrained in-house) - **Tables**: In-house high-performance table parser ### Training data sizes | Stage | Samples | Breakdown | | ------------------------ | ------- | ------------------------------------------------ | | Stage 0A (image-caption) | 558K | Modality alignment | | Stage 0B (VQA & OCR) | 665K | Instruction following | | Stage 1 (pretrain) | 6.9M | Layout 2.3M, text 2.4M, formula 1.1M, table 1.1M | | Stage 2 (fine-tune) | 630K | Layout 43K, text 300K, formula 147K, table 140K | Note: Stage 1 uses 2 epochs over 6.9M samples; Stage 2 uses 3 epochs over 630K samples. ### Fine-tuning set curation with IMIC **IMIC (Iterative Mining via Inference Consistency)**: Run the model multiple times with stochastic decoding and measure consistency via: - **PageIoU** for layout analysis - **TEDS** for tables - **CDM** for formulas Low-consistency samples (below threshold) are flagged as hard cases for human annotation. Complex tables may be pre-annotated by Gemini-2.5-Pro then expert-corrected via the Dingo QA tool, yielding a compact, high-value SFT set. ### Data augmentation (Table 2) Augmentations simulate scans and photographic artifacts: - **Spatial transforms**: Rotation, shearing, perspective (NOT applied to layout analysis samples to preserve box accuracy) - **Background variation**: Color jitter, background replacement - **Degradation effects**: Blur, noise, compression artifacts * * * ## Algorithms / Training ### Training stages (Table 1, p. 9) | Stage | Description | Seq len | Batch size | Epochs | Vision max res | | ----- | ----------------------------------------------- | ------- | ---------- | ------ | -------------------------- | | 0A | Modality alignment (freeze ViT & LM, train MLP) | 4096 | 128 | --- | 2048$\\times$28$\\times$28 | | 0B | VQA & OCR (unfreeze all) | 4096 | 64 | --- | 4096$\\times$28$\\times$28 | | 1 | Document parsing pretrain | 8192 | 256 | 2 | 2048$\\times$28$\\times$28 | | 2 | Hard-case fine-tuning | 16384 | 256 | 3 | 2048$\\times$28$\\times$28 | **Initialization**: - Vision encoder: from Qwen2-VL-2B-Instruct - LM decoder: from Qwen2-Instruct-0.5B Separate learning rates for ViT vs {MLP, LM}. Stage 2 uses the compact, hard-case mined dataset identified by IMIC. ### Key algorithmic contributions **PageIoU** (pp. 13--15): Page-level coverage score for layout using coverage maps and pixel-wise min/max aggregation over non-background region $M$. Unified tag set covers headers, footers, page numbers, code, algorithms, references, lists, and caption sub-types (Table 4, p. 14). Better matches qualitative layout quality than IoU@0.5. **ADR (Atomic Decomposition & Recombination)** (pp. 15--16): For multi-line/long expressions: 1. Classify formula as atomic vs compound 2. Decompose compound formulas into atomic lines via layout 3. Recognize each line independently 4. Recombine into LaTeX with alignment (e.g., `align` environment) **OTSL (Optimized Table Structure Language)** (pp. 16--17): Pipeline for table recognition: 1. Detect table + rotation 2. Rectify crop 3. Recognize to OTSL format 4. Convert to HTML Reduces structural tokens from $\\sim$28 to 5, shrinking sequences by $\\sim$50%. **IMIC (Iterative Mining via Inference Consistency)** (pp. 17--18): Data curation method: 1. Run model multiple times with stochastic decoding 2. Measure consistency via PageIoU (layout), TEDS (tables), CDM (formulas) 3. Low-consistency samples (below threshold) flagged for human QA 4. Results in compact, high-value SFT set * * * ## Evaluation ### OmniDocBench (Tables 5--6, p. 19) | Metric | MinerU2.5 | Notes | | ------------------------------ | --------- | ----- | | Overall | 90.67 | Best | | Text-Edit $\\downarrow$ | 0.047 | Best | | Formula CDM $\\uparrow$ | 88.46 | Best | | Table TEDS $\\uparrow$ | 88.22 | Best | | Table TEDS-S $\\uparrow$ | 92.38 | Best | | Reading-Order ED $\\downarrow$ | 0.044 | Best | **Overall score definition:** $$ \\text{Overall} = \\frac{(1 - \\text{TextEdit}) \\times 100 + \\text{TableTEDS} + \\text{FormulaCDM}}{3} $$ By document type (Text-Edit $\\downarrow$): Newspaper 0.0540 (best), Textbook 0.0499 (best), Slides 0.0294 (2nd), Financial 0.0104 (2nd). **Benchmark details**: DPI raised to 200 for Notes/Newspapers; EN/ZH balanced with +374 pages (total 1,355 pages). ### Ocean-OCR (Table 7, p. 20) Dense text recognition benchmark: | Split | ED | F1 | BLEU | METEOR | | ------- | ----- | ----- | ----- | ------ | | English | 0.033 | 0.945 | 0.909 | 0.950 | | Chinese | --- | 0.965 | 0.817 | 0.887 | ### olmOCR-bench (Table 8, p. 20) Overall: **75.2** (best among listed systems) - AR (math): 76.6 - Old Scans Math: 54.6 - Long Tiny Text: 83.5 **Note**: ExpRate (render-based evaluation) replaces AST-string CDM for math splits to avoid penalizing cosmetically different but semantically equivalent LaTeX (p. 21). ### Layout analysis (Table 9, p. 22) Top Full-Page F1@PageIoU scores across three benchmarks: - **OmniDocBench**: Leads across all categories - **D4LA**: Strong performance - **DocLayNet**: Leads or co-leads per-category (Textual/Image/Table/Equation/Page-margins) PageIoU metric uses unified tag set (Table 4, p. 14) covering headers, footers, page numbers, code, algorithms, references, lists, and caption sub-types. ### Table recognition (Table 10, p. 23) | Benchmark | TEDS | TEDS-S | | ------------------- | ----------- | ----------- | | PubTabNet | 89.07 (2nd) | 93.11 (3rd) | | FinTabNet | **95.97** | **97.61** | | CC-OCR (table) | 79.76 | --- | | OCRBench v2 (table) | 87.13 | --- | | In-house TR | 71.48 | 82.83 | FinTabNet results lead by a significant margin. ### Formula recognition (Table 11, p. 24) Character Detection Metric (CDM) scores: | Split | CDM | | ------------------- | ---- | | CPE | 96.6 | | HWE | 94.4 | | SCE | 96.4 | | SPE | 98.4 | | LaTeX-80MM (matrix) | 90.6 | | In-house Chinese | 90.7 | | In-house Fuzzy-Math | 92.6 | | In-house Complex | 82.2 | * * * ## Hardware / Production ### vLLM-based deployment pipeline (Table 3, p. 11) **Architecture**: - vLLM with asynchronous page-level batching - Stage I and Stage II run as independent tasks with decoupled scheduling - Stage II work starts as soon as Stage I results arrive (async backend) - Dynamic penalties (frequency/presence) conditioned on Stage I layout type to suppress degenerate repetition without harming legitimate repetition in tables/equations **Throughput benchmarks**: | GPU | Tokens/s | Pages/s | | ------------ | -------- | ------- | | A100-80G | 2337.25 | 2.12 | | RTX 4090-48G | 1875.82 | 1.70 | | H200-141G | 4938.31 | 4.47 | **Baseline** (no deployment optimizations): 1045 tokens/s, 0.95 pages/s on A100-80G. The deployment improvements come from: 1. Async task scheduling for Stage I/II decoupling 2. Dynamic sampling penalties conditioned on layout type 3. Batch submission optimizations ### Training infrastructure **Initialization**: - Vision encoder: from Qwen2-VL-2B-Instruct - LM decoder: from Qwen2-Instruct-0.5B Separate learning rates for ViT vs {MLP, LM}. Augmentations tuned by element type (Table 2). The paper emphasizes the data engine and deployment pipeline over full cluster specifications; training compute details are not explicitly provided.