# GOT-OCR2.0: Unified End-to-End OCR with General Optical Character Theory

> Analysis of GOT-OCR2.0, a 580M parameter unified encoder-decoder OCR model supporting plain text, formulas, tables, charts, sheet music, and geometric shapes via multi-stage training and synthetic data engines

- Authors: Hunter Heidenreich
- Published: 2024-09-03
- Updated: 2026-03-04
- Code: GOT-OCR2.0 Code <https://github.com/Ucas-HaoranWei/GOT-OCR2.0/>
- Model: GOT-OCR2.0 Model <https://huggingface.co/stepfun-ai/GOT-OCR2_0>
- Source: Bevaya Labs, <https://labs.bevaya.ai/>

---


## TL;DR

GOT-OCR2.0 is a unified 580M parameter encoder-decoder OCR model that treats diverse optical signals (plain text, formulas, tables, charts, sheet music, geometric shapes) as a single "character" space. The system introduces interactive region OCR via box/color prompts, dynamic resolution through multi-crop tiling, and multi-page OCR via decoder post-training. On the Fox benchmark for dense document OCR, GOT achieves edit distance 0.035 (EN) and 0.038 (ZH) with F1 scores of 0.972 and 0.980 respectively. The model supports only English and Chinese with high quality.

* * *

## What kind of paper is this?

**Dominant:** $\Psi_{\text{Method}}$ (new unified model architecture + three-stage training recipe + task unification framework).

**Secondary:** $\Psi_{\text{Resource}}$ (extensive synthetic data engines for formulas, molecules, tables, sheet music, geometry, and charts; open-source code and model release).

The paper introduces "General OCR Theory" (OCR-2.0) as a conceptual framework for treating diverse optical signals as characters, with a simple encoder-decoder architecture trained via multi-stage optimization. The resource component is substantial: the authors describe detailed synthetic data engines totaling $>$10M image-text pairs across stages, with open-source code and Apache 2.0 model weights provided.

## What is the motivation?

- **OCR-1.0 pipeline brittleness and cost:** Traditional multi-module systems (detection/cropping/recognition) are prone to local optima, cascading errors, and high maintenance overhead.
- **Task fragmentation:** Different specialized models exist for text detection, scene OCR, formula recognition, table extraction, etc. Practitioners must choose among many models for different subtasks, increasing deployment complexity.
- **Broadened "OCR" demand:** The authors frame the target as "intelligent processing of man-made optical signals" beyond plain text, including formulas, tables, charts, sheet music, molecular structures, and geometric diagrams.
- **Limited multilingual support in end-to-end systems:** Prior unified OCR models often focus narrowly on English or require separate models for different languages.

## What is the novelty?

**General OCR Theory ("OCR-2.0") framing:** Treats diverse optical signals (plain text, mathematical notation, tables, charts, sheet music, molecular structures, geometric shapes) as "characters" in a unified space, aiming for a single end-to-end model that handles multiple OCR tasks.

**Simple encoder-decoder OCR architecture:** Vision encoder (VitDet base with local attention, $\sim$80M params) + linear connector + language decoder (Qwen-0.5B, total 580M params). The encoder compresses $1024 \times 1024$ images to $256 \times 1024$ tokens, with the decoder supporting up to 8K context length.

**Feature extensions via decoder post-training:** Fine-grained region OCR (box and color prompts), dynamic resolution (multi-crop sliding window with max 12 tiles following InternVL-1.5), and multi-page OCR are added in Stage 3 without modifying the vision encoder. This preserves encoder pretraining while expanding capabilities.

**Staged training strategy with data mixing:** Three-stage pipeline (encoder pretrain with tiny OPT-125M, joint training with Qwen-0.5B on formatted/general OCR, decoder post-train for features) with 80% mixing of previous stage data to reduce regression.

**Extensive synthetic data engines:** Stage 1 uses 5M pure text pairs from LAION/Wukong/PDFs; Stage 2 adds 1M formulas (arXiv LaTeX $\rightarrow$ Mathpix format, $>$20$\times$ faster rendering), 1M molecules (ChEMBL SMILES), 0.3M tables, 1.2M full-page formatted docs, 0.5M sheet music (GrandStaff + Verovio), 1M geometry (TikZ), and 2M charts (Matplotlib/Pyecharts); Stage 3 constructs 60w fine-grained, 50w multi-crop, and 20w multi-page samples.

## What experiments were performed?

The authors evaluate across **five OCR task families:**

**1. Plain document OCR:** Fox benchmark (dense multi-page documents); word-level segmentation; metrics include edit distance, F1, precision, recall, BLEU, METEOR. Compares GOT (580M) against larger LVLMs.

**2. Scene text OCR:** 400 natural scene images (200 English, 200 Chinese) with manually corrected ground truth; character-level segmentation.

**3. Formatted document OCR:** 90 pages (English + Chinese) with Mathpix pseudo-labels manually corrected; evaluates single-scale vs multi-crop inference on formula and table metrics.

**4. Fine-grained OCR:** Box-guided and color-guided referential OCR evaluation in English and Chinese; compares GOT to Fox on region extraction accuracy.

**5. General OCR:** Chart OCR using ChartQA (structure-extraction version) and PlotQA benchmarks; reports AP@strict/slight/high for chart structure accuracy.

**Test data filtering:** The authors apply "strict text similarity filtering" to reduce overlap between training and test text, though details are not provided.

## What are the outcomes/limitations?

### Outcomes

**Plain document OCR (Fox benchmark):**

| Model | Lang | Edit Dist $\downarrow$ | F1 $\uparrow$ | Precision $\uparrow$ | Recall $\uparrow$ | BLEU $\uparrow$ | METEOR $\uparrow$ |
|-------|------|------------|-------|-----------|--------|------|--------|
| **GOT** | EN | **0.035** | **0.972** | **0.971** | **0.973** | **0.947** | **0.958** |
| **GOT** | ZH | **0.038** | **0.980** | **0.982** | **0.978** | **0.878** | **0.939** |

GOT achieves very strong dense document OCR metrics. The paper reports GOT ranks favorably against larger LVLMs in their comparison table, though specific baseline names and parameter sizes are included in the paper but numeric results are only provided for GOT.

**Scene OCR (400 images):**

| Model | Lang | Edit Dist $\downarrow$ | F1 $\uparrow$ | Precision $\uparrow$ | Recall $\uparrow$ | BLEU $\uparrow$ | METEOR $\uparrow$ |
|-------|------|------------|-------|-----------|--------|------|--------|
| **GOT** | EN | **0.112** | **0.926** | **0.934** | **0.927** | **0.676** | **0.896** |
| **GOT** | ZH | **0.096** | **0.928** | **0.914** | **0.954** | **0.641** | **0.928** |

Scene OCR edit distance is higher than document OCR (0.112 EN vs 0.035), reflecting the increased difficulty of natural scene text with perspective distortion, occlusion, and varied fonts.

**Formatted document OCR (90 pages, single vs multi-crop):**

| Setup | Formula F1 $\uparrow$ | Table METEOR $\uparrow$ |
|-------|------------|--------------|
| Single-scale | 0.749 | 0.760 |
| **Multi-crop** | **0.865** | **0.811** |

Multi-crop inference provides substantial gains for formula recognition (+11.6 F1 points) and table extraction (+5.1 METEOR points), demonstrating the effectiveness of dynamic resolution for high-detail regions.

**Chart OCR:**

| Dataset | AP@strict $\uparrow$ | AP@slight $\uparrow$ | AP@high $\uparrow$ |
|---------|-------------|-------------|-----------|
| **ChartQA-SE** | **0.747** | **0.845** | **0.867** |
| **PlotQA-SE** | **0.133** | **0.596** | **0.640** |

GOT shows stronger performance on ChartQA (0.747 strict AP) compared to PlotQA (0.133 strict AP), suggesting better generalization to the ChartQA structure extraction task.

**Fine-grained OCR:** The authors report GOT outperforms Fox on both box-guided and color-guided referential OCR in English and Chinese (Table 4 in paper), though specific numeric metrics are not transcribed in the rough draft.

### Limitations and open questions

**Language coverage:** The authors explicitly state they "mainly support English and Chinese" and "cannot guarantee OCR quality for other languages" even if some appear in crawled PDFs. Scaling to dozens of languages (Arabic, Cyrillic, Indic scripts, etc.) is not addressed. The single multilingual model design may face capacity constraints beyond 2-3 languages.

**Geometry scope:** The authors describe geometry rendering as "preliminary" and state the model "can only recognize basic geometry at present." Complex geometric diagrams, 3D figures, and advanced TikZ constructs are likely beyond current capabilities.

**Ablation gaps:** The paper does not ablate the contribution of individual components:
- How much gain comes from VitDet local attention vs other encoders?
- What is the impact of the 80% data mixing strategy vs training on new data only?
- How much do synthetic data engines contribute vs real data?

**No throughput or latency data:** Training hardware is reported (64 L40s), but the paper omits inference latency (ms/page), throughput (pages/s), and cost-per-page estimates. This limits comparison to olmOCR ($176/M pages on L40S) or MinerU2.5 (1.224 pages/s on A100).

**Test set construction and leakage:** The authors mention "strict text similarity filtering" to reduce train/test overlap but do not specify the threshold, filtering method, or amount of data removed. External validation on established benchmarks beyond Fox and ChartQA is limited.

**Multi-page OCR evaluation:** The paper describes 20w multi-page training samples (2-8 pages each, <8K tokens total) but does not evaluate multi-page OCR performance separately. It is unclear how well page-breaking and cross-page references are handled compared to single-page inference.

**Comparison baseline details:** The paper compares GOT to "larger LVLMs" on Fox but omits model names, parameter sizes, and numeric results for baselines in the extracted text. This limits reproducibility of the comparison.

* * *

## Model

### Architecture

**High-level design:** Vision encoder + linear connector + language decoder.

**Vision encoder:**
- **Backbone:** VitDet (base variant), chosen for local attention mechanism to reduce compute on high-resolution images.
- **Parameters:** $\sim$80M.
- **Tokenization:** Final layers compress $1024 \times 1024 \times 3$ input image to $256 \times 1024$ image tokens.
- **Projection:** Linear layer ($1024 \times 768$) projects image tokens into language model dimension during encoder pretraining (described in Stage 1).

**Language decoder:**
- **Stage 1 (encoder pretrain):** OPT-125M serves as a "tiny decoder" to efficiently pass gradients to the encoder.
- **Stage 2-3 (main model):** Qwen-0.5B replaces OPT-125M for broader OCR-2.0 training.
- **Context length:** Supports up to 8K tokens (increased from 4K in Stage 1 to 6K in Stage 2 to 8K in Stage 3).

**Total parameters:** 580M (encoder ~80M + decoder ~500M).

### Supported inputs and outputs

**Inputs:**
- Scene and document images (single-page or multi-page).
- Fine-grained region specification via bounding box coordinates (normalized $\times 1000$) or color-coded frames (red/green/blue).
- Multi-crop tiling for ultra-high-resolution documents (max 12 tiles, $1024 \times 1024$ per tile, InternVL-1.5 cropping strategy).

**Outputs:**
- **Plain text:** Character sequences for scene and document OCR.
- **Formatted outputs:** Mathpix-markdown (formulas, tables), TikZ (geometry), SMILES (molecules), custom notation (sheet music, charts) via prompting.

**Aspect ratio handling:** Input images of various shapes are resized to $1024 \times 1024$ via a "compromise" strategy (details not specified in paper).

### Design rationale

**VitDet local attention:** Reduces computational cost on high-resolution images compared to global attention mechanisms. The authors note this is critical for $1024 \times 1024$ inputs.

**High compression ratio:** $1024 \times 1024$ image $\rightarrow$ $256 \times 1024$ tokens represents $\sim$16$\times$ spatial compression (256 tokens for $1024^2$ pixels), enabling efficient processing of dense document pages.

**Decoder post-training for features:** Stage 3 freezes the vision encoder and only trains the decoder on fine-grained, multi-crop, and multi-page data. This design preserves encoder pretraining while adding capabilities, reducing compute cost vs end-to-end retraining.

> **Contrast to olmOCR2:** GOT uses VitDet with local attention for high-resolution processing, while olmOCR2 employs SigLIP with global attention and dynamic tiling. GOT compresses to 256 tokens per image vs olmOCR2's variable token count (up to 1984 tokens for high-detail regions).

> **Contrast to Nougat:** Nougat uses Swin Transformer encoder with academic document specialization, while GOT adopts VitDet for broader "OCR-2.0" coverage including charts, sheet music, and geometry. GOT's 8K context supports multi-page OCR, while Nougat processes single pages.

* * *

## Data

### Stage 1: Pure text recognition (encoder pretraining)

**Total:** $\sim$5M image-text pairs (3M scene OCR + 2M document OCR).

**Scene OCR sources (3M):**
- **Image sources:** LAION (English), Wukong (Chinese).
- **Pseudo ground truth:** PaddleOCR extracts text from images.
- **Processing:**
  1. Remove bounding boxes and concatenate text top-to-bottom, left-to-right.
  2. Crop text regions into slices, yielding additional $\sim$1M slice pairs.

**Document OCR sources (2M):**
- **PDFs:** Collected from Common Crawl.
- **Text extraction:** Fitz library extracts text.
- **Outputs:** $\sim$1.2M full-page pairs + 0.8M line/paragraph slice data.

**Preprocessing:** Images of various aspect ratios are resized to $1024 \times 1024$ via a "compromise" strategy (details not specified).

### Stage 2: Formatted OCR + general characters

**Formatted data sources:**

**Formulas (1M):**
- **Source:** arXiv LaTeX files.
- **Processing:** Extract formula fragments, convert to Mathpix-markdown format.
- **Rendering:** Mathpix-markdown-it library (HTML $\rightarrow$ SVG $\rightarrow$ PNG), claimed $>$20$\times$ faster than LaTeX rendering.

**Molecules (1M):**
- **Source:** ChEMBL_25 dataset with 2M SMILES strings.
- **Processing:** Produce $\sim$1M molecular structure image-text pairs.
- **Rendering:** Mathpix-markdown-it + rdkit.Chem library.

**Tables (0.3M):**
- **Source:** LaTeX table code.
- **Rendering:** LaTeX rendering (authors prefer this over Mathpix-markdown-it for tables).

**Full-page formatted documents (1.2M):**
- **English:** $\sim$0.5M markdown PDF-text pairs via Nougat method, converted to Mathpix format.
- **Chinese:** $\sim$0.5M markdown pairs via Vary-style processing, converted to Mathpix format.
- **In-house labeled:** $\sim$0.2M Mathpix-labeled data from books, papers, and financial reports.

**General OCR sources:**

**Sheet music (0.5M):**
- **Source:** GrandStaff dataset.
- **Additional rendering:** Verovio library.
- **Total:** $\sim$0.5M samples after rendering.

**Geometry (1M):**
- **Source:** TikZ text outputs.
- **Total:** $\sim$1M geometric TikZ data.

**Charts (2M):**
- **Rendering:** Matplotlib and Pyecharts libraries.
- **Composition:** 1M Matplotlib + 1M Pyecharts chart image-text pairs.

### Stage 3: Feature-specific data engines

**Fine-grained OCR (60w samples):**
- **Scene sources:** RCTW, ReCTS, ShopSign, COCO-Text for natural fine-grained OCR.
- **Document sources:** Parse PDFs (filter scanned format), use Fitz/PDFminer to record page images, line/paragraph boxes, and text.
- **Coordinate normalization:** Bounding boxes normalized then multiplied by 1000.
- **Color-guided variant:** Red/green/blue frame colors drawn on image to specify region.

**Ultra-large images via multi-crop (50w pairs):**
- **Tiling strategy:** Sliding window $1024 \times 1024$, InternVL-1.5 cropping method, max 12 tiles.
- **Synthetic data:** Horizontal and vertical stitching of single-page PDFs to create ultra-large documents.

**Multi-page OCR (20w pairs):**
- **Motivation:** Page-breaking is difficult for some formatted PDFs (e.g., arXiv LaTeX); training on multi-page pairs directly improves cross-page handling.
- **Construction:** Sample 2-8 pages; each page <650 tokens; ensure total length <8K context.
- **Language mixing:** Often mixes Chinese and English pages in single multi-page samples.

### Data quality and leakage control

**Pseudo-labeling quality:** Stage 1 uses PaddleOCR for pseudo ground truth. Stage 2 formatted data uses Mathpix-markdown-it rendering and manual correction for evaluation sets. The quality of pseudo-labels is not quantified.

**Test set filtering:** The authors apply "strict text similarity filtering" to reduce overlap between training and test text, but do not specify the threshold, filtering method, or amount of data removed.

**Language coverage:** Training data primarily covers English and Chinese. The authors state they cannot guarantee OCR quality for other languages even if some appear in crawled PDFs.

* * *

## Algorithms / Training

### Three-stage training strategy

**Stage 1: Encoder pretraining (pure text recognition)**
- **Objective:** Pretrain vision encoder using tiny OPT-125M decoder for efficient gradient propagation.
- **Data:** 5M pure text pairs (scene + document OCR).
- **Optimization:** Global batch size 128; 3 epochs; AdamW optimizer; cosine annealing schedule; starting LR $1e{-}4$; max token length 4096.

**Stage 2: Joint training (formatted + general OCR)**
- **Objective:** Connect encoder to Qwen-0.5B and train on broader OCR-2.0 "characters" (formulas, charts, geometry, sheet music, molecules, tables).
- **Data:** Stage 2 formatted/general data (1M formulas, 1M molecules, 0.3M tables, 1.2M full-page, 0.5M sheet music, 1M geometry, 2M charts) + 80% of Stage 1 data (4M pure text samples).
- **Optimization:** 1 epoch; max token length 6000; same optimizer settings as Stage 1 (AdamW, cosine annealing, LR $1e{-}4$).

**Stage 3: Decoder post-training (feature extensions)**
- **Objective:** Freeze vision encoder; train decoder on fine-grained, multi-crop, and multi-page OCR capabilities.
- **Data:** 60w fine-grained samples + 50w multi-crop pairs + 20w multi-page pairs + 80% of Stage 1-2 data.
- **Optimization:** 1 epoch; max token length 8192; starting LR $2e{-}5$ (lower than Stage 1-2).

### Data mixing strategy

**80% mixing across stages:** During each stage, the authors sample 80% of previous stage(s) data to reduce catastrophic forgetting when adding new features or data types. For example:
- Stage 2: 80% of Stage 1 pure text data (4M samples) + 100% of Stage 2 formatted/general data.
- Stage 3: 80% of Stage 1-2 data + 100% of Stage 3 feature-specific data.

### Hardware and compute

**Training hardware:** 64 L40s GPUs (described as "8$\times$8 L40s").

**Training time:** Not reported in the paper.

**Throughput:** Not reported in the paper.

* * *

## Evaluation

### Plain document OCR (Fox benchmark)

The Fox benchmark evaluates dense multi-page document OCR with word-level segmentation. Metrics include edit distance (character-level), F1/precision/recall (word-level), BLEU, and METEOR.

| Model | Lang | Edit Dist $\downarrow$ | F1 $\uparrow$ | Precision $\uparrow$ | Recall $\uparrow$ | BLEU $\uparrow$ | METEOR $\uparrow$ |
|-------|------|------------|-------|-----------|--------|------|--------|
| **GOT (580M)** | EN | **0.035** | **0.972** | **0.971** | **0.973** | **0.947** | **0.958** |
| **GOT (580M)** | ZH | **0.038** | **0.980** | **0.982** | **0.978** | **0.878** | **0.939** |

The paper includes a comparison table with larger LVLMs but the rough draft does not transcribe baseline names or numeric results. GOT is reported to achieve competitive or superior performance despite its smaller size (580M).

### Scene text OCR (400 images)

A custom evaluation set of 400 natural scene images (200 English, 200 Chinese) with manually corrected ground truth. Character-level segmentation is used for metrics.

| Model | Lang | Edit Dist $\downarrow$ | F1 $\uparrow$ | Precision $\uparrow$ | Recall $\uparrow$ | BLEU $\uparrow$ | METEOR $\uparrow$ |
|-------|------|------------|-------|-----------|--------|------|--------|
| **GOT (580M)** | EN | **0.112** | **0.926** | **0.934** | **0.927** | **0.676** | **0.896** |
| **GOT (580M)** | ZH | **0.096** | **0.928** | **0.914** | **0.954** | **0.641** | **0.928** |

Scene OCR edit distance is 3$\times$ higher than Fox document OCR (0.112 vs 0.035 for English), reflecting the increased difficulty of natural scene text with perspective distortion, occlusion, complex backgrounds, and varied fonts.

### Formatted document OCR (90 pages)

A custom evaluation set of 90 pages (English + Chinese) with Mathpix pseudo-labels manually corrected. Compares single-scale ($1024 \times 1024$) vs multi-crop inference.

| Inference Mode | Formula F1 $\uparrow$ | Table METEOR $\uparrow$ |
|----------------|------------|--------------|
| Single-scale | 0.749 | 0.760 |
| **Multi-crop (max 12 tiles)** | **0.865** | **0.811** |

Multi-crop inference provides +11.6 formula F1 points and +5.1 table METEOR points. This demonstrates the effectiveness of dynamic resolution for high-detail regions (small fonts, dense tables, complex formulas).

### Fine-grained OCR (box- and color-guided)

The paper evaluates referential OCR using bounding box coordinates and color-coded frames in English and Chinese. Results are compared to Fox.

The rough draft notes "GOT is reported as better than Fox on both region and color referential OCR in both languages (see Table 4)" but does not transcribe specific numeric metrics.

### Chart OCR (ChartQA-SE and PlotQA-SE)

Structure extraction variants of ChartQA and PlotQA benchmarks. Metrics are Average Precision at strict/slight/high thresholds.

| Dataset | AP@strict $\uparrow$ | AP@slight $\uparrow$ | AP@high $\uparrow$ |
|---------|-------------|-------------|-----------|
| **ChartQA-SE** | **0.747** | **0.845** | **0.867** |
| **PlotQA-SE** | **0.133** | **0.596** | **0.640** |

GOT shows substantially stronger performance on ChartQA (0.747 strict AP) compared to PlotQA (0.133 strict AP). This gap suggests dataset-specific generalization; ChartQA may have chart types or structure patterns more similar to GOT's training data (2M Matplotlib/Pyecharts charts).

### Metric definitions

**Edit distance:** Levenshtein distance at character level, normalized by ground-truth length. Lower is better.

**F1/Precision/Recall:** Word-level (Fox document OCR) or character-level (scene OCR) token matching. Higher is better.

**BLEU:** Bilingual Evaluation Understudy score, n-gram overlap between prediction and reference. Higher is better.

**METEOR:** Metric for Evaluation of Translation with Explicit ORdering, incorporates synonyms and paraphrasing. Higher is better.

**AP@strict/slight/high:** Average Precision at different IoU or tolerance thresholds for chart structure extraction. Higher is better.

* * *

## Hardware / Production

### Training infrastructure

**Hardware:** 64 L40s GPUs (described as "8$\times$8 L40s" setup).

**Training time:** **NR** (not reported in the paper).

**Throughput:** **NR** (pages/second or tokens/second not reported).

**Cost:** **NR** (no cost estimates provided).

### Inference performance

**Latency:** **NR** (milliseconds per page not reported).

**Throughput:** **NR** (pages/second not reported).

**Cost per page:** **NR** (no cost estimates provided for inference).

**Hardware requirements:** The paper does not specify minimum GPU memory or recommended inference hardware.

### Deployment

**Model availability:** Open-source weights released on HuggingFace ([stepfun-ai/GOT-OCR2_0](https://huggingface.co/stepfun-ai/GOT-OCR2_0)).

**License:** Apache 2.0.

**Code:** GitHub repository ([Ucas-HaoranWei/GOT-OCR2.0](https://github.com/Ucas-HaoranWei/GOT-OCR2.0/)) provides inference scripts and examples.

**Serving infrastructure:** The paper does not describe production serving infrastructure, batching strategies, or optimization techniques for deployment.

* * *

## Notes and open questions

**Observations:**

- GOT's "OCR-2.0" framing is ambitious, treating formulas, charts, sheet music, geometry, and molecules as "characters" in a unified space. The single-model approach is simpler than multi-module pipelines, but the capacity constraints of a 580M model may limit depth in each task.
- The three-stage training strategy with 80% data mixing is a practical approach to reducing catastrophic forgetting. However, the paper does not ablate the mixing ratio or compare to alternative continual learning strategies.
- Multi-crop inference provides substantial gains for formatted documents (+11.6 formula F1, +5.1 table METEOR), validating the dynamic resolution design. The max 12 tiles limit balances detail vs context length constraints (8K tokens).
- Scene OCR edit distance is 3$\times$ higher than document OCR (0.112 vs 0.035 for English), suggesting GOT is optimized primarily for document-style inputs. Natural scene text with perspective distortion and complex backgrounds remains challenging.

**Open questions:**

- **Ablation studies:** What is the contribution of VitDet local attention vs other encoders (e.g., SigLIP, Swin)? How much gain comes from the 80% data mixing strategy vs training on new data only? How do synthetic data engines compare to real data?
- **Geometry and sheet music quality:** The authors state geometry recognition is "preliminary" and limited to "basic geometry." How much of the 1M geometric TikZ training data does the model successfully learn? What is the error rate on GrandStaff sheet music notation?
- **Multilingual scaling:** GOT supports only English and Chinese with high quality. How would the 580M parameter budget scale to dozens of languages (Arabic, Cyrillic, Indic scripts, etc.)? Would capacity constraints require larger models or ensemble architectures?
- **Inference efficiency:** No latency or throughput data is reported. How does GOT compare to olmOCR ($176/M pages on L40S) or MinerU2.5 (1.224 pages/s on A100) in cost and speed?
- **Test set leakage:** The "strict text similarity filtering" method is not detailed. What threshold is used? How much test data is removed? External validation on held-out benchmarks (e.g., olmOCR-Bench, DocVQA) would strengthen confidence in generalization.
- **Multi-page OCR evaluation:** The paper describes 20w multi-page training samples but does not evaluate multi-page performance separately. How well does GOT handle page-breaking, cross-page references, and consistent formatting across pages compared to single-page inference?
- **Comparison baseline details:** The Fox benchmark comparison table in the paper includes larger LVLMs, but the rough draft omits model names, parameter sizes, and numeric results. Access to the full table would enable better positioning of GOT's performance.