# Modern OCR for the Large Language & Vision Model Era
> A reference page for all things Optical Character Recognition (OCR) using Large Language & Vision Models
- Authors: Hunter Heidenreich
- Published: 2025-10-27
- Source: Bevaya Labs,
---
> **Disclaimer:** This is a work-in-progress research compilation and personal draft. The coverage is not comprehensive, and the analysis reflects one individual's perspective on recent developments in the field. This resource should not be used as a definitive reference or benchmark for evaluating research contributions. Please refer to the original papers and conduct your own thorough review for academic or professional purposes.
## VLM-Based Models
End-to-end vision-language models with learned multimodal representations for general document OCR.
| Date | Paper | Notes | Weights | Code | License |
| ------- | ------------------------------------------------------ | ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | --------------------------- |
| 2026-01 | [GutenOCR](https://arxiv.org/abs/2601.14490) | | [3B](https://huggingface.co/rootsautomation/GutenOCR-3B), [7B](https://huggingface.co/rootsautomation/GutenOCR-7B) | [Roots-Automation/GutenOCR](https://github.com/Roots-Automation/GutenOCR) | Apache 2.0 (code), CC-BY-NC-4.0 (wts.) |
| 2025-11 | [Nemotron Parse 1.1](https://arxiv.org/abs/2511.20478) | [notes](2025-11-nemotron-parse.md) | [885M](https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-v1.1), [885M-TC](https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-v1.1-TC) | | NVIDIA Open Model |
| 2025-12 | [dots.ocr](https://arxiv.org/abs/2512.02498) | [notes](2025-12-dots-ocr.md) | [3B](https://huggingface.co/rednote-hilab/dots.ocr) | [rednote-hilab/dots.ocr](https://github.com/rednote-hilab/dots.ocr) | MIT |
| 2025-01 | [Ocean-OCR](https://arxiv.org/abs/2501.15558) | [notes](2025-01-ocean-ocr.md) | [3B](https://huggingface.co/guoxy25/Ocean-OCR) | [guoxy25/Ocean-OCR](https://github.com/guoxy25/Ocean-OCR) | Apache 2.0 |
| 2025-10 | [olmOCR 2](https://arxiv.org/abs/2510.19817) | [notes](2025-10-olmocr2.md) | [7B](https://huggingface.co/allenai/olmOCR-2-7B-1025) | [allenai/olmocr](https://github.com/allenai/olmocr) | Apache 2.0 |
| 2025-10 | [DeepSeek-OCR](https://arxiv.org/abs/2510.18234) | [notes](2025-10-deepseek-ocr.md) | [3B](https://huggingface.co/deepseek-ai/DeepSeek-OCR) | [deepseek-ai/DeepSeek-OCR](https://github.com/deepseek-ai/DeepSeek-OCR) | MIT |
| 2025-09 | [POINTS-Reader](https://arxiv.org/abs/2509.01215) | [notes](2025-09-points-reader.md) | [4B](https://huggingface.co/tencent/POINTS-Reader) | [Tencent/POINTS-Reader](https://github.com/Tencent/POINTS-Reader) | Apache 2.0 |
| 2025-09 | [MinerU2.5](https://arxiv.org/abs/2509.22186) | [notes](2025-09-mineru2_5.md) | [1.2B](https://huggingface.co/opendatalab/MinerU2.5-2509-1.2B) | [opendatalab/MinerU](https://github.com/opendatalab/MinerU) | AGPL-3.0 |
| 2025-06 | [Infinity-Parser](https://arxiv.org/abs/2506.03197) | [notes](2025-06-infinity-parser.md) | [7B](https://huggingface.co/infly/Infinity-Parser-7B) | [infly-ai/INF-MLLM](https://github.com/infly-ai/INF-MLLM) | Apache 2.0 |
| 2025-05 | [DocMark](https://arxiv.org/abs/2505.12377) | [notes](2025-05-docmark.md) | [2B/8B](https://huggingface.co/collections/HanXiao1999/docmark-adaptive-markup-language-generation-68284f713d046200532085e3) | [Euphoria16/DocMark](https://github.com/Euphoria16/DocMark) | Unclear |
| 2025-05 | [Dolphin](https://arxiv.org/abs/2505.14059) | [notes](2025-05-dolphin.md) | [322M](https://huggingface.co/ByteDance/Dolphin), [4B](https://huggingface.co/ByteDance/Dolphin-v2) | [ByteDance/Dolphin](https://github.com/ByteDance/Dolphin) | MIT |
| 2025-04 | [VISTA-OCR](https://arxiv.org/abs/2504.03621) | [notes](2025-04-vista-ocr.md) | | | |
| 2025-03 | [SmolDocling](https://arxiv.org/abs/2503.11576) | [notes](2025-03-smoldocling.md) | [256M](https://huggingface.co/ds4sd/SmolDocling-256M-preview) | [docling-project/docling](https://github.com/docling-project/docling) | CDLA-Permissive-2.0 |
| 2025-02 | [olmOCR](https://arxiv.org/abs/2502.18443) | [notes](2025-02-olmocr.md) | [7B](https://huggingface.co/allenai/olmOCR-7B-0825) | [allenai/olmocr](https://github.com/allenai/olmocr) | Apache 2.0 |
| 2024-09 | [GOT-OCR2.0](https://arxiv.org/abs/2409.01704) | [notes](2024-09-got-ocr2.md) | [580M](https://huggingface.co/stepfun-ai/GOT-OCR2_0) | [Ucas-HaoranWei/GOT-OCR2.0](https://github.com/Ucas-HaoranWei/GOT-OCR2.0/) | Apache 2.0 |
| 2023-08 | [Nougat](https://arxiv.org/abs/2308.13418) | [notes](2023-08-nougat.md) | [small](https://huggingface.co/facebook/nougat-small), [base](https://huggingface.co/facebook/nougat-base) | [facebookresearch/nougat](https://github.com/facebookresearch/nougat) | MIT (code), CC-BY-NC (wts.) |
## Pipeline Models
Modular systems combining specialized detection, recognition, and layout analysis components.
| Date | Paper | Notes | Weights | Code | License |
| ------- | -------------------------------------------------- | -------------------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------------- | ----------------------------------------- |
| 2025-10 | [PaddleOCR-VL](https://arxiv.org/abs/2510.14528v1) | [notes](2025-10-paddleocr-vl.md) | [0.9B](https://huggingface.co/PaddlePaddle/PaddleOCR-VL) | [PaddlePaddle/PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) | Apache 2.0 |
| 2025-07 | [PaddleOCR 3.0](https://arxiv.org/abs/2507.05595) | [notes](2025-07-paddleocr3.md) | [HuggingFace](https://huggingface.co/PaddlePaddle) | [PaddlePaddle/PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) | Apache 2.0 |
| 2025-06 | [MonkeyOCR](https://arxiv.org/abs/2506.05218) | [notes](2025-06-monkeyocr.md) | [3B](https://huggingface.co/echo840/MonkeyOCR-pro-3B) | [Yuliang-Liu/MonkeyOCR](https://github.com/Yuliang-Liu/MonkeyOCR) | Apache 2.0 |
| 2025-01 | [Docling v2](https://arxiv.org/abs/2501.17887) | [notes](2025-01-docling-v2.md) | [models](https://huggingface.co/docling-project/docling-models) | [DS4SD/docling](https://github.com/DS4SD/docling) | MIT (code), CDLA-Permissive-2.0 (weights) |
| 2024-09 | [MinerU](https://arxiv.org/abs/2409.18839) | [notes](2024-09-mineru.md) | [PDF-Extract-Kit](https://github.com/opendatalab/PDF-Extract-Kit) | [opendatalab/MinerU](https://github.com/opendatalab/MinerU) | Apache 2.0 |
| 2024-01 | [Surya](https://github.com/VikParuchuri/surya) | [notes](2024-01-surya.md) | [det3](https://huggingface.co/vikp/surya_det3) (38M), [layout2](https://huggingface.co/vikp/surya_layout2) (30M) | [VikParuchuri/surya](https://github.com/VikParuchuri/surya) | GPL-3.0 (code), CC-BY-NC-SA-4.0 (weights) |
## Datasets & Benchmarks
This section catalogs datasets and benchmarks organized by OCR sub-domain. Many benchmarks span multiple domains; they are listed under their primary focus.
Datasets are organized into three tiers based on licensing:
- **Commercial**: Permissive licenses (Apache-2.0, MIT, CC-BY) that allow commercial use
- **Research**: Non-commercial licenses (GPL, CC-NC) or explicit research-only restrictions
- **Unclear**: No license specified or mixed/complex licensing; verify before use
### General Document OCR
#### Research Use Only
| Date | Paper | Size | Tasks | Notes | Data | License |
| ------- | ------------------------------------------------ | ----------- | ---------------------------------------------------------- | ------------------------------ | ---- | ------- |
| 2024-12 | [OmniDocBench](https://arxiv.org/abs/2412.07626) | 1,355 pages | Text, formula, table, reading-order | Multi-domain benchmark (EN/ZH) | | |
| 2024-09 | [READoc](https://arxiv.org/abs/2409.05137) | 2,233 docs | PDF-to-Markdown structured extraction | ACL 2025 Findings. arXiv + GitHub docs. No spatial annotations. MIT license but listed here pending OCR index audit. | [HuggingFace](https://huggingface.co/datasets/lazyc/READoc), [GitHub](https://github.com/icip-cas/READoc) | MIT |
| 2024-05 | [Fox Bench](https://arxiv.org/abs/2405.14295) | 212 pages | Region/line/color OCR, translation, summary, multi-page OCR, cross-page VQA | [notes](2024-05-fox.md). EN/ZH bilingual; 9 sub-tasks | [HuggingFace](https://huggingface.co/datasets/ucaslcl/Fox_benchmark_data) | CC-BY-NC-4.0 |
| 2023-05 | [DUDE](https://arxiv.org/abs/2305.08455) | ~26K pages | Extractive QA, abstractive QA, list extraction | Multi-domain, multi-industry; 1860–2022 range; multi-page docs | | Research only |
| 2020-07 | [DocVQA](https://arxiv.org/abs/2007.00398) | 50K questions, 12K+ pages | Document VQA | Industry/government docs; human baseline 94.4% | [docvqa.org](https://www.docvqa.org/) | Research only |
#### License Unclear or Mixed
Datasets with unspecified licenses or complex multi-source licensing. Verify terms before use.
| Date | Paper | Size | Tasks | Notes | Data | License |
| ------- | ------------------------------------------------------- | -------------------------------------------- | ------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------- |
| 2024-03 | [ArXivCap / ArXivQA](https://arxiv.org/abs/2403.00231) | 6.4M figures, 572K papers; ArXivQA: 100K QA | Figure captioning (single, multi, contextualized), title generation | 32 scientific domains; extracted from LaTeX source; ArXivQA generated by GPT-4V | [mm-arxiv.github.io](https://mm-arxiv.github.io) | Unclear (arXiv CC-0 + per-paper content + GPT-4V terms) |
| 2022-11 | [VRDU](https://arxiv.org/abs/2211.15421) | ~10K pages | Structured entity extraction from business documents | Business docs (invoices, purchase orders); hierarchical entities; few-shot + conventional splits | | Unclear |
### Layout & Structure
For Layout Detection (e.g., DocLayNet, PubLayNet), see the dedicated **[Layout Analysis](../layout/)** page. For Table Structure Recognition (e.g., PubTables-1M, TableBank), see the **[TSR](../tsr/)** page.
* * *
### Charts & Visualizations
Chart understanding spans several task families:
- **Extraction**: Chart-to-table or chart-to-dict conversion (structured data recovery)
- **QA**: Visual question answering requiring numerical reasoning, comparison, or lookup
- **Summarization**: Natural language descriptions at varying semantic levels
Most datasets use synthetic charts (programmatically generated from tables) or web-scraped visualizations. Evaluation typically relies on exact/relaxed match for QA, BLEU/ROUGE for summarization, and F1 or RMS error for extraction. Scale varies dramatically: from ~6K charts (ChartY) to 28.9M QA pairs (PlotQA).
#### Commercial Use
Datasets with permissive licenses suitable for commercial training and deployment.
| Date | Paper | Size | Tasks | Notes | Data | License |
| ------- | ---------------------------------------------------- | ----------------------------- | ----------------------------------------------------- | ----------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------- |
| 2024-10 | [NovaChart](https://doi.org/10.1145/3664647.3680790) | 47K charts, 856K instr. pairs | 18 chart types, 15 tasks (understanding + generation) | [notes](2024-10-novachart.md) | [GitHub](https://github.com/Elucidator-V/NovaChart), [HuggingFace](https://huggingface.co/datasets/ympan/novachart) | MIT (code), Apache-2.0 (dataset) |
| 2024-04 | [TinyChartData](https://arxiv.org/abs/2404.16635) | 140K PoT pairs | Chart QA with program-of-thought learning | [notes](2024-04-tinychart.md) | [GitHub](https://github.com/X-PLUG/mPLUG-DocOwl), [HuggingFace](https://huggingface.co/datasets/mPLUG/TinyChartData) | Apache-2.0 |
| 2019-09 | [PlotQA](https://arxiv.org/abs/1909.00997) | 224K plots, 28.9M QA pairs | Plot question answering with OOV reasoning | [notes](2019-09-plotqa.md) | [GitHub](https://github.com/NiteshMethani/PlotQA) | MIT (code), CC-BY-4.0 (data) |
#### Research Use Only
Datasets with copyleft (GPL), non-commercial (CC-NC), or explicit research-only restrictions.
| Date | Paper | Size | Tasks | Notes | Data | License |
| ------- | -------------------------------------------------------------- | ------------------------------ | ----------------------------------------------------------------------------------- | -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ |
| 2024-04 | [OneChart / ChartY](https://arxiv.org/abs/2404.09987) | ~6K charts | Chart-to-dict structural extraction, bilingual (EN/ZH) | [notes](2024-04-onechart.md) | [Project](https://onechartt.github.io/), [GitHub](https://github.com/LingyvKong/OneChart) | Apache-2.0 (code); research use only |
| 2023-08 | [SciGraphQA](https://arxiv.org/abs/2308.03349) | 295K multi-turn, 657K QA pairs | Multi-turn scientific graph question answering | [notes](2023-08-scigraphqa.md) | [GitHub](https://github.com/findalexli/SciGraphQA), [HuggingFace](https://huggingface.co/datasets/alexshengzhili/SciGraphQA-295K-train) | Research only (Palm-2/GPT-4 terms) |
| 2023-08 | [VisText](https://aclanthology.org/2023.acl-long.401/) | 12,441 charts | Chart captioning with semantic richness (L1-L3) | [notes](2023-08-vistext.md) | [GitHub](https://github.com/mitvis/vistext) | GPL-3.0 |
| 2022-05 | [ChartQA](https://arxiv.org/abs/2203.10244) | 20,882 charts, 32,719 QA pairs | Chart question answering with visual and logical reasoning | [notes](2022-05-chartqa.md) | [GitHub](https://github.com/vis-nlp/ChartQA) | GPL-3.0 |
| 2022-03 | [Chart-to-Text](https://arxiv.org/abs/2203.06486) | 44,096 charts | Chart summarization: natural language text generation | [notes](2022-03-chart-to-text.md) | [GitHub](https://github.com/vis-nlp/Chart-to-text) | GPL-3.0 (+ source restrictions) |
| 2019-09 | [CHART-Infographics](https://doi.org/10.1109/ICDAR.2019.00203) | ~200K synthetic, 4.2K real | Chart classification, text detection/OCR, role classification, axis/legend analysis | [notes](2019-09-chart-infographics.md) | [Synthetic](https://tc11.cvc.uab.es/datasets/CHART2019-S_1), [PMC](https://tc11.cvc.uab.es/datasets/ICDAR-CHART-2019-PMC_1) | CC-BY-NC-ND 3.0 (S), CC-BY-NC-SA 3.0 (PMC) |
| 2018-04 | [DVQA](https://arxiv.org/abs/1801.08163) | 300K bar charts, 3.5M QA pairs | Bar chart question answering | [notes](2018-04-dvqa.md) | [GitHub](https://github.com/kushalkafle/DVQA_dataset) | CC-BY-NC 4.0 |
#### License Unclear or Mixed
Datasets with unspecified licenses or complex multi-source licensing. Verify terms before use.
| Date | Paper | Size | Tasks | Notes | Data | License |
| ------- | ------------------------------------------------------------------------------------------------- | ------------------------------ | ---------------------------------------------------------- | -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- |
| 2024-04 | [ChartThinker](https://arxiv.org/abs/2403.11236) | 595K charts, 8.17M QA pairs | Chart summarization QA | [notes](2024-04-chartthinker.md) | [GitHub](https://github.com/Notonion/ChartThinker), [HuggingFace](https://huggingface.co/datasets/ChartThinker/Chart-Sum-QA) | MIT (HF); sources include GPL/CC-NC |
| 2023-07 | [DePlot](https://doi.org/10.18653/v1/2023.findings-acl.660) | 516K plot-table, 5.7M QA pairs | Plot-to-table translation, chart question answering | [notes](2023-07-deplot.md) | [google-research](https://github.com/google-research/google-research), [HuggingFace](https://huggingface.co/google/deplot) | Apache-2.0 (model); mixed data licenses |
| 2023-05 | [UniChart](https://arxiv.org/abs/2305.14761) | 611K charts | Pretraining corpus: table extraction, reasoning, QA, summ. | [notes](2023-05-unichart.md) | [GitHub](https://github.com/vis-nlp/UniChart) | Varies by source (see notes) |
| 2023-04 | [ChartSumm](https://arxiv.org/abs/2304.13620) | 84K charts | Chart summarization with short and long summaries | [notes](2023-04-chartsum.md) | [GitHub](https://github.com/pranonrahman/ChartSumm), [Drive](https://drive.google.com/drive/folders/1HPsFUojoHctFD2AGuotRPKRz-o0jXfRJ) | Unspecified (no LICENSE file) |
| 2021-01 | [ChartOCR / ExcelChart400K](https://arxiv.org/abs/2010.05111) | 386,966 charts | Chart-to-table extraction: bar, line, pie | [notes](2021-01-chartocr.md) | [GitHub](https://github.com/soap117/DeepRule), [HuggingFace](https://huggingface.co/datasets/asbljy/DeepRuleDataset) | MIT (HF); crawled data, paper silent on license |
| 2018-04 | [Beagle](https://homes.cs.washington.edu/~leibatt/static/papers/CHI2018_cr_battle_02_05_2018.pdf) | 42K SVG | Visualization type classification | [notes](2018-04-beagle.md) | [UW](https://homes.cs.washington.edu/~leibatt/beagle.html) | MIT (code only); dataset license not stated |
* * *
### Mathematical Expression Recognition
Mathematical expression recognition addresses **printed**, **handwritten**, and **screen-captured** formulas with complex 2D spatial structure. The domain is characterized by large symbol inventories (101 classes in CROHME benchmarks, 245 in HME100K, extended vocabularies for LaTeX rendering) and structural relationships such as superscripts, subscripts, fractions, radicals, and matrix layouts.
Task families include:
- **Symbol recognition:** Isolated classification with reject options for non-symbol junk
- **Expression parsing:** Combined segmentation, classification, and structural relationship extraction
- **Image-to-LaTeX:** End-to-end conversion from formula images to markup
- **Matrix recognition:** Hierarchical evaluation at matrix, row, column, and cell levels
Evaluation typically measures expression-level exact match rates (ExpRate) alongside object-level metrics for symbol segmentation, classification, and spatial relation detection. CROHME benchmarks indicate structure parsing remains a bottleneck: 90% accuracy with perfect symbol labels versus 67% end-to-end. Recent large-scale datasets (UniMER-1M with 1M+ samples) target real-world complexity beyond clean academic benchmarks, including noisy screen captures, font inconsistencies, and long expressions (up to 7,000+ tokens).
Datasets are organized into three tiers based on licensing:
- **Commercial**: Permissive licenses (Apache-2.0, MIT, CC-BY) that allow commercial use
- **Research**: Non-commercial licenses (GPL, CC-NC) or explicit research-only restrictions
- **Unclear**: No license specified or mixed/complex licensing; verify before use
#### License Unclear or Mixed
Datasets with unspecified licenses or complex multi-source licensing. Verify terms before use.
| Date | Paper | Size | Tasks | Notes | Data | License |
| ------- | ----------------------------------------------- | ---------------------------------------- | -------------------------------------------------------------- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- |
| 2024-04 | [UniMER-1M](https://arxiv.org/abs/2404.15254) | 1,061,791 train; 23,757 test (4 subsets) | Image-to-LaTeX: printed, complex, screen-captured, handwritten | [notes](2024-04-unimernet.md) | [HuggingFace](https://huggingface.co/datasets/wanderkid/UniMER_Dataset), [OpenDataLab](https://opendatalab.com/OpenDataLab/UniMER-1M) | Apache-2.0 (HF tag); upstream sources have mixed licenses |
| 2024-04 | [MathWriting](https://arxiv.org/abs/2404.10690) | 626k total (230k human, 396k synthetic) | Online handwritten math expression recognition, image-to-LaTeX | [notes](2024-04-mathwriting.md) | [Google Storage](https://storage.googleapis.com/mathwriting_data/mathwriting-2024.tgz), [HuggingFace](https://huggingface.co/datasets/deepcopy/MathWriting-human) | CC-BY-NC-SA 4.0 |
| 2022-03 | [HME100K](https://arxiv.org/abs/2203.01601) | 74,502 train + 24,607 test images | Handwritten mathematical expression recognition | [notes](2022-03-san.md) | [GitHub](https://github.com/Phymond/HME100K), [Portal](https://ai.100tal.com/dataset) | Unspecified (no LICENSE file) |
#### Research Use Only
Datasets with copyleft (GPL), non-commercial (CC-NC), or explicit research-only restrictions.
| Date | Paper | Size | Tasks | Notes | Data | License |
| ------- | ----------------------------------------------------------------- | ------------------------------------------------------------- | ---------------------------------------------------------- | ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- |
| 2019-09 | [CROHME 2019 + TFD](https://ieeexplore.ieee.org/document/8978144) | 1,199 test expressions, 236 pages (TFD) | Handwritten math + typeset formula detection | [notes](2019-09-crohme-tfd.md) | [TC10/11 Package](https://tc101-demo.github.io/datasets/ICDAR2019-CROHME-TDF_1), [TFD GitHub](https://github.com/MaliParag/TFD-ICDAR2019) | CC-BY-NC-SA 3.0 |
| 2016-09 | [CROHME 2016](https://hal.science/hal-01374346) | 1,147 test expressions (Tasks 1/4), 250 test matrices | 4 tasks: formula, symbol, structure, matrix recognition | [notes](2016-09-crohme.md) | [TC10/11 Package](https://tc101-demo.github.io/datasets/) | CC-BY-NC-SA 3.0 |
| 2014-09 | [CROHME 2014](https://doi.org/10.1109/ICFHR.2014.75) | 986 test expressions (10K symbols), 175 matrices, 10K+9K junk | Symbol recognition with reject, expression, matrix parsing | [notes](2014-09-crohme.md) | [TC11](https://tc11.cvc.uab.es/datasets/CROHME-2014_2), [TC10/11 Package](https://tc101-demo.github.io/datasets/), [GitHub](https://github.com/DPRL/CROHME_2014) | CC-BY-NC-SA 3.0 |
* * *
### Handwriting Recognition
Handwriting recognition for natural language text focuses on word-level and line-level detection and recognition in unconstrained conditions. Unlike mathematical expressions, which require parsing 2D spatial structure, general handwriting tasks emphasize sequential text extraction from camera-captured images, historical documents, and field notes. Evaluation uses localization metrics (IoU-based measures) for detection and character/word accuracy rates for recognition.
Datasets are organized into three tiers based on licensing:
- **Commercial**: Permissive licenses (Apache-2.0, MIT, CC-BY) that allow commercial use
- **Research**: Non-commercial licenses (GPL, CC-NC) or explicit research-only restrictions
- **Unclear**: No license specified or mixed/complex licensing; verify before use
#### Commercial Use
Datasets with permissive licenses suitable for commercial training and deployment.
| Date | Paper | Size | Tasks | Notes | Data | License |
| ------- | ---------------------------------------------------------------------- | --------------------------------------- | ------------------------------------------------------ | ------------------------ | ------------------------------------------------------------------------------------------------ | --------- |
| 2021-09 | [GNHK](https://link.springer.com/chapter/10.1007/978-3-030-86337-1_27) | 687 images, 39,026 texts, 172,936 chars | Word-level detection and recognition (camera-captured) | [notes](2021-09-gnhk.md) | [GoodNotes](https://www.goodnotes.com/gnhk), [GitHub](https://github.com/GoodNotes/GNHK-dataset) | CC-BY 4.0 |
* * *
## Specialized Methods (No New Data)
Papers that introduce methods or training techniques but do not release new datasets. Included for completeness; see original papers for evaluation details.
### Chart Understanding
| Date | Paper | Notes | Weights | Code | License |
| ------- | --------------------------------------------- | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- | ----------------- |
| 2024-05 | [SIMPLOT](https://arxiv.org/abs/2405.00021) | [notes](2024-05-simplot.md) | | [GitHub](https://github.com/sangwu99/Simplot) | No license stated |
| 2024-04 | [TinyChart](https://arxiv.org/abs/2404.16635) | [notes](2024-04-tinychart.md) | [3B](https://huggingface.co/mPLUG/TinyChart-3B-768) | [X-PLUG/mPLUG-DocOwl](https://github.com/X-PLUG/mPLUG-DocOwl) | Apache 2.0 |
| 2023-05 | [UniChart](https://arxiv.org/abs/2305.14761) | [notes](2023-05-unichart.md) | [base](https://huggingface.co/ahmed-masry/unichart-base-960), [ChartQA](https://huggingface.co/ahmed-masry/unichart-chartqa-960) | [vis-nlp/UniChart](https://github.com/vis-nlp/UniChart) | MIT |
### Mathematical Expression Recognition
| Date | Paper | Notes | Weights | Code | License |
| ------- | --------------------------------------------- | ----------------------------- | ----------------------------------------------------------------- | ----------------------------------------------------------------- | ---------- |
| 2024-04 | [UniMERNet](https://arxiv.org/abs/2404.15254) | [notes](2024-04-unimernet.md) | [100M/202M/325M](https://huggingface.co/wanderkid/unimernet_base) | [opendatalab/UniMERNet](https://github.com/opendatalab/UniMERNet) | Apache 2.0 |
| 2022-03 | [SAN](https://arxiv.org/abs/2203.01601) | [notes](2022-03-san.md) | | Code not publicly released | --- |
### Evaluation & Metrics
| Date | Paper | Notes | Code | License |
| ------- | --------------------------------------- | ----------------------- | ------------------------------------------------------------------------------- | ---------- |
| 2024-09 | [CDM](https://arxiv.org/abs/2409.03643) | [notes](2024-09-cdm.md) | [opendatalab/UniMERNet](https://github.com/opendatalab/UniMERNet/tree/main/cdm) | Apache 2.0 |
### Handwriting Generation
| Date | Paper | Notes | Weights | Code | License |
| ------- | --------------------------------------------------------------- | ----------------------- | ------- | ---------------------------------------------------------------- | ---------------------------- |
| 2020-08 | [Decoupled Style Descriptors](https://arxiv.org/abs/2008.11354) | [notes](2020-08-dsd.md) | | [GitHub](https://github.com/brownvc/decoupled-style-descriptors) | Non-commercial research only |
* * *