# GNHK: A Dataset for English Handwriting in the Wild > Camera-captured English handwriting dataset with word-level annotations for in-the-wild text detection and recognition - Authors: Hunter Heidenreich - Published: 2021-09-01 - Updated: 2026-03-04 - Paper: GNHK Paper - Dataset: GNHK Dataset - Code: GNHK GitHub - Source: Bevaya Labs, --- ## TL;DR GNHK introduces a camera-captured, in-the-wild English handwriting dataset (687 images) with word-level quadrilateral annotations, line grouping, and handwritten-versus-printed tags. The paper provides baseline results for text localisation (Mask R-CNN / Faster R-CNN) and cropped-word recognition (Clova scene-text recognizer variants), with best reported recognition reaching CAR 0.861 and WAR 0.502 under TPS + BiLSTM + attention decoding. * * * ## What kind of paper is this? **Dominant:** $\\Psi_{\\text{Resource}}$ (dataset release + benchmark baselines) **Secondary:** $\\Psi_{\\text{Evaluation}}$ (establishes baseline protocols and metrics for localisation and recognition) **Justification:** The headline contribution is the dataset itself ("we created a dataset...") plus accompanying baselines, which matches the taxonomy guideline that dataset/tool releases are typically $\\Psi\_{\\text{Resource}}$-dominant. ## What is the motivation? - Existing widely-used handwriting datasets (e.g., IAM, RIMES) are largely flatbed-scanned and do not reflect camera-captured "in the wild" conditions. - Prior scene-text benchmarks are mostly printed text; the paper argues there is a gap for offline English handwriting captured via cameras under unconstrained conditions. - English handwriting varies across regions (lexicon and styles), motivating collection across Europe, North America, Asia, and Africa. ## What is the novelty? - A camera-captured English handwriting dataset modeled after scene-text datasets, explicitly including "in the wild" document types (shopping lists, sticky notes, diaries) and non-handwriting content (printed text, images). - Annotation schema designed for both detection/localisation and recognition: - Per-image JSON annotations containing multiple objects with `(text, polygon, line_idx, type)`. - Text values drawn from ASCII printable characters plus the British pound sign, with no whitespace characters included. - Three special tokens for polygon annotations: `%math%` (math expressions), `%SC%` (illegible scribbles), `%NA%` (no characters/math symbols). - Quadrilateral polygons listed clockwise starting at the top-left point. - `line_idx` groups texts that belong to the same line; `type` indicates handwritten vs printed. ## What experiments were performed? ### Text localisation (detection/segmentation) - Baselines: Mask R-CNN (instance segmentation) and Faster R-CNN (object detection), implemented in detectron2. - Evaluation: recall, precision, and $F$-measure at IoU $> 0.5$. ### Text recognition (cropped word recognition) - Benchmark setup: use ground-truth word boxes to crop word images, then recognize text from crops (segmented recognition). - Model family: Clova AI deep text recognition framework with four components (transformation, feature extraction, sequence modelling, prediction), evaluating eight configurations. - Data filtering for recognition eval: keep words and punctuation; remove unknown characters, scribbles, math symbols, and words that contain only punctuation. ## What are the outcomes/limitations? ### Key outcomes - Localisation: both Mask R-CNN and Faster R-CNN achieve $F$-measure $> 0.86$ (IoU $> 0.5$), with high precision in both cases. - Recognition: best reported configuration (TPS + BiLSTM + attention) reaches CAR 0.861 and WAR 0.502; attention decoding substantially outperforms CTC, and TPS improves CAR/WAR in the reported comparisons. ### Limitations and open ends (from the paper's setup) - Recognition benchmark is not end-to-end: it assumes ground-truth word crops rather than predicted boxes. - For localisation, the dataset lacks pixel-level masks separating word vs non-word; the baseline uses polygon-to-box conversion (min/max over polygon points) for the R-CNN box regression. - The paper explicitly points to future work on end-to-end approaches that do localisation and recognition sequentially in a single framework. * * * ## Model ### Localisation - Mask R-CNN baseline; bounding boxes derived from polygon min/max in $x$ and $y$, with the polygon used as the segmentation mask. - Implementation: detectron2; backbone ResNet-50 with FPN, pretrained on ImageNet; network pretrained on MS COCO for segmentation. - Comparator: Faster R-CNN in the same detectron2 framework. ### Recognition - Clova AI deep text recognition framework with configurable components: - Transformation: TPS vs none - Sequence modelling: BiLSTM vs none - Prediction: attention vs CTC ## Data ### Dataset size and composition - Total: 687 images, 172,936 characters, 39,026 texts, 9,363 lines. - "Texts" include words, ASCII symbols, and math expressions (via tokenization rules). - Regional sourcing across Europe, North America, Asia, and Africa. - Region-level counts (chars/texts/lines): EU 58,982 / 13,592 / 3,306; NA 47,361 / 10,967 / 2,099; AS 39,593 / 8,586 / 2,780; AF 27,000 / 5,881 / 1,178. - Text statistics: 39,026 total texts, 12,341 unique; median texts per image 57; mean 44.1. - Character set: 96 unique characters; median character count 486; mean 1,801; max per-character frequency up to 17,887. - Collection constraint: no more than 5 images per writer. ### Annotation format - Per-image JSON file with objects keyed by `text`, `polygon`, `line_idx`, `type`. - `polygon` is a quadrilateral with four $(x,y)$ points in clockwise order starting at top-left. - `type` indicates handwritten or printed. ### Splits - Train/test split: 75% training, 25% testing. ### Download and license - **Official download:** GoodNotes hosts the dataset with links (Google Drive / Baidu Netdisk) available after agreeing to terms and conditions on the [GNHK webpage](https://www.goodnotes.com/gnhk). - **Pointer repository:** The [GoodNotes/GNHK-dataset GitHub repo](https://github.com/GoodNotes/GNHK-dataset) points to the official dataset page. - **License:** Creative Commons Attribution 4.0 (CC BY 4.0). You can use, share, and adapt the dataset (including commercially) as long as you provide attribution and include the license notice (and indicate if you made changes). - **Note:** Third-party mirrors exist (e.g., on Hugging Face), but the GoodNotes page is the authoritative source. ## Algorithms / Training - Localisation: Mask R-CNN trained in detectron2 with ResNet-50 + FPN backbone; initialization includes ImageNet pretraining and MS COCO pretraining for segmentation. - Recognition: segmented recognition using ground-truth crops, evaluated across eight Clova framework configurations (TPS/none × BiLSTM/none × attention/CTC). Training hyperparameters such as learning rate, batch size, and epochs are not specified in the paper. ## Evaluation ### Localisation metrics and results - **Metric:** Recall, precision, and $F$-measure with IoU $> 0.5$. - **Results** (Table 5): - Mask R-CNN: recall 0.8237, precision 0.9079, $F$-measure 0.864 - Faster R-CNN: recall 0.8077, precision 0.9215, $F$-measure 0.860 - **Qualitative note:** $F$-measure differences are described as coming from high precision (fewer false positives). ### Recognition metrics and results - **Metrics:** - Character accuracy rate (CAR): average over words of $1 - \\frac{\\text{edit distance}(gt_i, pred_i)}{N_i}$ - Word accuracy rate (WAR): fraction of words with zero edit distance - **Results** (Table 6): best configuration TPS + BiLSTM + attention gives CAR 0.861 and WAR 0.502. - **Comparative findings:** - Attention decoding outperforms CTC across the reported configurations. - TPS improves CAR and WAR in like-for-like comparisons. - BiLSTM generally improves CAR; the paper notes a case where WAR decreases when adding BiLSTM without TPS (0.377 vs 0.430). ## Hardware / Production Not specified.