# Decoupled Style Descriptors > Explicit linear factorization of handwriting style for controllable generation - Authors: Hunter Heidenreich - Published: 2020-08-26 - Updated: 2026-03-04 - Paper: DSD Paper - Code: DSD Code & Models - Source: Bevaya Labs, --- ## TL;DR Proposes **Decoupled Style Descriptors (DSDs)** that explicitly factor handwriting into a writer style vector $w$ and a content-conditioned character matrix $C_{ct}$, producing writer-character descriptors $w_{ct}=C\_{ct}w$. Human preference studies show **88%** preference over DeepWriting, with writer identification accuracy reaching **99.70%** on 50-word samples. * * * ## Five Key Questions ### What kind of paper is this? **Dominant: $\\Psi\_{\\text{Method}}$.** Introduces a new modeling construction for handwriting generation: $w_{ct}=C_{ct}w$ with an explicit invertible content matrix to factor out writer style. **Secondary: $\\Psi\_{\\text{Evaluation}}$.** Runs human similarity judgments against DeepWriting and analyzes invertibility, writer ID performance. **Secondary: $\\Psi\_{\\text{Resource}}$.** Introduces the BRUSH dataset with design and collection details. ### What is the motivation? Classic encoder-decoder handwriting models learn writer-dependent and character-dependent latent vectors $w\_{ct}$ but do not provide a mechanism to represent character-independent writer style directly. They argue explicit structure is needed to extract writer style cleanly. ### What is the novelty? **Linear latent factorization:** $$w_{ct} = C_{ct} w,\\quad w = C_{ct}^{-1} w_{ct}$$ assuming $C\_{ct}$ is invertible. **Sequence-aware $C$ construction:** $C_{ct}$ is predicted from character substrings using a character encoder $g_\\theta$ that embeds characters, passes them through an LSTM to capture history, then maps to a $256\\times256$ matrix. **Writer style estimation by averaging per-prefix inversions:** for a sample producing multiple $w_{ct}$ across prefixes, they estimate $$w = \\frac{1}{M}\\sum_{t=1}^{M} C^{-1}_{ct} w_{ct}.$$ **Multiple synthesis routes:** producing $w\_{ct}$ via the encoder, or via "Method $\\alpha$" (mean writer style) and "Method $\\beta$" (sampling from a database of writer-character DSDs). **Unsupervised character segmentation ($k\_\\theta$):** trains a segmentation network with a modified CTC-style objective so stroke points can be attributed to characters and end-of-character indices can be identified. ### What experiments were performed? **Human similarity preference on IAM vs DeepWriting:** 25 participants on MTurk, 40 sentence-level target samples, 15 assessments per case totaling 600, with randomized ordering. **Writer identification:** reports **89.38%** from a single word and **99.70%** from 50 words on holdout writers. **Invertibility checks:** explicitly evaluates matrix ranks for $C\_{ct}$ for 1-, 2-, and 3-character strings, noting a small number of rare non-invertible 3-character cases. ### What are the outcomes/limitations? **Outcomes** Generated handwriting achieves **88%** human preference over the baseline. The sampling variant is chosen **5.22x more often** as most similar to target. Strong writer-identification performance demonstrates the learned style vectors capture meaningful writer characteristics. Structural decoupling via $C$ performs better than style-transfer baselines where the network must implicitly separate content and style. **Limitations / failure modes** Delayed strokes, such as dotting an _i_ after completing the letter body, are challenging for substring-based representations. Missed delayed strokes shown as a limitation case. Relies on segmentation quality (predicting end-of-character indices); errors propagate into $w\_{ct}$ extraction and synthesis. * * * ## Reproducibility Details ### Model **Input stroke representation:** each point $p_t$ stores $(\\Delta x_t, \\Delta y_t)$ and an end-of-stroke flag $eos\\in{0,1}$, giving $x\\in\\mathbb{R}^{(N,3)}$. **Outputs (decoder):** MDN parameters $(\\pi_t,\\mu_x,\\mu_y,\\sigma_x,\\sigma_y,\\rho)$ plus probabilities for $eos$ and $eoc$ (end-of-character). **Core latent variable shapes:** $w\\in\\mathbb{R}^{256}$, $C_{ct}\\in\\mathbb{R}^{256\\times256}$, $w_{ct}\\in\\mathbb{R}^{256}$. **Character encoder $g_\\theta$ for producing $C_{ct}$:** FC layer $\\rightarrow$ LSTM $\\rightarrow$ FC mapping to 65,536 dims and reshape to $256\\times256$; FC2 layer is approximately one-third of total parameters. **Segmentation network $k\_\\theta$:** bidirectional LSTM taking 23 features per point; trained with a modified CTC-style loss to label each point with a character and find eoc indices. ### Data **BRUSH dataset design:** baseline shown in every drawing box so the initial action includes the shift from baseline to start point; 170 individuals; 488 common words across 192 sentences. **Character inventory:** 86 characters (space + 85 listed including digits, upper/lowercase, punctuation/symbols). **Collection constraints:** 170 writers on MTurk; 60-minute time limit; word selection based on 3,036 Gutenberg books to cover character pairs. **Coverage:** selection achieves **99.9%** coverage of character pairs with 3,894 pairs. **Cleaning / ordering:** correcting missed delayed strokes by adding them back and sorting strokes left-to-right. **License:** Dataset may only be used for non-commercial research purposes; contact Prof. James Tompkin for other uses. **Download:** [Google Drive ZIP](https://github.com/brownvc/decoupled-style-descriptors) (566 MB) linked from repository README. ### Algorithms / Training **Training uses teacher forcing in the decoder:** during training feeds true point sequences; at runtime feeds predicted points. **Loss (decoder location term):** negative log-likelihood of the next point under the MDN mixture distribution, with additional EOS/EOC terms. **Invertibility enforced implicitly:** $L\_{wct}$ terms penalize failures where $CC^{-1}\\neq I$, discouraging singular $C$. **Backprop through matrix inverse:** uses $\\frac{dC^{-1}}{dx}=-C^{-1}\\frac{dC}{dx}C^{-1}$ to enable end-to-end training. **Optimization + key hyperparameters:** Adam, learning rate 0.001, gradient clipping to $[-10,10]$, 5 sentence-level samples per batch, 3-layer stacked LSTMs. **Sampling Method $\\beta$ (database-driven):** procedure to build/query a database $D$ of writer-character DSDs keyed by substrings and sample $w\_{ct}$ for synthesis. ### Evaluation **Human study protocol:** 25 MTurk participants; 40 sentence-level targets; 15 assessments per case; randomized ordering; total 600 assessments. **Primary reported preference signal:** sampling method selected 5.22x more often as "most similar." **Writer ID metrics:** 89.38% from a single word and 99.70% from 50 words. ### Hardware / Production **Compute constraints:** baseline comparisons adjusted training settings due to 8GB VRAM GPU constraint, requiring batch size reduction. * * * ## License & Availability **Code license:** Brown University copyright; non-commercial use only. Grants permission for use/copy/modify/distribute except incorporation into a commercial product or service. **Dataset license:** BRUSH dataset restricted to non-commercial research purposes per repository README. Commercial or other uses require contacting the authors.