# GLM-4.5V and GLM-4.1V-Thinking

> Multimodal RL with curriculum training achieves GLM-4.1V-9B-Thinking outperforming Qwen2.5-VL-72B across coding, math, OCR, and GUI tasks

- Authors: Ben
- Published: 2025-08-22
- Source: Bevaya Labs, <https://labs.bevaya.ai/>

---


# GLM Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
**Paper**: [GLM-4.5V and GLM-4.1V-Thinking](https://arxiv.org/abs/2507.01006)  
**Presenter**: Yosh  
**Note Taker**: Ben  
**Date**: August 22, 2025  

---

## Overview
Presents a multimodal reinforcement learning approach aimed at broad coverage across coding, math, GUI agents, visual grounding, and OCR. Reports that GLM-4.1V-9B-Thinking outperforms larger models such as Qwen2.5-VL-72B. The training recipe combines curriculum-based RL with a vision–language stack updated for long sequences and varied image aspect ratios.

---

## Novelty
- Curriculum RL: training data organized by difficulty, estimated via pretrained models’ pass@k; intent is to keep examples neither trivial nor infeasible as the model strengthens.  
- Architecture:  
  - 3D convolutions to enable temporal downsampling.  
  - 2D-RoPE integrated into ViT self-attention.  
  - Aspect-ratio handling by normalizing images to the ViT’s absolute-position grid via bicubic interpolation.  
- Data packing: concatenates variable-length samples to approach maximum context length.  
- Training pipeline:  
  - Cold start with long chain-of-thought SFT before RL.  
  - RL phase includes RLVR and RLHF.  
  - KL term removed from the loss.  
  - Rewards provided either by extracting answers from structured responses or by using an LLM evaluator.

---

## Learnings
- Curriculum sampling is used to address the observation that easy items late in training give little signal, while hard items too early impede learning.  
- Chain-of-thought SFT is used for initialization, then RL refines behavior.  
- Removing the KL term is part of the reported setup; alignment proceeds without an explicit reference-model regularizer.  
- Vision-side changes (3D conv, 2D-RoPE, aspect-ratio normalization) are presented as enabling more stable multimodal processing.  
- Pretraining covers captioning, interleaved image–text, OCR, visual grounding, and instruction-tuning sources; long-sequence packing is used to maximize utilization.


---

## Notes for Application
- Curriculum RL (difficulty-ranked data with pass@k estimation) is a practical template for multi-task training where some tasks (e.g., rare form types) are harder than others; worth applying to document extraction RL runs.
- The cold-start SFT → RL pipeline mirrors our standard approach; the reported KL-free RL variant suggests it may be viable to drop the reference-model regularizer once cold-start quality is high enough.
- GLM-4.1V-9B-Thinking's strong OCR performance at 9B parameters is directly relevant: a compact multimodal model with RL training can match or exceed much larger models on OCR-centric tasks.
- The 3D conv and 2D-RoPE vision changes address aspect-ratio and temporal consistency, worth tracking if we move toward video or multi-page document inputs.