# Visual-RFT > GRPO-style reinforcement fine-tuning applied to vision-language models for classification and detection - Authors: Hunter - Published: 2025-03-07 - Source: Bevaya Labs, --- # Visual-RFT **Paper**: [Visual-RFT](https://arxiv.org/abs/2503.01785) **Presenter**: Yosh **Note Taker**: Hunter **Date**: March 7, 2025 ## Discussion Notes - DeepSeek GRPO style RL applied to VLMs (open vocab classification, object detection). - Long tail / rare object detections are learned more easily with RFT. This should apply to our use cases with generalization to new forms (ACORDs and more generally). Also, this approach would help with our checkbox detection use cases, even when checkboxes are funky looking. - They have the model produce confidence scores as numbers in generations and then parse those numbers, incorporating it in their reward. A cool idea to try, beyond just visual applications. - We talked about having a smaller discretization than in this paper. Seems to be a bit extreme to do 100 bins. However, the idea makes practical and intuitive sense. You can't do this with SFT, but you can with RFT. - No need for differentiable rewards, so can easily incorporate confidence. In fact, one can have an "episode" where many predictions are done and ECE is minimized. Would require a batch based reward possibly? But GRPO is already configured in groups, so may be straightforward. - Seems a straightforward way to get better document-level and table-level confidence scores. For tables, we could also analyze things row-wise and column-wise. - Stark differences in performance between RFT and SFT. This is a good sign for us. The 2B parameter model makes massive gains with RFT. Smaller models can become quite capable with RFT. When scaling up model size, the gap between SFT and RFT shrinks but is still significant.