Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps

This Hugging Face blog is a public recipe, not a frontier-lab result. The authors fine-tune LiquidAI's LFM2.5-350M with Group Relative Policy Optimization in TRL, evaluate on Liquid AI's IFStruct schema-compliance benchmark, and publish a notebook sized for a free Colab or Kaggle GPU. About 500 samples and 100 steps take the local llama.cpp/BF16 baseline from 452/2000 passed (22.6%) to 594/2000 (29.7%). The IFStruct blog had reported 21.1% for the base model; they use 22.6 as their serving-stack baseline.

Training data is nvidia/Nemotron-RL-instruction_following-structured_outputs, augmented so 40% of prompts require fenced code blocks and a disjoint 20% become top-level arrays. LoRA targets LFM hybrid attention/convolution modules. Three rewards score JSON form, field count, and schema validation. After training, JSON pass rate rises from 18.0% to 31.9%; YAML is mostly unchanged. Bare-list pass rate jumps from 16.6% to 29.7%, matching wrapper-key. The tuned 350M still sits below Qwen3.5-2B's 33.15% on IFStruct, which is the point: light, cheap, task-specific RL closes much of a size gap on format.

Vendor-adjacent tutorial, and a good one. Hugging Face is selling TRL and the idea that small models plus GRPO are enough for the boring production problem — valid JSON — that everyone pretends is solved. 29.7% overall is still a failing grade if the downstream system cannot tolerate garbage. The win is relative and localized: fences, lists, field counts. Required-field-missing is still the dominant error (7,331 times).

Do not read this as '350M beats 2B.' It does not. It approaches a 2B on one structured-output board after the 2B's general training, using a reward that only cares about form. That is still a useful production pattern. Most teams overbuy intelligence for schema adherence.

Signal Desk's read: the industry keeps folding structured output into 'reasoning' evals and then wondering why agents break parsers. IFStruct isolating schema compliance is the correct measurement. A 100-step LoRA that moves JSON by 14 points on a 16GB GPU is the sort of unglamorous result that actually ships. The remaining miss rate says you still need a validator in front of the database.

Nemotron data plus a Liquid model plus HF training code is also a stack story. Everyone's parts, one notebook, no 70B.

Structured outputs are how models talk to software. Benchmarks usually hide format failures inside broader scores. IFStruct and this recipe exist because a 22% valid-JSON model cannot be an API, no matter how clever it sounds in chat.