IBM’s LLMON markup lets small Granite models tell instructions from data, cutting prompt injection

IBM’s Granite team released two experimental preview models, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, trained on a new markup called LLMON, short for LLM Object Notation. Instead of flattening instructions, documents, tool output and constraints into one long string, every piece gets a labeled block, and a separate execution block says which instruction runs on which data. The release includes a GitHub toolkit with lossless JSON-to-LLMON converters, native support in IBM’s Mellea library and an attention visualization demo on Hugging Face Spaces.

IBM says the models were post-trained from Granite 4.1 base checkpoints on a re-serialized instruction corpus that included deliberately unbound decoy instructions, followed by several GRPO reinforcement stages and a final DPO stage built on the model’s own responses to injection attempts. A ScienceWorld demo shows teams defining their own tag vocabulary and tuning a LoRA adapter on top.

When an agent reads a web page or tool response, a hidden command in that data can hijack it. The industry has spent two years asking models to behave safely while feeding them context with no reliable way to tell a command from a quotation. IBM argues JSON was built for parsers, not language models: a quote mark inside a document and a quote mark closing a field look identical to a model. Fixing the format rather than adding another filter is the right instinct. Prompt injection has been a weak point even for frontier systems, which is why claims like Google’s for Gemini 4 Carbon deserve independent testing (our analysis).

IBM says its LLMON models mostly ignore planted instructions, with attack success on some benchmarks dropping to nearly zero, while general capability stays on par with the regular Granite 4.1 Instruct models. On the OPI benchmark, the previews avoided 47% more attacks than the best frontier model tested and 95% more than the best open-source model, with similar trends on ASB and StruQ. If that holds under outside testing, it is among the more meaningful security results for small, cheap models, the ones companies want to run inside agents at scale. IBM Research’s Palette work on small models building on-brand decks shows the same focus on making small models dependable (coverage).

These are static injection benchmarks; adaptive multi-turn attacks are still being worked on. The privilege rule, under which executed instructions are authoritative and everything else is material to read, is learned, not enforced. The models are labeled experimental previews, not for production. A learned defense can be learned around, so no team should treat this as a security boundary yet.

LLMON also adds reserved special tokens to the tokenizer, so its benefits exist only in models trained for it. That makes it a format bet as much as a model release. The concept, explicit token-level separation of instruction and data, will likely outlast this syntax. Agents that can read untrusted email, web pages and tool results without treating them as orders are a precondition for useful automation. More in our cybersecurity coverage and research coverage.