IBM Releases LLMON: A Markup Language That Makes AI Models More Reliable and Resistant to Prompt Injection
Summarized by AI from reporting by Hugging Face Blog, published under our editorial policy.
IBM has introduced LLMON (LLM Object Notation), a markup language that explicitly structures conversation components for AI models. The Granite 4.1 LLMON preview models show near-zero success rates for indirect prompt injection attacks while maintaining general capability.

Key takeaways
- IBM released LLMON (LLM Object Notation), a markup language that explicitly structures conversation components for AI models using reserved special-token delimiters.
- The Granite 4.1 LLMON preview models (3B and 8B) show near-zero attack success rates for indirect prompt injections while maintaining general capability relative to Granite 4.1 Instruct models.
- LLMON uses unique IDs to bind specific instructions to specific data payloads, preventing unrelated passages or old chat turns from being pulled into the answer.
- IBM provides conversion tools and example notebooks that translate JSON to LLMON and back in a lossless manner.
- LLMON enriches chat templates by telling the model what each component means and how components relate to each other, within and across roles.
IBM has released LLMON (LLM Object Notation, pronounced "lemon"), a markup language designed to make large language models more reliable by explicitly communicating structure and semantics. The first models to support LLMON are two experimental previews derived from Granite 4.1: granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview. IBM has also released a GitHub repository with a toolkit for working with LLMON and examples of workflows and templates.
How LLMON Structures AI Context
LLMs rely on their context to generate the next set of tokens. Over a long trajectory, the context fills with numerous types of components: a system prompt, past conversation turns, instructions that act on retrieved documents, tool responses or user data, requirements constraining the allowed output, and reasoning chains. The context carries a rich structure, most of which is either implicit to the model or encoded in formats such as JSON that were not designed to be digested by LLMs. This is one of the main reasons LLMs, particularly those on the smaller end of the scale, are brittle and sometimes fail in unexpected ways. The lack of explicit structure can also be exploited by attackers, for example, in the form of prompt injections.
LLMON addresses this by marking semantically coherent components with meaningful tags wrapped in reserved special-token delimiters. Data schemas, including nesting, are stated in a way that is unambiguous to a causal LLM. Unique IDs make it possible to point to specific instructions, which could reside anywhere in the context, and bind them to the data payloads they should act on, analogous to the way functions are called and arguments are passed to them in programming languages.
Near-Zero Attack Success Rates for Prompt Injections
Indirect prompt injections, hidden in user data, are mostly ignored by LLMON models due to explicit instruction/data separation. For some benchmarks, attack success rates drop to nearly zero, whereas regular instruction-tuned models get hijacked with high probability. This is achieved while maintaining general capability relative to Granite 4.1 Instruct models.
Unrelated passages, old chat turns, and tool outputs don't get pulled into the answer because each instruction is bound to the exact data it applies to. Instructions, data, and requirements are clearly marked so that their structure and meanings at the application's code level can be leveraged by the model.
LLMON vs. JSON: Designed for Models, Not Parsers
JSON was designed for parsers, not for causal language models. The tokenizer is not constrained to respect the grammar, and structure is encoded by tokens ({, ", :) that may also appear in normal text. To a model, a quotation mark inside a document and a quotation mark that closes a field are the same token. Escaping rules that are clear to a parser are only statistical patterns to a model. Nesting depth is encoded by counting brackets, which transformers are known to be weak at, especially over long spans.
LLMON, on the other hand, is an XML-like markup whose structural delimiters are reserved special tokens rather than characters, so structure and content use separate parts of the vocabulary. Tags are stated at the beginning of each span, and the boundaries between spans are explicit. Nested structures are stated locally so the model does not need to keep track of the hierarchy over long spans.
Since most teams already have pipelines built around JSON, IBM created conversion tools and example notebooks that translate JSON to LLMON and back in a lossless manner.
LLMON Enriches Chat Templates, Doesn't Replace Them
LLMON does not replace chat templates; rather, it enriches them. Where regular chat templates tell the model which part belongs to which role (system, user, assistant), LLMON tells it what each component means and how components relate to each other, within and across roles.
The LLMON preview models ship with their own chat template, so you build the same messages list you would pass to any chat model and call apply_chat_template as usual. The only difference is that a turn can now name its parts and mark each one exec: True or False. A part marked True is bound into the exec block at the bottom of the rendered output, which states which instruction to run and what it operates on; a part marked False is still rendered into the context but is referenced by nothing.
The two-character block IDs are drawn from a fixed set that this tokenizer encodes as single tokens, so addressing a block always costs two tokens regardless of which ID is drawn.
How to Try LLMON Today
You can try LLMON in Mellea, which has native LLMON support enabled by automatically LLMONizing its LLM calls. You no longer need to worry about forcing LLMs to observe JSON structure. Use IBM's converters to turn existing JSON into LLMON without loss. You can also define your own structure and associated tags and tune a LoRA on top of the LLMON models to understand it.
Frequently asked
- What is LLMON and how is it different from JSON?
- LLMON (LLM Object Notation) is a markup language whose structural delimiters are reserved special tokens rather than characters, so structure and content use separate parts of the vocabulary. JSON was designed for parsers, not for causal language models, and its structure is encoded by tokens that may also appear in normal text.
- How does LLMON prevent prompt injection attacks?
- LLMON prevents prompt injections by explicitly separating instructions from data using unique IDs that bind each instruction to the exact data it applies to. Indirect prompt injections hidden in user data are mostly ignored, reducing attack success rates to nearly zero on some benchmarks.
- Can I use LLMON with my existing JSON data?
- Yes, IBM provides conversion tools and example notebooks that translate JSON to LLMON and back in a lossless manner, making it easy to integrate LLMON into existing workflows.
- Which models support LLMON?
- IBM has released two experimental preview models: granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, both derived from Granite 4.1 base models and trained to consume and generate LLMON-marked text.