LLMON support comes to Mellea v0.9.0
Mellea v0.9.0, released on November 12, 2026, adds support for LLMON (LLM Object Notation), which launched the same day as an experimental preview. LLMON is a markup language that models are trained on to keep instructions, data, and requirements in separate, labeled blocks, which reduces the risk of prompt injection. Mellea already knows which part of a component is which, so with an LLMON model it passes that structure through without you writing any markup.
The release also includes reliability fixes for validation, aLoRA adapters, and tool calling, plus smaller improvements. If you use SOFAI with a judge, register your own aLoRA, or write custom sampling strategies, read behavior changes before you upgrade.
LLMON support
With an LLMON model, your code looks the same as it does with any other model.
LocalHFBackend switches to the new LLMONFormatter automatically for the two
Granite preview models,
granite-4.1-3b-llmon-preview
and
granite-4.1-8b-llmon-preview,
which are also available as IBM_GRANITE_4_1_3B_LLMON_PREVIEW and
IBM_GRANITE_4_1_8B_LLMON_PREVIEW in mellea.backends.model_ids:
import mellea
from mellea.backends.huggingface import LocalHFBackend
from mellea.stdlib.context import ChatContext
MODEL_ID = "ibm-granite/granite-4.1-3b-llmon-preview"
m = mellea.MelleaSession(backend=LocalHFBackend(model_id=MODEL_ID), ctx=ChatContext())
thunk = m.instruct(
"Summarize this customer feedback in one sentence.",
grounding_context={"feedback": "loved the product, but shipping was very slow"},
)
print(thunk.value)
What the model receives
The formatter sends each part of a Mellea component in the message fields that
the model's chat template turns into LLMON. In the example above, the instruction
becomes an instr block and the feedback becomes a data block labeled
feedback. Requirements become req blocks, and documents attached to a
message become doc blocks. Content passing through the formatter is
sanitized, so if a retrieved page contains LLMON's own special tokens, it
can't close its data block and open an instruction of its own. The reply
comes back as plain text, with no markup to strip.
Structured output with format=, tool calling, and requirement validation all
work as usual.
Current limitations
Like the Granite models, Mellea's LLMON support is a preview, with a few limitations:
LocalHFBackendonly. Other backends, including vLLM and Ollama, rebuild each message from its standard keys and drop the fields LLMON needs. They raiseValueErrorfor an LLMON model rather than silently sending a prompt without its structure.- Not every component is supported.
@generativefunctions and instructions with anoutput_prefixraise an error. - Sanitization covers what passes through the formatter. A system prompt
set with
ModelOption.SYSTEM_PROMPT, tool definitions, and tool-call arguments reach the chat template as they are. - Use a strategy that repairs.
LocalHFBackenddecodes greedily, so the defaultRejectionSamplingStrategyresends the same prompt and gets the same answer. UseMultiTurnStrategyorRepairTemplateStrategywhen you add requirements. Check requirements that involve counting, such as an exact word count, with a validation function rather than the LLM judge.
To try it, install the llmon extra:
pip install "mellea[hf,llmon]"
The
LLMON concept page
shows exactly what the model receives, and the full example is
instruct_round_trip.py.
We'll take a closer look at LLMON with Mellea in a follow-up post.
Reliability fixes
- Async validators run.
Requirement.validatenow awaits an asyncvalidation_fn, so SOFAI'sjudge_backendand your own async validators return real verdicts, and repair and S2 escalation trigger when a requirement fails (#1729). - aLoRA adapters activate on
LocalHFBackend. An aLoRA you register yourself withadd_adapter()is now applied during generation (#1685). Mellea also checks that the adapter's invocation sequence appears in the prompt before the call. If it doesn't, a direct adapter call raisesAloraActivationError, and requirement routing falls back to LLM-as-a-judge (#1684). The defaultcore.requirement_check()andcore.check_certainty()wrappers use a LoRA and are unaffected; the adapter docs cover each path. - Granite 4.2 tool calls on
LocalHFBackend.parse_tools()reads Granite 4.2's XML tool-call format, so the tools run, and a malformed call logs a warning (#1692). - Tool arguments keep their types. A
list[int]arrives as integers, Pydantic model elements are validated against their fields, tool schemas carryitemsandFieldconstraints such asminimum, and optional parameters acceptnull(#1694).
Other improvements
- Prompt templates in traces. Chat generations emit the OpenInference
llm.prompt_template.templateattribute, plusllm.prompt_template.variableswhenMELLEA_TRACES_CONTENT=true, so a trace shows the template separately from the values substituted into it; see the tracing docs (#1672). - One sampling lifecycle per call. Every top-level
sample()now emits exactly one sampling lifecycle, whatever the strategy, so hooks and traces see SOFAI, budget forcing, and majority voting the same way they see the default loop (#1584). - No more empty prompts. OpenAI, LiteLLM, and Ollama raise
ValueErrorbefore sending a conversation with no user content, instead of letting the model ramble at an empty prompt (#1632). - Ollama model lookup.
is_model_available()matches tags case-insensitively and in full, sogranite4.1:3bno longer matches an installedgranite4.1:3b-instruct-q4(#1691). - OpenAI
reasoning_effort.reasoning_effort="none"is now omitted for real OpenAI servers, which reject it for non-reasoning models (#1709). - Adapter removal on OpenAI.
OpenAIBackend.remove_adapter()unregisters an adapter and frees its name for reuse (#1676). - One
extra_bodymerge. All three merge paths now share the publicModelOption.merge_extra_body(#1695). - Docs and examples. A new example,
llamacpp_session.py, runs Mellea against a local llama.cpp server, with a helper script to start one (#1656). Example READMEs link their files (#1711), the vision and Ollama examples run reliably in nightly CI (#1671), and the "Making Agents Reliable" tutorial handles tool results correctly (#1803).
Behavior changes
Some changes in this release can affect existing code.
| What changed | Who is affected | What to do | PR |
|---|---|---|---|
SamplingStrategy.sample() is @final. Custom strategies implement _sample() instead. | Anyone who wrote their own sampling strategy | Rename your sample() override to _sample(); see building extensions | #1584 |
| Majority voting emits one lifecycle for the whole vote, not one per inner sample. | Hook or trace consumers of majority voting | Expect one sampling_loop_start/end pair per call | #1584 |
| Async validators and SOFAI judges really run, so outputs that miss a requirement now fail. | SOFAI with judge_backend, async validation_fn users | Expect success: False where you saw passes before; those passes were never checked | #1729 |
A direct adapter-function call on an aLoRA that can't activate raises AloraActivationError. | Self-registered aLoRAs on LocalHFBackend | Catch it, or fix the prompt so the invocation sequence appears | #1684 |
Chat requests with no non-empty user content raise ValueError. | OpenAI, LiteLLM, and Ollama callers passing an empty action | Pass real content in the action | #1632 |
A non-dict chat_template_kwargs raises TypeError instead of being silently dropped. | Anyone passing chat_template_kwargs through extra_body | Pass a dict | #1695 |
Upgrading
pip install --upgrade mellea
To try LLMON, add the extra: pip install --upgrade "mellea[hf,llmon]". Check
behavior changes first, especially if you write custom
sampling strategies or rely on SOFAI's judge. The
full release notes
list all 29 PRs, including first contributions from three new people.