Context management using Mellea sessions
Component
s themselves, which generally contain all of the context needed for a single-turn request. MObjects manage context using fields and methods, and Instructions have a grounding_context for RAG-style requests.
Context
, which stores and represents a (sometimes partial) history of all previous requests to the LLM made during the current session.
Context
mechanism.
When you use the start_session()
method, you are actually instantiating a Mellea
with a default inference engine, a default model choice, and a default context manager. The following code is equivalent to m.start_session()
:
SimpleContext
— which is the only context we have used so far — is a context manager that resets the chat message history on each model call. That is, the model’s context is entirely determined by the current Component. Mellea also provides a LinearContext
, which behaves like a chat history. We can use the LinearContext to interact with chat models:
Context
object provides a few useful helpers for introspecting on the current model context; for example, you can always get the last model output: