Backends
mellea.backends.formatter
Module: mellea.backends.formatter
Abstract interfaces for Formatters.
Functions
mellea.backends.formatter._simplify_model_string(input: str)
Removes special chars from the given string and lower cases it to simplify matching.
mellea.backends.formatter._get_package_name(module: str)
Given a module, attempts to get the package and verifies it exists.
Classes
class mellea.backends.formatter.Formatter()
A Formatter converts Components into strings and parses ModelOutputThunks into Components (or CBlocks).
Methods
mellea.backends.formatter.Formatter.print(c: Component | CBlock)
Renders a component for input to a model.
mellea.backends.formatter.Formatter.print_context(ctx: Context)
Renders a Context for input to a model.
mellea.backends.formatter.Formatter.parse(source_component: Component | CBlock, result: ModelOutputThunk)
Parses the output from a model.
mellea.backends.formatter.Formatter.to_chat_messages(cs: list[Component | CBlock])
Helper method that converts a linearized chat history into a list of messages. The purpose of this helper is to prepare a sequence of Messages for input to a chat endpoint.
class mellea.backends.formatter.TemplateFormatter(model_id: str | ModelIdentifier, template_path: str = '', use_template_cache: bool = True)
Formatter that uses jinja2 templates.
Constructor
A TemplateFormatter use jinja2 templates.Arguments
model_id: Describes the model for which templates will be looked up. Should match the template dir structure.template_path: Specify an alternate location where templates can be found. Will be preferred over all other template dirs even if a less exact match is found.use_template_cache: Cache the location of the most recent templates so that future lookups don’t need to be performed. Set to false if you plan on changing the model_id or template_path after the TemplateFormatter has been created.
Methods
mellea.backends.formatter.TemplateFormatter.parse(source_component: Component | CBlock, result: ModelOutputThunk)
Parses the output and updates the result’s parsed_repr.
mellea.backends.formatter.TemplateFormatter._parse(source_component: Component | CBlock, result: ModelOutputThunk)
Parses the output from a model.
mellea.backends.formatter.TemplateFormatter.print_context(ctx: Context)
Renders a Context for input to a model.
mellea.backends.formatter.TemplateFormatter._stringify(c: str | Component | CBlock | Iterable | Mapping | TemplateRepresentation | None)
A recursive function that ensures an object is stringified.
For strings and CBlocks, this is just getting their value.
For Components, this means traversing the fields of their template args dictionary and stringifying each part.
Iterables and Mappings should only be encountered as parts of a component’s template args. We process each item in them while maintaining the structure.
mellea.backends.formatter.TemplateFormatter.print(c: Component | CBlock)
Uses a jinja2 template to pretty-print components.
mellea.backends.formatter.TemplateFormatter._load_template(repr: TemplateRepresentation)
This method makes an attempt at auto-loading a Template for the Component.
Iterates over template order in the template representation. If it finds
a ’*’ it will check for a template with the name of the object in ‘obj’.
Once it finds a template, it stops searching the list.
To find a template, it looks for it in the following places in order:
- the user provided template location
- the component’s package or the mellea package if none is found
mellea.backends.formatter.TemplateFormatter._get_template(root_path: str, template_name: str)
Attempts to walk the provided directory structure to find the best matching template.
Prefers the most exact match (meaning deepest directory tree) by:
- Looking for directory names matching the model name
- Looking in the
prompts/default/directory for a template
mellea.backends.formatter.TemplateFormatter._get_model_id()
Gets a string representation of the formatter’s model id.
class mellea.backends.formatter.FormatterBackend(model_id: str | ModelIdentifier, formatter: Formatter, model_options: dict | None = None)
FormatterBackends support legacy model types.
The mellea library was designed to support generative computing with spanned attention over generative programming primitives.
In the ideal world, context management is handled via span scope-relations and all generative programming primitives are baked into the model via fine-tuning.
I.e., the model’s instruction tuning is done in terms of generative programming primitives, and the model is then prompted with the same set of templates that were used for that tuning.
Today, most models do not yet support spans and even those that do are not properly tuned to leverage generative programming primitives.
The mellea library supports these legacy models primarily through prompt engineering surfaced via FormatterBackends.
A FormatterBackend is a backend that uses hand-engineered prompts for rendering generative programming primitives to a model and parsing responses from the model back into mellea.
By default, a FormatterBackend uses jinja2 templates for pretty-printing, and relies on the user’s ad-hoc logic for parsing.
Constructor
Initializes a formatter-based backend formodel_id.
Arguments
model_id:str: The model_id to use.formatter:Formatter: The formatter to use for converting components into (fragments of) prompts.model_options:Optional[dict]: The model options to use; if None, sensible defaults will be provided.