​
Module: mellea.backends.ollama

A model backend wrapping the Ollama Python SDK.

​
Classes

​
class mellea.backends.ollama.OllamaModelBackend(model_id: str | ModelIdentifier = model_ids.IBM_GRANITE_3_3_8B, formatter: Formatter | None = None, base_url: str | None = None, model_options: dict | None = None)

A model that uses the Ollama Python SDK for local inference.

​
Constructor

Initializes an ollama backend for local models. WARNING: may use up a lot of your machine’s memory.

​
Arguments

  • model_id: str | ModelIdentifier: Ollama model ID. If ModelIdentifier, then an ollama_name must be provided by that ModelIdentifier.
  • base_url: str: Endpoint that is serving the model API; defaults to env(OLLAMA_HOST) or http://localhost:11434
  • model_options: dict: Ollama model options
  • formatter: Formatter: formatter for creating input

​
Methods

mellea.backends.ollama.OllamaModelBackend._get_ollama_model_id()
Gets the ollama model id from the model_id that was provided in the constructor. Raises AssertionError is the ModelIdentifier does not provide an ollama_name.
mellea.backends.ollama.OllamaModelBackend._check_ollama_server()
Requests generic info about the Ollama server to ensure it’s running.
mellea.backends.ollama.OllamaModelBackend.is_model_available(model_name)
Checks if a specific Ollama model is available locally.

​
Arguments

  • model_name: The name of the model to check for (e.g., “llama2”).
True if the model is available, False otherwise.
mellea.backends.ollama.OllamaModelBackend._pull_ollama_model()
Either gets the cached ollama model or else attempts to pull the provided model from Ollama. Raises an exception of the model cannot be pulled. This code was generated by ChatGPT.
mellea.backends.ollama.OllamaModelBackend._simplify_and_merge(model_options: dict[str, Any] | None)
Simplifies model_options to use the Mellea specific ModelOption.Option and merges the backend’s model_options with those passed into this call. Rules:
  • Within a model_options dict, existing keys take precedence. This means remapping to mellea specific keys will maintain the value of the mellea specific key if one already exists.
  • When merging, the keys/values from the dictionary passed into this function take precedence.
Because this function simplifies and then merges, non-Mellea keys from the passed in model_options will replace Mellea specific keys from the backend’s model_options.

​
Arguments

  • model_options: the model_options for this call
a new dict
mellea.backends.ollama.OllamaModelBackend._make_backend_specific_and_remove(model_options: dict[str, Any])
Maps specified Mellea specific keys to their backend specific version and removes any remaining Mellea keys.

​
Arguments

  • model_options: the model_options for this call
a new dict
mellea.backends.ollama.OllamaModelBackend.generate_from_context(action: Component | CBlock, ctx: Context, format: type[BaseModelSubclass] | None = None, model_options: dict | None = None, generate_logs: list[GenerateLog] | None = None, tool_calls: bool = False)
See generate_from_chat_context.
mellea.backends.ollama.OllamaModelBackend.generate_from_chat_context(action: Component | CBlock, ctx: Context, format: type[BaseModelSubclass] | None = None, model_options: dict | None = None, generate_logs: list[GenerateLog] | None = None, tool_calls: bool = False)
Generates a new completion from the provided Context using this backend’s Formatter. This implementation treats the Context as a chat history, and uses the ollama.Client.chat() interface to generate a completion. This will not always work, because sometimes we want to use non-chat models.
mellea.backends.ollama.OllamaModelBackend._generate_from_raw(actions: list[Component | CBlock], format: type[BaseModelSubclass] | None = None, model_options: dict | None = None, generate_logs: list[GenerateLog] | None = None)
Generate using the generate api. Gives the input provided to the model without templating.
mellea.backends.ollama.OllamaModelBackend._extract_model_tool_requests(tools: dict[str, Callable], chat_response: ollama.ChatResponse)