Nous.Model (nous v0.17.3)

Model configuration for LLM providers.

This module defines the model configuration structure used by all model adapters to connect to various LLM providers.

Example

model = Model.new(:openai, "gpt-4",
  api_key: "sk-...",
  default_settings: %{temperature: 0.7}
)

Summary

Types

provider()

t()

Functions

new(provider, model, opts \\ [])

Create a new model configuration.

parse(model_string, opts \\ [])

Parse a model string into a Model struct.

Types

provider()

@type provider() ::
  :openai
  | :anthropic
  | :gemini
  | :vertex_ai
  | :groq
  | :ollama
  | :lmstudio
  | :llamacpp
  | :openrouter
  | :together
  | :vllm
  | :sglang
  | :mistral
  | :custom

t()

@type t() :: %Nous.Model{
  api_key: String.t() | nil,
  base_url: String.t() | nil,
  beta: String.t() | [String.t()] | nil,
  default_settings: map(),
  enable_long_context: boolean(),
  headers: [{String.t(), String.t()}],
  model: String.t(),
  organization: String.t() | nil,
  project: String.t() | nil,
  provider: provider(),
  receive_timeout: non_neg_integer(),
  stream_normalizer: module() | nil
}

Functions

new(provider, model, opts \\ [])

@spec new(provider(), String.t(), keyword()) :: t()

Create a new model configuration.

Parameters

provider - Provider atom (:openai, :groq, :ollama, etc.)
model - Model name string
opts - Optional configuration

Options

:base_url - Custom API base URL
:api_key - API key (defaults to environment config)
:organization - Organization ID, sent as openai-organization by :openai and the OpenAI-compatible providers
:project - Project ID for project-scoped OpenAI keys, sent as openai-project (:openai only)
:enable_long_context - Opt in to Anthropic's 1M-token context window beta (anthropic-beta: context-1m-2025-08-07; :anthropic only)
:beta - Further Anthropic beta features, a string or a list of strings, sent as anthropic-beta (:anthropic only)
:receive_timeout - HTTP receive timeout in milliseconds (default: 180000). Increase this for local models that may take longer to respond.
:default_settings - Default model settings (temperature, max_tokens, etc.)
:stream_normalizer - Custom stream normalizer module implementing Nous.StreamNormalizer behaviour
:headers - Extra HTTP headers sent with every request to the provider, as a list of {name, value} string tuples or a map. Use it for whatever an intermediary in front of the provider needs: an AI gateway's routing and auth headers (cf-aig-gateway-id, cf-aig-authorization), request metadata, a tenant id. They are applied last, so a header with the same name as one Nous builds (compared case-insensitively) replaces it. Names must be HTTP tokens and values may not contain CR, LF or NUL; anything else raises ArgumentError here rather than failing inside the HTTP client on every request. Excluded from inspect/1. Not used by :llamacpp, which makes no HTTP request.

Example

model = Model.new(:openai, "gpt-4",
  api_key: "sk-...",
  default_settings: %{temperature: 0.7, max_tokens: 1000}
)

# For slow local models, increase the timeout
model = Model.new(:lmstudio, "qwen/qwen3-4b",
  receive_timeout: 120_000  # 2 minutes
)

# Through an AI gateway that routes and tags by header
model = Model.new(:custom, "openai/gpt-4.1",
  base_url: "https://api.cloudflare.com/client/v4/accounts/ACCOUNT/ai/v1",
  headers: [{"cf-aig-gateway-id", "my-gateway"}]
)

parse(model_string, opts \\ [])

@spec parse(
  String.t(),
  keyword()
) :: t()

Parse a model string into a Model struct.

Supports the format "provider:model-name" for convenient model specification.

Supported Formats

"openai:gpt-4" - OpenAI models
"anthropic:claude-3-5-sonnet-20241022" - Anthropic Claude
"gemini:gemini-1.5-pro" - Google Gemini
"vertex_ai:gemini-2.0-flash" - Google Vertex AI
"groq:llama-3.1-70b-versatile" - Groq models
"mistral:mistral-large-latest" - Mistral models
"ollama:llama2" - Local Ollama
"lmstudio:qwen3-vl-4b-thinking-mlx" - Local LM Studio
"llamacpp:local" - Local LlamaCpp NIF (requires :llamacpp_model option)
"vllm:qwen3-vl-4b-thinking-mlx" - vLLM server
"sglang:meta-llama/Llama-3-8B" - SGLang server
"openrouter:anthropic/claude-3.5-sonnet" - OpenRouter
"together:meta-llama/Llama-3-70b-chat-hf" - Together AI
"custom:my-model" - Custom OpenAI-compatible endpoint (requires :base_url option)

Note: The "custom:" prefix is the recommended approach for any OpenAI-compatible endpoint. The legacy "openai_compatible:" prefix still works for backward compatibility.

Examples

iex> %Model{provider: provider, model: model} = Model.parse("openai:gpt-4")
iex> {provider, model}
{:openai, "gpt-4"}

iex> %Model{provider: provider, model: model} = Model.parse("ollama:llama2")
iex> {provider, model}
{:ollama, "llama2"}

iex> model = Model.parse("custom:my-model", base_url: "http://localhost:8080/v1")
iex> {model.provider, model.model, model.base_url}
{:custom, "my-model", "http://localhost:8080/v1"}

Custom Providers with base_url

The custom: prefix works with any OpenAI-compatible endpoint:

# Groq
Model.parse("custom:llama-3.1-70b",
  base_url: "https://api.groq.com/openai/v1",
  api_key: System.get_env("GROQ_API_KEY")
)

# Together AI
Model.parse("custom:meta-llama/Llama-3-70b",
  base_url: "https://api.together.xyz/v1",
  api_key: System.get_env("TOGETHER_API_KEY")
)

# Local server (LM Studio, Ollama, etc.)
Model.parse("custom:qwen3", base_url: "http://localhost:1234/v1")

Also supports CUSTOM_API_KEY and CUSTOM_BASE_URL environment variables as defaults (can be overridden per-call).