AGENTS.md

Quick-reference for AI coding agents (Claude, Cursor, Copilot, Codex, etc.) working with the Nous Elixir AI agent framework. This file is for agents that want to use the library, not for agents maintaining the library itself (see CONTRIBUTING.md and docs/ for that). Conforms to https://agents.md.

What Nous is

Multi-provider LLM framework for Elixir/OTP. Provides:

One-shot LLM calls (Nous.generate_text/2,3, Nous.stream_text/2,3)
Stateful agents with tool-calling, memory, plugins (Nous.new/2, Nous.run/2,3)
Pluggable providers — OpenAI, Anthropic, Gemini, Vertex AI, Groq, Mistral, OpenRouter, Together, Ollama, LM Studio, vLLM, SGLang, LlamaCpp, and a generic custom: adapter for any OpenAI-compatible endpoint
Tool system — file ops, bash, web fetch + search, plus easy custom tools
Pluggable HTTP backend (Req default, hackney alternative)
Streaming (Req default; opt into the hackney :async, :once pull-mode backend for strict backpressure)

Minimal API surface (start here)

# Drop-in: one-shot text generation
{:ok, text} = Nous.generate_text("openai:gpt-4o", "Explain GenServer in one sentence.")

# Streaming
{:ok, stream} = Nous.stream_text("anthropic:claude-sonnet-4-5", "Write a haiku")
Enum.each(stream, &IO.write/1)

# Stateful agent with tools
agent =
  Nous.new("openai:gpt-4o",
    tools: [Nous.Tools.FileRead, Nous.Tools.FileGrep],
    system_prompt: "You are a code reviewer."
  )

{:ok, result} = Nous.run(agent, "Find all TODOs in lib/")
# result.text, result.messages, result.usage

# Streaming agent run (text deltas only, no tool execution)
{:ok, stream} = Nous.run_stream(agent, "Summarize this repo")

# Streaming + tool execution in the same call (Nous 0.15.3+)
{:ok, result} = Nous.run(agent, "Search and summarize",
  stream: true,
  callbacks: %{
    on_llm_new_delta: fn _, t -> IO.write(t) end,
    on_llm_new_thinking_delta: fn _, t -> IO.write(["[thinking] ", t]) end
  }
)

That's 90% of what most apps need. Everything else is configuration.

Provider quick-pick (model strings)

Format is "<provider>:<model_id>". Pick one:

If you want…	Use
Best general-purpose, high quality	`openai:gpt-4o` or `anthropic:claude-sonnet-4-5-20250929`
Cheap and fast	`groq:llama-3.1-70b-versatile` or `gemini:gemini-2.0-flash`
Local / no API key	`lmstudio:<your-loaded-model>` (default port 1234)
Local high-throughput inference	`vllm:<huggingface-id>` (default port 8000)
Local with structured generation	`sglang:<model>` (default port 30000)
Anything else with an OpenAI-compatible API	`custom:<model>` + `:base_url` opt

Auth picks up the env var by convention: OPENAI_API_KEY, ANTHROPIC_API_KEY, GROQ_API_KEY, GEMINI_API_KEY, OPENROUTER_API_KEY, etc. Local providers don't need a key. Override per-call with api_key: opt.

Key opts you'll actually use

Nous.new("openai:gpt-4o",
  # LLM behavior
  system_prompt: "...",
  temperature: 0.7,
  max_tokens: 2_000,
  receive_timeout: 60_000,        # ms; 120_000 for local models

  # Tools (modules implementing Nous.Tool.Behaviour)
  tools: [Nous.Tools.Bash, MyApp.MyTool],
  parallel_tool_calls: true,      # default false; fan out multi-call turns (side effects interleave)

  # Plugins (optional, composable). LoopGuard notices the model repeating the
  # same tool call and injects escalating guidance; it never blocks.
  plugins: [Nous.Plugins.SubAgent, Nous.Plugins.HumanInTheLoop, Nous.Plugins.LoopGuard],

  # OS confinement for subprocesses (Nous.Tools.Bash). Sibling to :permissions —
  # permissions decide whether a tool runs, the sandbox confines what its
  # subprocess may touch. :read_only | :workspace_write | :danger_full_access.
  sandbox: :workspace_write,

  # Resilience
  fallback: ["anthropic:claude-sonnet-4-5", "groq:llama-3.1-70b-versatile"],

  # Vendor-specific body params (vLLM/SGLang/LM Studio/llama.cpp)
  extra_body: %{top_k: 50, repetition_penalty: 1.1}
)

Pluggable backends are not Nous.new/2 options — they are maps in deps, read by the plugin or subsystem that owns them. Passing them to Nous.new/2 silently does nothing:

Nous.run(agent, prompt,
  deps: %{
    workspace_root: "/srv/agent_workspace/#{user_id}",

    # Memory (requires Nous.Plugins.Memory in :plugins)
    memory_config: %{store: Nous.Memory.Store.ETS},

    # LLM-powered compaction (requires Nous.Plugins.Summarization in :plugins)
    summarization_config: %{max_context_tokens: 170_000, keep_recent: 10},

    # Spill oversized tool results to a store and show the model a locator
    spill_config: %{
      store: Nous.Spill.Local,
      opts: [root: "/var/lib/nous/spill"],
      max_inline_bytes: 65_536
    }
  }
)

Built-in tools

In Nous.Tools.*. The five most useful:

Nous.Tools.Bash — execute shell commands (requires approval handler in production)
Nous.Tools.FileRead / FileWrite / FileEdit — workspace-sandboxed file ops
Nous.Tools.FileGlob / FileGrep — find files / search content
Nous.Tools.WebFetch — fetch + extract text from a URL (SSRF-protected)
Nous.Tools.TavilySearch / BraveSearch — web search

File tools enforce a workspace root. Default is cwd. Override per-agent:

Nous.new("openai:gpt-4o",
  tools: [Nous.Tools.FileRead],
  deps: %{workspace_root: "/path/to/project"}
)

Building a custom tool

defmodule MyApp.WeatherTool do
  @behaviour Nous.Tool.Behaviour

  @impl Nous.Tool.Behaviour
  def metadata do
    %{
      name: "get_weather",
      description: "Get current weather for a city",
      parameters: %{
        "type" => "object",
        "properties" => %{
          "city" => %{"type" => "string", "description" => "City name"}
        },
        "required" => ["city"]
      }
    }
  end

  # Context comes FIRST, args second.
  @impl Nous.Tool.Behaviour
  def execute(_ctx, %{"city" => city}) do
    {:ok, "Weather in #{city}: 72°F, sunny"}
  end
end

Pass the module in the tools: list (tools: [MyApp.WeatherTool]) — bare behaviour modules are converted via Nous.Tool.from_module/1 automatically. The _ctx arg gives access to deps, the workspace root, and the approval handler. Use Nous.Tool.Validator for input validation — it runs automatically when validate_args: true (the default).

HTTP backend (don't change unless you need to)

Default backend is Nous.HTTP.Backend.Req — Req on top of Finch. It's faster under parallel batching than the alternative. Override only if:

You need HTTP/3 → NOUS_HTTP_BACKEND=hackney
You want one HTTP family across streaming + non-streaming → same

Pool config (hackney pool, used by the Hackney backend):

config :nous, :hackney_pool,
  max_connections: 200,
  timeout: 1_500   # idle keepalive ms (hackney 4 caps at 2_000)

Streaming defaults to Nous.HTTP.StreamBackend.Req (push-based, with an 8 MB in-flight-byte window that parks the producer — and therefore the socket — until the consumer drains). For STRICT pull-based backpressure (:async, :once — one chunk read per consumer request, no in-flight window at all), opt into the Hackney stream backend via config :nous, :http_stream_backend, Nous.HTTP.StreamBackend.Hackney, NOUS_HTTP_STREAM_BACKEND=hackney, or the per-call stream_backend: option. See docs/benchmarks/http_backend.md.

Critical rules (security & correctness)

These are project-wide and non-negotiable. If you write code that breaks these, it will be rejected.

Never String.to_atom/1 on untrusted input. Use String.to_existing_atom/1 with rescue, or pattern-match on a whitelist of literal strings. The atom table is finite and a prompt-injection input can OOM the BEAM.
Tools requiring approval are rejected without an :approval_handler. Bash, FileWrite, FileEdit need one wired in RunContext or they refuse to run. Don't disable this.
File tools enforce a workspace root. Don't bypass PathGuard. Pass paths within the workspace; the guard rejects .. traversal, absolute paths outside, and symlink escapes.
Nous.Tools.Bash is confined by Nous.Sandbox when a mode is set. confine/2 wraps argv so the OS enforces the policy; it is pure and never spawns. Fail closed: with no usable provider the tool refuses to run rather than running unconfined — don't add a passthrough. Runner failure ("bwrap could not start") is classified before denial, because "the command never ran" must not read as "confinement worked". The default is still :danger_full_access (unconfined, warns once); set config :nous, :sandbox_mode, :workspace_write or sandbox: per agent/run. FileGrep is a documented exemption — neither provider restricts reads.
HTTP from agents goes through UrlGuard. Don't make raw Req.get/1 calls from a tool to a user-controlled URL — use Nous.Tools.WebFetch or call UrlGuard.validate/2 first. Blocks RFC1918, loopback, link-local, cloud-metadata IPs.
PromptTemplate rejects <% ... %> blocks — only <%= @var %> substitution is allowed. Don't try to enable EEx evaluation on LLM-touched templates; it's an RCE vector.
Sub-agent deps don't auto-forward. If you spawn a sub-agent via Nous.Plugins.SubAgent, declare which deps it sees with :sub_agent_shared_deps, [:key1, :key2]. The default [] is correct for security.

Common workflows

Streaming to LiveView

# In your LiveView mount or handle_event:
{:ok, stream} = Nous.stream_text("anthropic:claude-sonnet-4-5", prompt)

stream
|> Stream.each(fn chunk ->
  send(self(), {:llm_chunk, chunk})
end)
|> Stream.run()

For strict backpressure under LiveView fan-out (so the stream paces itself to match diff/push throughput, one chunk at a time), opt into the Hackney stream backend (see the HTTP backend section); the default Req stream backend bounds in-flight chunks at 8 MB rather than pacing per chunk.

Tool-using agent loop

agent =
  Nous.new("openai:gpt-4o",
    tools: [Nous.Tools.FileGrep, Nous.Tools.FileRead, Nous.Tools.Bash],
    max_iterations: 10
  )

{:ok, result} = Nous.run(agent, "Find the bug in lib/foo.ex and explain it")

# result.messages contains the full transcript including tool calls
# result.usage gives token counts per provider

Provider failover

agent =
  Nous.new("openai:gpt-4o",
    fallback: [
      "anthropic:claude-sonnet-4-5-20250929",
      "groq:llama-3.1-70b-versatile"
    ]
  )

Falls through on transport errors, 5xx, and rate-limit (429) responses.

Code Mode: one program instead of a chain of tool calls

# Needs the optional :tyrex dep in your app, then:
config :nous, :code_runtime, {Nous.CodeRuntime.JS, timeout_ms: 30_000}

agent = Nous.new("openai:gpt-4o", tools: [MyApp.Search, MyApp.Fetch], code_mode: :both)

The model gains one tool, run_code, whose description carries a generated typed SDK for every tool it may call. It writes a program that loops, branches and fans out in a single round trip; only what the program logs or returns re-enters the conversation.

:both (the default) shows native tools and run_code, and behaves as :native when no runtime is configured. :code shows only run_code. It is not an unconditional token saving — the SDK is a prompt prefix that can rival the native schemas it replaces, so it wins on multi-step and fan-out work and loses on a single bash. Measure your own workload; see docs/guides/code_mode.md.

Approval is per sub-call: approving a run_code call approves running that program, not whatever it then decides to call. Each requires_approval: true sub-call consults your handler with the real tool name and arguments, and is refused if there is no handler.

Local dev with LM Studio

# 1. Start LM Studio, load a model, start the server (default port 1234).
# 2. In Elixir:
{:ok, text} = Nous.generate_text("lmstudio:<exact-model-name-shown-in-lmstudio>",
                                  "Hello!")

# Or override the URL:
agent = Nous.new("lmstudio:my-model", base_url: "http://gpu-host:1234/v1")

Testing your code that uses Nous

# Use the test helpers in Nous.Tool.Testing for tool unit tests.
# For end-to-end agent tests, the recommended pattern is to use Bypass to
# stub the LLM HTTP endpoint:

setup do
  bypass = Bypass.open()
  base = "http://localhost:#{bypass.port}/v1"
  {:ok, bypass: bypass, base: base}
end

test "agent calls the model", %{bypass: bypass, base: base} do
  Bypass.expect_once(bypass, "POST", "/v1/chat/completions", fn conn ->
    conn
    |> Plug.Conn.put_resp_header("content-type", "application/json")
    |> Plug.Conn.resp(200, ~s({"choices":[{"message":{"content":"hi!"}}]}))
  end)

  agent = Nous.new("custom:test-model", base_url: base, api_key: "test")
  assert {:ok, %{text: "hi!"}} = Nous.run(agent, "hello")
end

Don't mock Req/hackney directly — Bypass is the supported test seam.

What NOT to use

The public API is Nous.* and Nous.Tools.*. The rule is mechanical: if a module carries a @moduledoc and appears in https://hexdocs.pm/nous, it is public and covered by semver; if it is @moduledoc false, it is not. Nothing here is "internal by convention" — the code is the contract.

Currently hidden, do not call:

Nous.Application, Nous.Persistence.ETS.TableOwner, Nous.Workflow.Checkpoint.ETS.TableOwner — internal supervision tree and ETS table owners
every Nous.AgentRunner.* submodule (prompt assembly, request dispatch, the iteration loop, streaming, tool execution) — internal to the runner; the entry point is Nous.AgentRunner itself
Nous.OutputSchema.UseMacro — implementation of use Nous.OutputSchema
Nous.Workflow.Engine.Executor, Nous.Workflow.Engine.ParallelExecutor, Nous.Workflow.Engine.StateMerger — internal node dispatch; use Nous.Workflow to build and Nous.Workflow.Engine.execute/1,2 to run
Nous.JSON — internal pretty-printing wrapper over the standard JSON module
Nous.Util — small shared helpers (atom coercion, option splitting) used across internals
Nous.Memory.Embedding.Bumblebee.ServingSupervisor and Nous.Memory.Embedding.Bumblebee.ServingHolder — process plumbing behind the public Nous.Memory.Embedding.Bumblebee provider, and only compiled when Bumblebee is available

Up to 0.17.0 this section also claimed Nous.AgentRunner, Nous.AgentServer, Nous.Providers.HTTP, Nous.HTTP.Backend.*, Nous.HTTP.StreamBackend.* and Nous.Workflow.Engine were private. They never were: all six are documented extension points that this file, docs/guides/http_backends.md, the LiveView guide and examples/ already tell you to call. The list was wrong, not the code — they stay public.

Stick to the documented modules and your code will survive minor version bumps.

Where to look for more

Hex docs: https://hexdocs.pm/nous
Getting started: docs/getting-started.md
Production guides: docs/guides/ (skills, hooks, LiveView integration, best practices, tool development, troubleshooting, evaluation, structured output, workflows, memory, context, knowledge base)
Examples: examples/
CHANGELOG: behavioral changes per release; read the "Behavioral / breaking changes" sections before upgrading.

← Previous Page Changelog

Next Page → Contributing