# `Nous.Providers.LlamaCpp`
[🔗](https://github.com/nyo16/nous/blob/v0.17.1/lib/nous/providers/llamacpp.ex#L2)

LlamaCpp NIF-based provider for local LLM inference.

Runs GGUF models directly in-process via `llama_cpp_ex` NIF bindings.
No HTTP server needed.

Requires optional dep: `{:llama_cpp_ex, "~> 0.6.5"}`

## Usage

    # Load model once at app start
    :ok = LlamaCppEx.init()
    {:ok, llm} = LlamaCppEx.load_model("model.gguf", n_gpu_layers: -1)

    # Use with Nous
    agent = Nous.new("llamacpp:local",
      llamacpp_model: llm,
      instructions: "You are helpful."
    )

    {:ok, result} = Nous.run(agent, "What is Elixir?")

## Configuration

The `llamacpp_model` (the loaded model reference) must be passed via options
when creating the model or agent. It is stored in `default_settings`.

No API key or base URL is needed since inference runs locally via NIFs.

## Settings Mapping

Nous settings are mapped to LlamaCppEx options:

| Nous Setting | LlamaCppEx Option | Description |
|---|---|---|
| `:temperature` | `:temp` | Sampling temperature |
| `:max_tokens` | `:max_tokens` | Maximum tokens to generate |
| `:top_p` | `:top_p` | Nucleus sampling |
| `:json_schema` | `:json_schema` | Constrained JSON output |
| `:enable_thinking` | `:enable_thinking` | Enable/disable thinking tokens |

## Thinking Models

Models like Qwen3 emit `<think>...</think>` tags by default. To disable:

    agent = Nous.new("llamacpp:local",
      llamacpp_model: llm,
      model_settings: %{enable_thinking: false}
    )

Or via `generate_text`:

    {:ok, text} = Nous.generate_text("llamacpp:local", "Hello",
      llamacpp_model: llm,
      enable_thinking: false
    )

# `api_key`

```elixir
@spec api_key(keyword()) :: String.t() | nil
```

Get the API key from options, environment, or application config.

Lookup order:
1. `:api_key` option passed directly
2. Environment variable (LLAMACPP_MODEL_PATH)
3. Application config: `config :nous, llamacpp, api_key: "..."`

# `base_url`

```elixir
@spec base_url(keyword()) :: String.t()
```

Get the base URL from options, application config, or default.

Lookup order:
1. `:base_url` option passed directly
2. Application config: `config :nous, llamacpp, base_url: "..."`
3. Default: local

# `count_tokens`

```elixir
@spec count_tokens(list()) :: integer()
```

Count tokens in messages (rough estimate).

Override this in your provider for more accurate counting.

# `request`

High-level request with message conversion, telemetry, and error wrapping.

Default implementation that:
1. Converts messages to provider format
2. Builds request params
3. Calls chat/2
4. Parses response
5. Emits telemetry events
6. Wraps errors

# `request_stream`

High-level streaming request with message conversion and telemetry.

---

*Consult [api-reference.md](api-reference.md) for complete listing*