# `Nous.ModelDispatcher`
[🔗](https://github.com/nyo16/nous/blob/v0.17.1/lib/nous/model_dispatcher.ex#L1)

Dispatches model requests to the appropriate provider implementation.

Routes requests to providers based on the model's provider field:
- `:anthropic` → `Nous.Providers.Anthropic`
- `:gemini` → `Nous.Providers.Gemini`
- `:vertex_ai` → `Nous.Providers.VertexAI`
- `:mistral` → `Nous.Providers.Mistral`
- `:lmstudio` → `Nous.Providers.LMStudio`
- `:llamacpp` → `Nous.Providers.LlamaCpp`
- `:vllm` → `Nous.Providers.VLLM`
- `:sglang` → `Nous.Providers.SGLang`
- `:openai` → `Nous.Providers.OpenAI`
- `:custom` → `Nous.Providers.Custom`
- Others → `Nous.Providers.OpenAICompatible`

## Swapping the dispatcher

Every model request in Nous goes through a *resolved* dispatcher module
rather than through this module directly. See `resolve/1` for the precedence
rules, `config :nous, :model_dispatcher, MyDispatcher` for the production
injection point, and `put_dispatcher/1` for the process-scoped testing seam.

# `count_tokens`

```elixir
@spec count_tokens(Nous.Model.t(), list()) :: integer()
```

Count tokens (uses appropriate provider implementation).

# `provider_module`

```elixir
@spec provider_module(atom()) :: module()
```

Resolve the provider module for a provider atom.

Unknown providers fall back to `Nous.Providers.OpenAICompatible`.

# `put_dispatcher`

```elixir
@spec put_dispatcher(module() | nil) :: :ok
```

Install a process-scoped dispatcher override. **Testing seam only.**

Unlike `config :nous, :model_dispatcher, MyDispatcher` — which remains the
production mechanism — this writes no global state. The module is stored in
the calling process's dictionary and is visible to that process and to any
process that lists it in `$callers`; `Task.async/1`, `Task.async_stream/3`
and `Task.Supervisor.async/2` all propagate `$callers`, so work the runner
fans out still sees the override. Concurrent (`async: true`) test modules can
therefore each install their own stub without contending for the application
environment.

The override lives and dies with the calling process. ExUnit gives every test
a fresh one, so there is nothing to tear down; pass `nil` to clear it early.

`$callers` is *not* propagated across `GenServer.start_link/3`, so a run
driven through `Nous.AgentServer` executes in the server process and will not
see the override. Those tests still need the application environment, and
must stay `async: false` because of it.

    test "retries on a provider error" do
      Nous.ModelDispatcher.put_dispatcher(FlakyDispatcher)
      assert {:ok, _} = Nous.generate_text("openai:gpt-4", "hi")
    end

# `request`

```elixir
@spec request(Nous.Model.t(), list(), map()) :: {:ok, map()} | {:error, term()}
```

Dispatch request to the appropriate provider implementation.

# `request_stream`

```elixir
@spec request_stream(Nous.Model.t(), list(), map()) ::
  {:ok, Enumerable.t()} | {:error, term()}
```

Dispatch streaming request to the appropriate provider implementation.

# `resolve`

```elixir
@spec resolve(module() | nil) :: module()
```

Resolve the module that services model requests, highest precedence first:

  1. `override` — an explicit per-call module, such as `Nous.LLM`'s
    `:model_dispatcher` option. `nil` means "not specified".
  2. A process-scoped override from `put_dispatcher/1`, searched in the
    calling process and then along `$callers`.
  3. `config :nous, :model_dispatcher, MyDispatcher`.
  4. `Nous.ModelDispatcher`.

---

*Consult [api-reference.md](api-reference.md) for complete listing*