# `Nous.Providers.OpenAICompatible`
[🔗](https://github.com/nyo16/nous/blob/v0.17.1/lib/nous/providers/openai_compatible.ex#L1)

Generic OpenAI-compatible provider implementation.

> **Note**: This module implements the underlying HTTP API for OpenAI-compatible
> endpoints. For normal usage, prefer the `custom:` model prefix which routes
> through `Nous.Providers.Custom` with better configuration support.

Works with any server that implements the OpenAI Chat Completions API:
- Groq (api.groq.com)
- Together (api.together.xyz)
- OpenRouter (openrouter.ai)
- LM Studio (localhost)
- Ollama (localhost)
- vLLM (localhost)
- SGLang (localhost)
- Mistral (api.mistral.ai)
- Any other OpenAI-compatible endpoint

For OpenAI-specific features (Responses API, Assistants, etc.), use `Nous.Providers.OpenAI`.

## Configuration

Configuration is looked up in the following precedence (highest to lowest):

1. **Direct options** passed to functions or `Nous.new/2`:
   ```elixir
   Nous.new("custom:my-model",
     base_url: "https://api.example.com/v1",
     api_key: "sk-..."
   )
   ```

2. **Environment variables**:
   ```bash
   export CUSTOM_BASE_URL="https://api.example.com/v1"
   export CUSTOM_API_KEY="sk-..."
   ```

3. **Application config** (in `config/config.exs`):
   ```elixir
   config :nous, :custom,
     base_url: "https://api.example.com/v1",
     api_key: "sk-..."
   ```

4. **Defaults** (used by this module directly):
   - `base_url`: `"https://api.openai.com/v1"`
   - `api_key`: From `OPENAI_API_KEY` env var

## Options

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `base_url` | `String.t()` | `"https://api.openai.com/v1"` | API endpoint URL |
| `api_key` | `String.t()` | `nil` | Authentication token |
| `organization` | `String.t()` | `nil` | OpenAI organization ID |
| `timeout` | `non_neg_integer()` | `60000` | Request timeout (ms) |

## Usage

### With the `custom:` prefix (Recommended)

Use this for any OpenAI-compatible endpoint:

    # Groq
    agent = Nous.new("custom:llama-3.1-70b",
      base_url: "https://api.groq.com/openai/v1",
      api_key: System.get_env("GROQ_API_KEY")
    )

    # Together AI
    agent = Nous.new("custom:meta-llama/Llama-3-70b",
      base_url: "https://api.together.xyz/v1",
      api_key: System.get_env("TOGETHER_API_KEY")
    )

    # OpenRouter
    agent = Nous.new("custom:anthropic/claude-3.5-sonnet",
      base_url: "https://openrouter.ai/api/v1",
      api_key: System.get_env("OPENROUTER_API_KEY")
    )

    # LM Studio (local)
    agent = Nous.new("custom:qwen3",
      base_url: "http://localhost:1234/v1"
    )

    # Using environment variables
    # export CUSTOM_BASE_URL="http://localhost:1234/v1"
    # export CUSTOM_API_KEY="not-needed"
    agent = Nous.new("custom:my-model")

### Direct Provider Usage (Low-level)

For direct API access without the agent:

    # Using with Groq
    {:ok, response} = Nous.Providers.OpenAICompatible.chat(
      %{model: "llama-3.1-70b", messages: messages},
      base_url: "https://api.groq.com/openai/v1",
      api_key: System.get_env("GROQ_API_KEY")
    )

    # Using with local LM Studio
    {:ok, response} = Nous.Providers.OpenAICompatible.chat(
      %{model: "qwen2", messages: messages},
      base_url: "http://localhost:1234/v1"
    )

    # Streaming
    {:ok, stream} = Nous.Providers.OpenAICompatible.chat_stream(params, opts)
    Enum.each(stream, fn event -> IO.inspect(event) end)

## Backward Compatibility

The legacy `openai_compatible:` prefix still works and is equivalent to `custom:`:

    # Legacy (still works)
    agent = Nous.new("openai_compatible:my-model", base_url: "...")

    # Recommended
    agent = Nous.new("custom:my-model", base_url: "...")

See `Nous.Providers.Custom` for the dedicated custom provider implementation.

# `api_key`

```elixir
@spec api_key(keyword()) :: String.t() | nil
```

Get the API key from options, environment, or application config.

Lookup order:
1. `:api_key` option passed directly
2. Environment variable (OPENAI_API_KEY)
3. Application config: `config :nous, openai_compatible, api_key: "..."`

# `base_url`

```elixir
@spec base_url(keyword()) :: String.t()
```

Get the base URL from options, application config, or default.

Lookup order:
1. `:base_url` option passed directly
2. Application config: `config :nous, openai_compatible, base_url: "..."`
3. Default: https://api.openai.com/v1

# `count_tokens`

```elixir
@spec count_tokens(list()) :: integer()
```

Count tokens in messages (rough estimate).

Override this in your provider for more accurate counting.

# `request`

High-level request with message conversion, telemetry, and error wrapping.

Default implementation that:
1. Converts messages to provider format
2. Builds request params
3. Calls chat/2
4. Parses response
5. Emits telemetry events
6. Wraps errors

# `request_stream`

High-level streaming request with message conversion and telemetry.

---

*Consult [api-reference.md](api-reference.md) for complete listing*