# `Nous.Eval.Evaluator`
[🔗](https://github.com/nyo16/nous/blob/v0.17.1/lib/nous/eval/evaluator.ex#L1)

Behaviour for evaluating agent outputs against expected results.

Evaluators determine whether an agent's output matches expectations and
provide a score indicating the quality of the match.

## Built-in Evaluators

- `Nous.Eval.Evaluators.ExactMatch` - Exact string match
- `Nous.Eval.Evaluators.FuzzyMatch` - Similarity-based match
- `Nous.Eval.Evaluators.Contains` - Check for substrings
- `Nous.Eval.Evaluators.ToolUsage` - Verify tool calls
- `Nous.Eval.Evaluators.Schema` - Validate structured output
- `Nous.Eval.Evaluators.LLMJudge` - LLM-based evaluation

## Custom Evaluators

    defmodule MyEvaluator do
      @behaviour Nous.Eval.Evaluator

      @impl true
      def evaluate(actual, expected, config) do
        # Your evaluation logic
        if my_check(actual, expected) do
          %{score: 1.0, passed: true, reason: nil, details: %{}}
        else
          %{score: 0.0, passed: false, reason: "Did not match", details: %{}}
        end
      end
    end

## Result Format

Evaluators must return a map with:

  * `:score` - Float from 0.0 to 1.0
  * `:passed` - Boolean indicating pass/fail
  * `:reason` - String explaining failure (or nil)
  * `:details` - Map with additional details

# `result`

```elixir
@type result() :: %{
  score: score(),
  passed: boolean(),
  reason: String.t() | nil,
  details: map()
}
```

# `score`

```elixir
@type score() :: float()
```

# `evaluate`

```elixir
@callback evaluate(actual :: term(), expected :: term(), config :: map()) :: result()
```

Evaluate an actual output against expected output.

## Parameters

  * `actual` - The actual output from the agent
  * `expected` - The expected output
  * `config` - Configuration map for the evaluator

## Returns

A result map with score, passed status, and details.

# `name`
*optional* 

```elixir
@callback name() :: String.t()
```

Optional: Name of the evaluator for display purposes.

# `fail`

```elixir
@spec fail(String.t(), map()) :: result()
```

Create a failing result helper.

# `get_evaluator`

```elixir
@spec get_evaluator(atom()) :: module() | nil
```

Get the evaluator module for an eval_type.

# `partial`

```elixir
@spec partial(float(), String.t() | nil, map()) :: result()
```

Create a partial match result helper.

# `pass`

```elixir
@spec pass(map()) :: result()
```

Create a passing result helper.

# `run`

```elixir
@spec run(atom(), term(), term(), map()) :: result()
```

Run evaluation using the appropriate evaluator.

---

*Consult [api-reference.md](api-reference.md) for complete listing*