# `Nous.Plugins.Summarization`
[🔗](https://github.com/nyo16/nous/blob/v0.17.1/lib/nous/plugins/summarization.ex#L1)

Plugin for managing context window size via pruning and summarization.

When the conversation exceeds a configurable token threshold, this plugin
first prunes oversized tool results (free, no model call) and only pays for
an LLM summarization if the transcript is still over budget afterwards.

## Usage

    agent = Agent.new("openai:gpt-4",
      plugins: [Nous.Plugins.Summarization],
      deps: %{
        summarization_config: %{
          max_context_tokens: 170_000,
          keep_recent: 10,
          summary_model: "openai:gpt-4o-mini"
        }
      }
    )

## Configuration (via deps)

- `:max_context_tokens` - Token threshold to trigger compaction (default: 100_000)
- `:keep_recent` - Number of recent messages to keep intact (default: 10)
- `:max_result_chars` - Tool results longer than this are truncated before
  any model call is considered (default: 8192)
- `:summary_model` - Model string for the summarization call (default: the
  conversation's own model, which is also the only setting that preserves
  the provider's prefix cache — see below)
- `:summary_count` - Read-only counter of completed summarizations
- `:compaction_in_progress` - Read-only marker; see "Crash visibility"

## Compaction pipeline

1. Prune oversized tool results in place. This never changes the message
   count or order, so it can never break a `tool_call`/`tool_result` pair.
2. Re-estimate the transcript. If pruning alone brought it under the
   threshold, stop here — no model call, no cost.
3. Otherwise split off the recent messages and replace the older ones with
   a model-written summary.

The pruned messages are kept whichever branch runs, and whether or not the
summarization call succeeds: they are strictly cheaper than what they
replaced.

## Where the logic lives

This plugin is the live compaction entry point — it is what actually runs
during an agent run. `Nous.Transcript` is the shared library it calls for
the parts that must behave identically everywhere: pruning
(`Nous.Transcript.prune_tool_results/2`), tool-pair-safe boundaries
(`Nous.Transcript.balance_tool_call_boundary/2`) and token estimation
(`Nous.Transcript.estimate_messages_tokens/1`). There is deliberately only
one implementation of each; this module holds no boundary logic of its own.

## Prefix-cache friendliness

The summarization request replays the conversation's own system prompt,
tools and messages verbatim and appends the instruction as a final user
turn. Providers key their KV prefix cache on the literal request prefix, so
a bespoke summarizer prompt with no tools would miss the cache entirely and
re-bill every token of the conversation being summarized.

Only the response TEXT is kept: reasoning and tool calls are discarded, so
a summary can never plant an unanswered `tool_call` in the transcript.

Setting `:summary_model` to a different model gives up that cache reuse,
and — because the conversation is replayed verbatim — sends the raw
conversation, tool results included, to that model's provider.

## Telemetry

Every compaction emits, in order:

  * `[:nous, :compaction, :start]` —
    measurements `%{system_time:, monotonic_time:}`,
    metadata `%{agent_name:, messages_before:}`
  * `[:nous, :compaction, :stop]` —
    measurements `%{duration:, bytes_before:, bytes_after:}`,
    metadata `%{agent_name:, messages_before:, messages_after:, llm_called:,
    summarized:, provider:, model:, usage:}`
  * `[:nous, :compaction, :exception]` —
    measurements `%{duration:}`,
    metadata `%{agent_name:, messages_before:, messages_after:, llm_called:,
    kind:, reason:, stacktrace:}`

## Crash visibility

`:compaction_in_progress` is set in `deps[:summarization_config]` before the
`:start` event and cleared only after `:stop`. A compaction that dies in the
middle therefore leaves the marker set and an orphaned `:start` with no
matching `:stop`, instead of looking like a clean no-op.

## Safety

- Never splits tool_call/tool_result pairs
- Keeps system prompt intact
- Falls back to keeping the (pruned) messages if summarization fails

---

*Consult [api-reference.md](api-reference.md) for complete listing*