Nous.CodeMode.Scheduler (nous v0.17.1)

Copy Markdown View Source

Orders the tool sub-calls a Code Mode program makes.

A program is a batch of tool calls the model wrote as code — loops, fan-out, branches — and nothing in the program says how those calls may interleave. The scheduler decides, and it decides conservatively.

One driver lane

Every ordering decision (classify, start, commit) happens inside this GenServer's own process, so the lane is the serialized mailbox: there is no bespoke lock, no ETS counter, nothing to get wrong under contention. State is a submission-ordered pending queue, an inflight set, and a head-of-line cursor that commits settled sub-calls strictly in submission order — a call that finishes early waits for its predecessors, so the program observes one deterministic order regardless of how the work actually raced.

Execution itself fans out through Task.Supervisor.async_nolink/2 on Nous.TaskSupervisor — the same supervisor and the same a-crash-is-a-result policy Nous.AgentRunner.ToolExecution uses for parallel native tool calls. That module uses Task.Supervisor.async_stream_nolink/4 because it has the whole batch up front; here sub-calls arrive one at a time from a running program, so a stream over a finished list is the wrong shape. $callers is propagated the same way regardless (see "Caller chain" below).

Exclusive by default

A tool is treated as EXCLUSIVE unless its module exports concurrency_safe?/1 and that function returns exactly true for the call's arguments. Absent attribute, unloadable module, wrong return, raise, throw or exit — all mean exclusive. This is fail-safe, not fail-open: a tool that does not say it is safe to run concurrently is assumed unsafe, because the failure mode of guessing wrong is interleaved side effects on someone else's data.

An exclusive call drains the pool, runs alone, and holds the barrier through its commit — not merely through execution, which would let a concurrent call commit inside its window. See mode/0 and the drive/1 comments.

What the program can see

A failed sub-call arrives inside the program as {:error, %{"tool" => name, "message" => message}} — a plain map of two strings. Never a struct, never a stacktrace, never a %Nous.Errors.ToolError{} with its :original_error still attached. Model-authored code is one prompt-injection hop from the transcript, so the operator's Logger gets the detail and the program gets two strings.

Arguments are snapshotted twice

At submission the scheduler materializes two independent JSON-shaped snapshots of the arguments: one dispatched to the tool, one written to the session log. Neither path can influence the other, so a tool cannot desync the audit record from what actually ran. Normalizing both is also what makes the audit record true: what the tool received and what the log says are the same shape, and a program cannot smuggle a live BEAM handle (pid, ref, port, closure) into a tool through the transport.

Session events, outside model history

Each sub-dispatch appends exactly one BOOKKEEPING :tool_call event to the Nous.Agent.Context the scheduler was handed, at the moment it starts, in submission order. Bookkeeping events project to no message (Nous.Session.Log.derive_messages/1), so ctx.messages is unchanged by any number of sub-dispatches: only what the program logs or returns re-enters the model's context.

There are two ways to read that trail back. context/1 is for a caller that handed in a real Nous.Agent.Context. logged_events/1 is for one that owns none: Nous.Tools.RunCode is handed a %Nous.RunContext{}, which has no session log at all, so it forwards the pairs as Nous.Tool.ContextUpdate.log_event/3 operations and the runner appends them to the context that does exist.

Caller chain

$callers is not propagated across GenServer.start_link/3, so the scheduler captures the starting process's chain and reinstalls it in its own process dictionary. Sub-call tasks therefore see [scheduler, starter | ...] and process-scoped overrides — Nous.ModelDispatcher.put_dispatcher/1, Mox allowances, Ecto sandbox ownership — keep working inside a sub-call.

Shape

{:ok, sched} = Scheduler.start_link(dispatch: dispatch, context: ctx)

# Phase D hands this straight to Nous.CodeMode.bindings/4 as `:dispatch`,
# and every granted-tool closure routes through the lane.
Scheduler.dispatch_fun(sched)

# Or drive it directly: submit many, await in any order.
{:ok, ticket} = Scheduler.submit(sched, tool, %{"q" => "elixir"}, run_ctx)
Scheduler.await(sched, ticket)

The scheduler is one plain module plus one GenServer. It is deliberately not supervised: it lives and dies with a single run_code call, and stop/2 releases every outstanding caller with an error rather than leaving a program blocked on a lane that no longer exists.

Summary

Types

How a sub-call actually reaches the tool pipeline.

What a failed sub-call looks like inside the program: two strings and nothing else, under the keys "tool" and "message".

:parallel may share the pool with up to max_parallel - 1 others. :exclusive requires an empty pool and holds the barrier through its commit.

The result of one sub-call, as the program sees it.

A scheduler process.

Handle for one submitted sub-call. Carries the tool name so await/3 can name the tool even when the scheduler dies before answering.

Functions

Block until a submitted sub-call commits.

Submit a sub-call and block until it commits.

Returns a specification to start this module under a supervisor.

Whether tool may run alongside other sub-calls, given args.

The context, carrying one bookkeeping :tool_call event per started sub-dispatch.

The scheduler as a dispatch/0 function.

The bookkeeping events this scheduler appended, as {type, data} pairs in submission order.

One independently materialized, JSON-shaped snapshot of args.

Start a scheduler for one Code Mode run.

Tear the lane down, releasing every outstanding caller with an error.

Queue a sub-call and return immediately.

Types

dispatch()

@type dispatch() :: (Nous.Tool.t(), map(), term() -> {:ok, term()} | {:error, term()})

How a sub-call actually reaches the tool pipeline.

Supplied by the caller — the scheduler never calls Nous.ToolExecutor itself, so it can be driven by a stub with no runtime, no registry and no agent. {:ok, value} | {:error, reason} only: any other return is a contract breach by Nous, logged and surfaced to the program as an opaque failure rather than passed through unvetted.

error()

@type error() :: %{required(String.t()) => String.t()}

What a failed sub-call looks like inside the program: two strings and nothing else, under the keys "tool" and "message".

mode()

@type mode() :: :parallel | :exclusive

:parallel may share the pool with up to max_parallel - 1 others. :exclusive requires an empty pool and holds the barrier through its commit.

outcome()

@type outcome() :: {:ok, term()} | {:error, error()}

The result of one sub-call, as the program sees it.

server()

@type server() :: GenServer.server()

A scheduler process.

ticket()

@type ticket() :: {reference(), String.t()}

Handle for one submitted sub-call. Carries the tool name so await/3 can name the tool even when the scheduler dies before answering.

Functions

await(scheduler, ticket, timeout \\ :infinity)

@spec await(server(), ticket(), timeout()) :: outcome()

Block until a submitted sub-call commits.

Monitors the scheduler, so a lane that dies mid-call answers with an error instead of blocking the program forever. timeout defaults to :infinity: bounding a run is the runtime provider's job — its budgets are the deadline that a program cannot ask to extend — and the scheduler must not invent a second, shorter one.

call(scheduler, tool, args, run_ctx)

@spec call(server(), Nous.Tool.t(), map(), term()) :: outcome()

Submit a sub-call and block until it commits.

This is what a binding closure calls. Failure is always data: the caller gets {:error, %{"tool" => _, "message" => _}}, never an exit and never a raise.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

concurrency_mode(tool, args)

@spec concurrency_mode(Nous.Tool.t(), map()) :: mode()

Whether tool may run alongside other sub-calls, given args.

Exclusive unless tool.module exports concurrency_safe?/1 and it returns exactly true. A missing module, a missing export, a non-true return, or a raise / throw / exit inside the predicate all mean :exclusive.

context(scheduler)

@spec context(server()) :: Nous.Agent.Context.t()

The context, carrying one bookkeeping :tool_call event per started sub-dispatch.

messages is untouched — that is the whole point of logging sub-dispatches as bookkeeping events.

dispatch_fun(scheduler)

@spec dispatch_fun(server()) :: dispatch()

The scheduler as a dispatch/0 function.

This is the Phase D seam: hand it to Nous.CodeMode.bindings/4 as :dispatch and every granted-tool closure routes through the lane, with zero changes on either side.

logged_events(scheduler)

@spec logged_events(server()) :: [{Nous.Session.Event.type(), map()}]

The bookkeeping events this scheduler appended, as {type, data} pairs in submission order.

This is the read for a transport that owns no Nous.Agent.Context, and so has nothing useful to do with context/1: Nous.Tools.RunCode forwards these as Nous.Tool.ContextUpdate.log_event/3 operations, which is how a sub-dispatch reaches the real session log.

A scheduler that is already gone answers [] and says so at warning level. Silently losing an audit trail is the one outcome this path exists to prevent; it is unreachable while the caller still holds the link start_link/1 made, and loud if a future caller drops it.

snapshot(args)

@spec snapshot(term()) :: term()

One independently materialized, JSON-shaped snapshot of args.

Scalars pass through, maps and lists are rebuilt with string keys, and anything that is not JSON — a struct, pid, reference, port or closure — is rendered with inspect/1. The escape hatch costs an exotic value its type; it never costs the snapshot its serializability, which is what keeps a session event appendable and keeps a live BEAM handle out of both the audit log and the tool.

Called twice per sub-call, on the raw arguments, so the dispatched and logged snapshots are separate terms that cannot influence each other.

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

Start a scheduler for one Code Mode run.

Options

  • :dispatch (required) — dispatch/0, how a sub-call reaches the tool pipeline.
  • :context — the Nous.Agent.Context sub-dispatch events are appended to. Defaults to a fresh Nous.Agent.Context.new/0. Read it back with context/1.
  • :max_parallel — positive integer, default 10. The ceiling on concurrently executing sub-calls; exclusive calls ignore it downwards by requiring a ceiling of one.
  • :call_id — the outer run_code call id, used to build the "<call_id>:code:<n>" correlation id on each session event.
  • :name — optional GenServer name.

Returns {:error, {:contract, message}} for misconfiguration, matching Nous.CodeRuntime's vocabulary: a malformed scheduler is Nous misusing its own seam, and the run_code call that carried it should fail cleanly rather than take down the run.

stop(scheduler, reason \\ :normal)

@spec stop(server(), term()) :: :ok

Tear the lane down, releasing every outstanding caller with an error.

Queued sub-calls are abandoned, in-flight tasks are terminated, and a sub-call that already produced a result still delivers it — in submission order. Safe to call on a scheduler that is already gone.

submit(scheduler, tool, args, run_ctx, owner \\ self())

@spec submit(server(), Nous.Tool.t(), map(), term(), pid()) ::
  {:ok, ticket()} | {:error, error()}

Queue a sub-call and return immediately.

The outcome is delivered to owner as {:sub_call, ref, outcome} when the call commits — that is, when every earlier sub-call has already committed. One process may hold many outstanding tickets, which is what lets a provider bridge an async guest without a process per pending host call.

Returns {:error, error} only when the scheduler is already gone; a submission never raises, because a binding that explodes is a worse failure than a program that reads an error.