Stream parser for JSON array responses.
Parses streaming HTTP responses where the body is a JSON array of objects:
[{"candidates":[...]},{"candidates":[...]},...]Used by providers (like Gemini) that stream responses as a JSON array
rather than Server-Sent Events. Has the same interface as
Nous.Providers.HTTP.parse_sse_buffer/1 so it can be used as a
drop-in :stream_parser for HTTP.stream/4.
How it works
Chunks arrive at arbitrary byte boundaries. The parser accumulates them
in a buffer, skips array-level syntax ([, ], ,, whitespace), and
extracts complete top-level JSON objects by tracking {} nesting depth
while respecting string literals and escape sequences.
Summary
Functions
Parse a buffer containing chunks of a JSON array into individual events.
Resumable form of parse_buffer/1.
Functions
Parse a buffer containing chunks of a JSON array into individual events.
Returns {events, remaining_buffer} where events is a list of parsed
JSON maps (same contract as HTTP.parse_sse_buffer/1).
Examples
iex> JSONArrayParser.parse_buffer(~s|[{"text":"hi"},{"text":"there"}]|)
{[%{"text" => "hi"}, %{"text" => "there"}], ""}
iex> JSONArrayParser.parse_buffer(~s|[{"text":"hi"},{"tex|)
{[%{"text" => "hi"}], ~s|{"tex|}
iex> JSONArrayParser.parse_buffer("")
{[], ""}@spec parse_buffer(String.t(), Nous.HTTP.Buffer.scan_state()) :: {list(), String.t(), Nous.HTTP.Buffer.scan_state()}
Resumable form of parse_buffer/1.
Returns {events, remaining_buffer, scan_state}. Pass the returned
scan_state back on the next chunk and the object scan picks up where
it stopped instead of re-walking the incomplete object from byte 0.
scan_state is nil (nothing partially scanned) or the
{pos, depth, in_string} triple the byte walker threads internally.
It is only valid against a buffer that still carries the already-scanned
prefix — i.e. the exact remaining_buffer returned alongside it, with
more bytes appended. Anything else falls back to a full rescan.
This is the optional arity of the :stream_parser contract;
Nous.HTTP.Buffer probes for it with function_exported?/3. Without it
a single JSON object spanning n chunks costs O(n²) byte steps — measured
at 226 ms for one 480 KB object across 342 chunks, 55x the one-shot
baseline at 240 KB (perf-audit, HIGH).
Examples
iex> JSONArrayParser.parse_buffer(~s|[{"a":1|, nil)
{[], ~s|{"a":1|, {6, 1, false}}
iex> JSONArrayParser.parse_buffer(~s|{"a":1}]|, {6, 1, false})
{[%{"a" => 1}], "", nil}