~/problems / AI infrastructure

Streaming response parser

On a phone? Coding is easier on a laptop: email this problem to yourself . Meanwhile: fight a boss.

medium assessment 4 levels ~45 min

Level 1 Events from whole lines

A chat model streams its answer as server-sent events: plain text lines over one long HTTP response. Your client receives that text in chunks and must turn it into events.

Build SSEParser() with feed(chunk: str) -> list[tuple[str, str]]. It returns the events completed by this chunk, in order, each as (type, data).

For this level the stream is simple:

  • Every line ends with \n, and every chunk holds whole lines.
  • A field line is a name, a colon, a space, then the value: data: hello has field data and value hello. Split at the first colon; the value may contain more colons.
  • data: <text> gives the event's data. event: <name> gives its type. Each event has exactly one data line and at most one event line.
  • An empty line ends the event. Its type is "message" if it had no event line.
p = SSEParser()
p.feed("data: hello\n\n")              # [("message", "hello")]
p.feed("event: price\ndata: 42\n\n")   # [("price", "42")]
p.feed("data: a\n\ndata: b\n")         # [("message", "a")]  (b isn't finished yet)
p.feed("\n")                           # [("message", "b")]

The stream is ASCII text, up to 10^6 characters in total over all chunks.

Level 2 unlocks when level 1 passes.

Level 3 unlocks when level 2 passes.

Level 4 unlocks when level 3 passes.

Topic: AI infrastructure. The plumbing around models: request batching, streaming responses, prompt caches, sampling, token limits and eval harnesses.

0:00
Ctrl ' run · Ctrl ↵ submit
esc