Metrics pipeline

medium 60 min Python Stripe Bug Squash-styleGoogle code comprehension-style

1 The same minute, three rows
2 Build a feature
3 Make it fast

repo

metrics/
tests/
visible/

read-only

# metrics

Turns our API servers' access logs into per-minute metrics for the
dashboard. A log line looks like this:

    2026-09-14T21:59:58.120+02:00 web-3 GET /api/orders/88?full=1 200 123ms

timestamp (ISO 8601 with its UTC offset; no offset means UTC), host, method,
path, HTTP status, latency. Servers log in their own local time: the
Frankfurt hosts use `+02:00`, the Virginia ones `-04:00`, some use `Z`. The
log collector merges every host into one file **in time order**.

    metrics/parse.py      parse_line(): one line -> Event, or None for junk
    metrics/endpoints.py  endpoint_for(): /api/orders/88?full=1 -> /api/orders/:id
    metrics/buckets.py    minute_of(): which UTC minute an event belongs to
    metrics/aggregate.py  Stats per (minute, endpoint)
    metrics/export.py     the CSV the dashboard loads
    metrics/pipeline.py   run(lines, out): parse, aggregate, write CSV
    metrics/cli.py        python -m metrics.cli access.log > metrics.csv

The CSV has one row per UTC minute and endpoint, sorted by minute, then
endpoint:

    minute,endpoint,count,errors,avg_ms,max_ms
    2026-09-14T19:59Z,/api/orders/:id,412,3,87.4,1630

`errors` counts 5xx responses. Lines that don't parse are skipped and counted.

Run the tests with `python -m pytest` (or press Run in the browser).

Phase 1 of 3 0:00

The same minute, three rows

Ticket #3107 · priority: high · from: SRE

Since we added the Frankfurt and Virginia hosts, the latency dashboard looks wrong. Each minute's traffic shows up as three separate rows, hours apart, and the 19:59 spike from Tuesday's incident appears at 21:59 and at 15:59 too. Rows are supposed to be per UTC minute, whichever host served the request.

Hosts log in their own local time, with their UTC offset (see the README). The CSV comes from run() in metrics/pipeline.py.

tests/visible/test_pipeline.py reproduces it. Make it pass.
On Submit, hidden tests try more offsets (including half-hour ones), dates that change across midnight and new year, and events right on the edge of a minute.

Bring your own AI: paste the copied repo into Claude, ChatGPT or Gemini. In a real interview the assistant usually reads the files itself. Or download the zip, work in your own editor, and upload the files you changed.