The `timings.py` module#

Summary#

Classes

TimingsCollector

Process-wide singleton. Thread-safe; no async required.

Functions

get_collector

Return the collector.

Description#

Lightweight in-process latency collector for the gateway.

We need to attribute “the system feels slow” to the right layer: HTTP endpoint, tool dispatch, or live backend call. Doing that with logs alone is hard because the user has to scroll a noisy file. This module keeps a fixed-size counter table per scope and exposes it through GET /v1/diagnostics/timings.

Design constraints#

Always-on, near-zero overhead. A timed block does one time.perf_counter() pair plus three dict updates under a single threading.Lock. No allocations per call beyond the key string the caller already has.
Bounded memory. Each scope (http / tool / backend) keeps at most _MAX_KEYS distinct keys (1024). When the table is full, new keys are silently dropped — we prefer stale-but-bounded to unbounded growth.
No external deps. Stdlib only. The collector is importable from
any module without pulling FastAPI, product SDKs, or asyncio in.
Reset on demand. GET /v1/diagnostics/timings?reset=true clears the counters atomically so the user can take a fresh sample around a specific user action.

Records kept per (scope, key)#

count — number of completed calls.
total_ms — cumulative wall time.
min_ms / max_ms — extremes.
last_ms — most recent sample (helps catch cold-start outliers).
errors — number of calls that left the timed block via an exception. We still record the elapsed time so a slow failure doesn’t disappear from the table.

Module detail#

timings.get_collector() → TimingsCollector#