Three Perf PRs Land Together: Token Accounting, Wire Client Reuse, Dashboard Query

X API search was unavailable for this roundup (402 Payment Required). Discovery pivoted to the GitHub PR feed.

Three performance PRs merged within hours of each other on July 28, all targeting different latency paths in the Hermes Agent runtime. Two are salvages of earlier community work by Soju06; the third is a parallel optimization. Combined, they add 2,046 lines across 17 files.

Token accounting off the turn thread

PR #73359 takes per-API-call token and cost accounting off the critical path. Before this change, SessionDB applied deltas synchronously on the turn thread -- p50 ~0.54 ms, with spikes up to 300 ms on cold multi-GB state databases. The fix moves accounting to a background single-writer queue, cutting the turn-thread cost to ~0.0005 ms while keeping persisted totals byte-exact.

The PR is a salvage of #64171 with three follow-up hardening commits. It touches 7 files (+1,030/-16), with the core change in hermes_state.py (+304/-3) adding the queue-based writer.

OpenAI wire client reuse

PR #73375 eliminates repeated TCP and TLS handshakes (~20-40ms each) by reusing the per-request httpx pool across sequential LLM calls. Previously, every call rebuilt a fresh client pool, adding connection overhead to every turn. The reuse only triggers when the previous request completed cleanly on its owning thread -- errors, interrupts, and cross-thread aborts skip reuse to avoid state corruption.

Another salvage, this one of #64170. 6 files, +899/-30, with a 289-line test suite for the reuse logic.

Dashboard query optimization

PR #73362 replaces a materialization loop over list_sessions_rich(limit=10000) with a single GROUP BY query. The /api/sessions/stats source histogram dropped from ~575ms to under 1ms on large databases. 4 files, +117/-3.

Other notable merges

The same window saw desktop UX improvements:

• PR #73172 (+428/-42): the model picker now defaults to the newest 5 models per lab instead of showing every curated model. Collapsible providers and select-all in Edit Models.
• PR #73220 (+497/-24): delegate_task calls now render as live child rows with goals, model names, and timers instead of a single grey row.
• PR #73221 (+344/-12): the attachment data-URL size limit is now configurable from Settings (1-4,096 MB, default 16 MB).

The perf cluster is the headline: three PRs landing together, two salvaging community contributions that had been sitting open since #64170 and #64171. The pattern -- community-authored perf work, picked up and landed by a core contributor -- is how the latency-critical parts of the runtime advance between releases.

The combined effect of these three changes is measurable. Token accounting no longer blocks turns. HTTP connections no longer get torn down and rebuilt between calls. The dashboard stats endpoint returns in under a millisecond instead of half a second. Each optimization targets a different subsystem, but together they reduce the per-turn overhead that compounds across long-running agent sessions.

[^1]: PR #73359 -- perf: take per-API-call token accounting off the turn thread. kshitijk4poor, merged 2026-07-28. [^2]: PR #73375 -- perf(agent): reuse the per-request OpenAI wire client. kshitijk4poor, merged 2026-07-28. [^3]: PR #73362 -- perf(dashboard): use GROUP BY for session stats. kshitijk4poor, merged 2026-07-28. [^4]: PR #73172 -- feat(picker): curated model defaults. OutThisLife, merged 2026-07-28. [^5]: PR #73220 -- Show a delegation as its subagents, live. OutThisLife, merged 2026-07-28. [^6]: PR #73221 -- feat(desktop): configurable attachment size limit. Adolanium, merged 2026-07-28.