Hermes Background Review Cut Cost 39x With One Config Field
Hermes Agent merged PR #49252 on June 22, 2026. The change adds one config field -- auxiliary.background_review.{provider,model} -- and a single internal decision: when the review fork runs on a different model than the main chat, replay a digest instead of the full history. The reported result on a real 102K-token session: 39x cheaper at the cost of one missed skill and one missed memory.
The author is @teknium1 (Teknium, the project lead). The PR adds 341 lines across 4 files.
What the review is
After every agent turn, Hermes forks a background review whose job is to decide whether anything in that turn is durable enough to save -- a memory, a skill patch, a correction. It runs the full conversation history through the main chat model, reads the response, and writes to the memory or skills store if warranted.
The cost of that loop is non-trivial. On the headline benchmark in the PR, a single review on Opus 4.8 cost ~$3.34 against a 102K-token session. Multiply by every turn, every session, every user, and it dominates the agent's marginal cost.
The one design decision
The PR's central observation is about the prompt cache:
Your main chat is already warm in the prompt cache, so the review's default full-history replay on the main model is cheap cache reads. That's optimal and left untouched. A different model can't reuse that cache (different key), so a routed review is "cold" regardless -- replaying the full transcript would just cold-write it all. So the policy is simply:
- Review on the same model as your agent (default
auto) -> replay the full history. Warm cache reads. Unchanged.- Review on a different (cheaper) model -> replay a compact digest (recent turns verbatim + a summary of older ones). The cache is cold either way, so writing fewer tokens is a pure win.
The selector is a one-line check: does the configured review model differ from the main model? If yes, route and digest. If no, replay the full history. The default is auto, which means "use the main chat model" -- leaving the default behavior untouched.
The benchmark
Teknium ran the comparison live against the Anthropic API on a reconstructed copy of the PR's own build session (~102K tokens, 5 ground-truth durable signals: 3 skill, 2 memory). Two reps per arm.
| Review arm | Skill capture | Memory capture | Cost / review | Multiplier |
|---|---|---|---|---|
| Opus 4.8 full (default) | 5/6 | 4/4 | ~$3.34 | 1.0x |
| Haiku 4.5 digest (routed) | 4/6 | 3/4 | ~$0.086 | ~39x cheaper |
On a real, messy session with corrections buried under benchmark / CI / infographic tool-noise, Haiku dropped one skill and one memory versus Opus. The hardest signal -- a diffuse "verify premises before claiming" meta-lesson spread across many corrections -- was missed by Haiku both reps and Opus once. Even the full Opus arm wasn't perfect capture (5/6 in 1 of 2 reps).
The honest reading: the routed arm is cheaper by a factor of 39 and weaker by a signal or two per session. For a background review whose job is to opportunistically surface durable patterns, that's a real trade, not a free lunch.
Where the gap narrows
On 7 hand-built synthetic scenarios with signals less deeply buried, the gap closes:
| Review arm | Skill capture | Memory capture | Distractor false-saves |
|---|---|---|---|
| Opus 4.8 full (default) | 12/12 | 9/9 | 3/6 |
| Haiku 4.5 digest (routed) | 11/12 | 9/9 | 0/6 |
On cleaner signals, Haiku matches Opus on memory, loses one skill, and is cleaner on retracted distractors (0/6 false-saves vs Opus 3/6). Synthetic scenarios are not real sessions -- behavior at the 200K+ token range where attention dilution might favor the digest is untested -- but the conservative end of the range is the real-session number above.
What the digest does
_digest_history() in agent/background_review.py builds the compact replay used only on the routed path. From the PR test list:
- Tail preservation -- the most recent turns stay verbatim, since they're the highest-signal context for "what just happened."
- Alternation safety -- user/assistant turns are kept in the right order, with the model still seeing a real conversation shape.
- Arc capture -- older turns get summarized in a way that preserves multi-turn arcs (corrections that span many messages, retracted-then-restated positions).
The digest is never built on the default auto path. The main model always sees the full history, and the warm prompt cache stays warm.
What changes for the user
Nothing, by default. The change ships with auxiliary.background_review = auto, which is the same behavior as before -- the review runs on the main chat model and replays the full history.
To opt in, set the field in ~/.hermes/config.yaml:
auxiliary:
background_review:
provider: anthropic
model: claude-haiku-4-5
# base_url, api_key, timeout, extra_body also accepted
The selector checks: does the configured review model differ from the main chat model? If yes, route and digest. If no, full replay on the main model. The decision is automatic, not user-coded.
The PR also preserves a codex_app_server -> codex_responses downgrade for OpenAI Codex routes, and the fork only shares the parent's warm cached system prompt when not routed (so the digest arm doesn't accidentally benefit from a cache it can't reuse).
Why this matters
Three details from PR #49252 are worth carrying forward:
Cache-warm and cache-cold are different cost regimes. The same model gets cheap cache reads on the review fork because the parent turn already wrote the system prompt. A different model can't reuse that cache, so the cost comparison isn't "Opus price vs Haiku price" -- it's "Opus cache-read price vs Haiku cold-write price." The PR frames the whole routing decision around that asymmetry.
The default stays expensive. No budget guards, no cadence changes, no pre-pass, no behavior change unless the user opts in. Teknium's stated reason: the main-model review catches the most signals, and forcing the cheaper arm would silently degrade capture. The opt-in respects the trade.
Capture metrics matter as much as cost. A 39x cost cut is a headline, but the per-signal capture numbers (5/6 vs 4/6, 4/4 vs 3/4) are what make the feature defensible. The synthetic scenarios close the gap; the real-session numbers set the floor.
The full benchmark harness, raw results, and cache-cost recompute are in the reproducibility gist.
[^1]: Teknium. "feat(background-review): aux-model routing + context digest + adaptive cadence to cut self-improvement cost." NousResearch/hermes-agent. Merged 2026-06-22. [^2]: Teknium. "Hermes background-review cost-control benchmark -- data & harnesses." Reproducibility gist for PR #49252. [^3]: Anthropic. "Prompt caching." Cache-warm and cache-cold read pricing, mid-2026.