Your prompt cache expires quietly. Hermes now shows you when


Friday night you leave a long refactoring session open. Every turn that afternoon was cheap: the same 40,000-token system prompt and file context went up once, and the provider charged a fraction of the price for sending it again. Monday morning you resume the session, send one small follow-up, and the bill for that single turn looks like a fresh conversation. Nothing changed in your setup. You just have no idea whether the cache expired over the weekend, whether the model switch you tried on Friday reset it, or whether a context compaction quietly rewrote the prefix the cache was holding.

That uncertainty is what this change is about. Not a new cache, not a new provider feature — a clock that records what actually happened to your prompt cache, and a single line in /context that tells you how long you can still count on it.

Prompt caching is a discount with a deadline

The idea is simple. Providers let you mark part of a request as reusable. They keep the processed prefix on their side for a while, and when the next request starts with the same prefix, they bill the cached part at a much lower rate instead of processing it again. On a long agent session with a big system prompt, stable tool schemas and a growing transcript, that discount is most of your bill.

The catch is the “for a while” part. Every provider words it differently:

  • Anthropic caches default to five minutes and can be requested with a one-hour tier. A cache read restarts the clock.
  • OpenAI’s newer models take an explicit prompt_cache_options.ttl — 30 minutes by default, 12 hours on request. A read keeps the 30-minute floor but does not restart a 12-hour window.
  • Older OpenAI routes and implicit caches have no published minimum at all. They may keep entries longer than requested; nobody promises an exact expiry.

Notice what none of these give you: an expiry timestamp. They give you a minimum retention. “Five minutes” means “at least five minutes”, not “gone at five minutes and one second”. Any code that treats it as an expiry is guessing, and Hermes used to do that guessing independently in every place that cared.

What was actually wrong

Two separate problems hid behind the same vague idea of “the cache is probably warm”.

The first was duplicated guesses. Different parts of the agent derived a TTL on their own. When two of them disagree, the cheaper one wins by accident: you make a decision assuming warmth that a different piece of code would have called cold.

The second was worse, and it cost real money. On Nous Portal, Hermes asked for the one-hour cache tier. The Portal keeps the cache for about five minutes and drops it. So Hermes paid the higher cache-write rate for a one-hour entry, and the entry was gone before the next turn. The request said “1h” on the wire; reality was “5m”.

PR #156101 fixes both by making the cache window a recorded fact instead of a recomputed guess.

One clock, written down per attempt

The new module is agent/prompt_cache_clock.py. For every successful model attempt that touched the cache, it records one event:

  • the full route (provider, normalized base URL, model),
  • the tier actually sent on the wire — read from the payload, not from config: Anthropic’s cache_control ttl, OpenAI’s prompt_cache_options.ttl, older prompt_cache_retention,
  • the send time,
  • read and write token counts,
  • the tier or tiers the response proved (Anthropic’s ephemeral 5m/1h counters, OpenAI’s ttl_30m/ttl_12h counters),
  • and the resulting coverage: one deadline per tier that the cached prefix holds.

That last part is the useful bit. The clock does not store “TTL = 300 seconds”. It stores “this cached prefix is covered until 08:11, because a 5-minute tier was written at 08:06 and the response confirmed it”. A deadline is something you can compare against the current time; a TTL is something you have to re-derive, and every re-derivation is a chance to be wrong.

Because the event is bound to the session id, the full route, and the session’s durable context generation, it can also say unknown honestly. Switch models, fall back to another provider, or run a compaction that archives rows, and the old deadline no longer describes the prompt you are about to send. Hermes drops the window to unknown instead of pretending.

The refresh rules differ per provider, so they are data

A cache deadline is not “write time plus TTL”. It depends on what a read does to the window, and providers disagree:

Route class On a cache read Deadline rule
Anthropic (native, OpenRouter Claude, Anthropic-wire gateways) restarts the tier read time + tier
OpenAI GPT-5.6 and newer, 30 m tier refreshes the floor max(existing, read + 30 m)
OpenAI GPT-5.6 and newer, 12 h tier does not restart it write time + 12 h, fixed
Everything else no published guarantee no guaranteed deadline, estimated end only

This is why the module keeps a dict of per-route handlers instead of one formula. The Anthropic rule and the OpenAI rule are genuinely different behaviours, and a single “add TTL to last activity” rule would be wrong for at least one of them. On the 12-hour route it would be badly wrong: a long conversation would keep looking warm forever because every read pushed the deadline out.

Two ends of one window, because a minimum is not an expiry

Here is the design decision that makes the line in /context honest. Every window has two ends:

  • guaranteed_until — the earliest coverage deadline. If a response wrote both a 5-minute and a one-hour tier, the prefix is only guaranteed for five minutes, so that is the number. None means no guarantee at all.
  • estimated_until — the point after which the entry is certainly gone. For Anthropic tiers this is exact, because the tier is exact. For OpenAI it is last touch plus 24 hours, because OpenAI says entries may be retained longer than the requested minimum and keeps cache state for at most a day. For the old in_memory mode it is up to one hour.

Between the two, the honest answer is “may still be warm”. That is a real answer, not a cop-out: it tells you a miss is possible and a hit is possible, and it stops you from planning around a guarantee that does not exist. The consumers ask cache_certainly_warm or cache_certainly_cold, and anything unknown is neither.

What /context prints now

Four shapes, all one line:

Prompt cache: warm until 08:11 (4m 57s left; 5m, declared).
Prompt cache: may still be warm until 08:36 (29m 57s left; 30m, proven).
Prompt cache: expired 12m ago (5m, proven).
Prompt cache: unknown — the route changed since this cache was written.

The trailing label is a confidence level, and it matters:

  • proven — the response itself named the tier it kept. This is the strongest evidence available.
  • declared — Hermes sent that tier on a route known to honour it, and nothing has invalidated it.
  • assumed — no published guarantee; treat it as a guess.
  • unknown — the cache moved (route switch, fallback, compaction, session rotation), no published retention, or the stored event could not be read.

One more detail worth knowing: reading the clock is advisory. If the SQLite read fails, or a worker’s owner RPC is unavailable, the window becomes unknown with a reason. It never raises into the turn that produced your reply. A cost-reporting feature should not be able to break a conversation.

The numbers from the live run

The PR’s author ran the same two-turn scenario on three routes — turn one writes the cache, turn two resumes the conversation in a new process with --resume, so the saved proof has to survive a restart:

Route Before After
Nous Portal via custom, cache_ttl: 1h wire said ttl: 1h, no cache line in /context wire says ephemeral (5 m); 7,128 tokens written; warm until 08:11 (4m 57s left; 5m, declared)
OpenAI gpt-5.6-luna, Responses, default 30 m no cache line, write-token details dropped 6,662 written; warm until 08:36 (29m 57s left; 30m, proven), estimated end +24 h
OpenAI gpt-5.6-luna, ttl: 12h via extra_body n/a 6,664 written; warm until 20:06 (11h 59m left; 12h, proven); the resumed read left the deadline unchanged

Billed estimates from Hermes’ own pricing: about $0.029 for the Nous run and $0.009 for the 30-minute OpenAI run. The 12-hour run went through a custom route Hermes does not price, so it shows $0.00. Those are small because the prompts are about 6.7k tokens — the point is the shape of the difference, not the cents.

How to use it today

You do not have to configure anything to get the /context line; it appears once a turn has read or written the cache on a route Hermes knows how to class. Two knobs are worth knowing:

1. Stop asking for a tier the provider does not keep. If you pinned a one-hour cache on a route that drops entries after five minutes, Hermes now sends what the route actually honours. On Nous Portal that is the 5-minute tier.

2. Ask for a longer tier where it is genuinely supported. OpenAI’s newer models accept a longer TTL through prompt_cache_options. Set it per custom provider:

custom_providers:
  - name: openai-long-cache
    base_url: https://api.openai.com/v1
    api_key_env: OPENAI_API_KEY
    model: gpt-5.6-luna
    extra_body:
      prompt_cache_options:
        ttl: 12h

Then check /context after the first turn. If the line says 12h, proven, you have a real 12-hour window. If it says unknown, the route did not confirm the tier — fall back to treating it as a guess.

What it does not do yet

Be precise about the scope. This change records and reports; it does not yet act on the window. The module docstring names the intended future consumers: keep-alive pings before guaranteed_until, cold-cache pruning, and idle compaction that avoids throwing away a prefix you are about to reuse. None of those ship here.

It is also on main only, not in v0.23.1 (tagged November 9, 2026). If you run a stable release, /context has no cache line yet. And three routes were not exercised live because the keys were unavailable: OpenRouter Claude with the one-hour tier, native Anthropic ephemeral_1h proof, and mixed 5 m/1 h writes. They are covered by unit tests, which is weaker evidence than a live run, so treat those specific claims accordingly.

Update (2026-11-17): these changes shipped in v0.23.2 (November 16) — hermes update brings them in. The “not in v0.23.1” wording above is a snapshot from publication time.

A short checklist for keeping the cache warm

The clock mainly makes an existing cost problem visible. Once you can see the line, the habits follow:

  • Do not switch models mid-task if you care about cache economics. A model switch changes the route, and the window becomes unknown by design.
  • Watch compactions. A compaction archives rows, changes the context generation, and invalidates the recorded window — even if the new prefix happens to look similar.
  • Keep the head of the prompt stable. System prompt, tool schemas and early file context are what gets cached. Reordering tools or editing the system prompt at turn 30 throws away the discount for everything after it.
  • Read the confidence label, not just the time. proven is evidence; declared is intent; assumed is a guess. Planning a long autonomous run around an assumed window is how you get a surprise bill.

If you have been treating “the cache is probably still warm” as a fact, this is the first time Hermes can tell you whether it is. For the token-side view of the same problem, our earlier posts on sibling subagent prompt caching and Claude window metering cover what else moves the bill; the configuration reference covers where these keys live.