Your Codex quota reopened days ago, but Hermes is still on the fallback

Your Codex quota runs out on a Friday afternoon. Hermes switches to a fallback model and the work keeps moving — exactly as designed. Monday morning you check and the quota has been open for a while, yet that long-running session is still answering on the fallback until you restart it. That is not caution. It is an entire decision path that never got called.
Here is where the switch and the restore each happen, why a long-lived session gets stuck, and the reverse problem the same fix had to solve: when it is better not to restore at all.
The timescale fallback actually works on
Hermes fallback is turn-scoped: at the start of every turn it tries to return to the primary model. If it can, it does; if it cannot, it stays on the fallback. That is a good design — the moment a quota reopens, the next turn goes home, no restart and no manual switch required.
What blocks the restore is usually not code but time. A 429 puts the primary on cooldown until a specific moment, taken from the provider’s resets_at. For a weekly window that moment can be days away. And when the truth turns out to be the opposite — the quota reopened early because of a top-up, a redeemed reset, or a plan change — that cooldown is simply a pessimistic estimate.
Why a long-lived session never comes back
The credential pool already had the ability to notice an early reopen: a throttled /usage probe, iterated on more than once. The problem is which path it hangs off. The probe only runs inside select() — the code that picks a usable credential out of the pool. A session already pinned to its fallback never reaches select(), because it is not asking for a primary credential at all.
That creates a quiet loop: the probe knows the quota is back, but nobody asks it; the session waits for a provider-declared moment that may be days away, or for a restart.
The fix: make “try to go home” the probe point
PR #130716 hangs that check on the restore path:
CredentialPool.lift_reopened_cooldowns()performs the part ofselect()that clears elapsed or early-reopened cooldowns, but without leasing a credential — it only asks “is anything available now?”.restore_primary_runtimecalls it when the primary pool is benched. If an entry is usable again, the cooldown is cleared and the normal restore proceeds.- On any error, both cooldowns stay. Doubt resolves to “not yet”, never to “try the primary anyway”, and the path only applies when the primary is a Codex provider.
With that in place the loop is broken: every attempt to return to the primary also asks the probe, so a reopened quota gets noticed on the next turn.
The other half: do not retry every turn when the probe lies
The more interesting half of the same PR explains why a naive restore would be worse than none.
The probe’s verdict is cached for 300 seconds, and a fresh 429 does not clear that cache. So there is an awkward state where /usage reports quota left but real requests keep returning 429. With no throttle, the restore path becomes a machine with no brakes: restore the primary, fail, fall back, and do the whole dance again next turn. The PR’s validation table makes it concrete:
| Scenario | Before | First version of the fix | Now |
|---|---|---|---|
| A genuine early reopen | stays on the fallback | restores | restores |
| Probe lies, primary keeps 429ing, 6 turns | 0 restores | 6 restores, 6 failed calls, 6 notices | 1 restore, 1 failed call, 1 notice |
The rule now is at most one check per 300-second probe interval per agent, and the throttle is stamped before the pool is loaded. The turns in between do not even touch the pool — they could only replay the same cached verdict anyway.
What to look at on your machine
- This path is Codex-primary specific. Fallback behaviour for every other provider is unchanged; if you are not on a Codex primary, this fix does not alter anything for you.
- Do not treat a restart as the fix. Long-lived gateway and CLI sessions now come back on their own. If yours does not, first work out whether the probe says “no quota” or “quota yes, requests still failing” — the two need completely different handling.
- Manual control and inspection. The entry points for querying and switching live in the hermes fallback reference; the three layers of the mechanism (credential rotation, model fallback, chained fallback) are covered in fallback’s three layers.
- Which build has it. The fix merged into
mainon 2026-10-01, and the newest tag (v0.21.5) is dated 9-24, so it is not in the tag yet. Git installs trackingmainalready have it.
The takeaway
The shape here is two pieces of correct logic wired to nothing: the early-reopen probe already existed, the restore flow already existed, and neither called the other — so long-lived sessions sat on a pessimistic estimate. The fix is unglamorous: hang the probe on the restore path instead of inventing a new mechanism.
The other half is the part worth remembering. Every restore path needs a matching throttle. A probe that can be wrong, combined with a restore that runs every turn, does not buy you faster recovery — it buys you a failed attempt per turn. What shipped is two numbers — restore when it is real, try once when the probe lies — not “more aggressive”.