The Agent Went Quiet at Minute Four — Hermes' Timeout Layers, Explained

It is 2:40 in the morning and the cron job you started at 2:05 still says running. The last log line is the request going out. No error, no crash, no retry — just an HTTP call sitting there, the connection open, the agent waiting for a first token that is never coming. Meanwhile in your editor, a different flavour of the same problem: a model that thinks for four minutes gets cut off at three, and the retry produces the same cut-off, and you conclude the model is broken. Both of these are timeout questions, and Hermes answers them in layers — several of which quietly changed on 2026-09-25.
Silence is the failure mode nobody designs for
The two failures people bring up most often look opposite but have one root: an agent that is doing something is easy to reason about, and an agent that is silent is not. Silence can mean a model is deep in a thinking phase, or it can mean the socket is a zombie. The same 60 seconds of quiet covers both.
So the stack does not ask “is it slow?” It asks “how long may silence last, at this specific layer?” — and the answer differs per layer on purpose:
| Layer | Default | What it actually watches |
|---|---|---|
| Socket read timeout | 120s (1800s on local) | httpx waiting for the next chunk |
| Stale stream detection | 180s (900s ceiling on local) | SSE keep-alives arriving with no content |
| Stale non-stream detection | 90s | A non-streaming call that never answers |
| Responses first-event watchdog | 120s (900s ceiling on local) | Connection accepted, no stream event at all |
| API call (non-streaming) | 1800s | The whole non-streaming request |
| Post-terminal stream drain | 2s | Relays that never close the SSE socket |
The single most useful sentence in this table: the stale detectors auto-adjust only while they sit at their implicit defaults. Set one explicitly and you have taken responsibility for it — which is the right call for a slow local server, and the wrong one for a cloud model having a bad night.
The 180-second default is a lie for reasoning models
Chat models and reasoning models are not the same animal. A chat model that accepts a request and emits nothing for three minutes is almost certainly dead. A reasoning model doing that is frequently just thinking. The upstream proxies in between do not care about the difference, and idle-kill the stream mid-think, which arrives as a broken pipe or a protocol error rather than anything that says “your model was still working”.
So the project keeps a floor table — a per-model minimum patience that scales the stale detector upward with max(default, floor). It never lowers a threshold and it never overrides a value you set yourself. The current floors, read out of the source:
| Floor | Models covered (selection) |
|---|---|
| 600s | Nemotron-3 Ultra/Super, DeepSeek reasoning and V4 lines, OpenAI o1/o3/o3-pro family, the gpt-5.6 and gpt-6 named lines, claude-fable |
| 300s | Nemotron-3 Nano, QwQ-32B, o3-mini/o4-mini, the explicit xAI Grok reasoning pairs, ox-alpha, Inkling, MiniMax M2.x |
| 240s | Claude Opus 4.x and 5.x thinking variants |
| 180s | qwen3 family, Claude Sonnet 5 / 4.5 / 4.6 |
MiniMax M2.x is the newest entry, and it arrived because of three bug reports rolled into PR #121555: M2.x streams reasoning content before its first content token and had been observed stalling for around 240 seconds mid-think — past the 180s default, so it was being killed while healthy.
Context size scales the same way, because a 100k-token prompt is a slower first token than a 2k one: above 50k estimated tokens the streaming detector rises to at least 240s, above 100k to at least 300s. The non-streaming side has its own tiers. Only the implicit defaults move — your explicit numbers are untouched.
The run budget that a hung call could ignore
agent.run_budget_seconds (or hermes chat --run-budget 850 -q "...") exists to stop an autonomous run from spending forever. Two things happen when it is set: at 80% of the budget the agent gets a one-time wrap-up notice telling it to stop exploring and produce the deliverable, and implicit stale timeouts get capped at half the remaining budget, with a 60-second floor, so one silently-hung call cannot swallow the rest of the run.
The bug was the gap between documentation and implementation on the streaming path: the cap applied to non-streaming calls, but a hung stream kept its full stale window, and the wrap-up notice could not fire while a call was in flight. A run with a 10-minute budget and a wedged provider stream could sail past its own deadline — the notice was reachable in theory and unreachable in practice. The fix routes both resolvers through one cap_to_run_budget, so the streaming and non-streaming stale timeouts are computed the same way. One asymmetric detail is deliberate: an explicit HERMES_STREAM_STALE_TIMEOUT is treated as your deadline and is never capped to the run budget, while implicit values are.
Cron: the second call that hung
The third fix in the same PR is the one that explains a class of “my scheduled job just never finishes”. Cron Codex calls ran on the worker-thread path and could hang on the second consecutive streaming call — the first worked, the second waited forever. Cron runs (chat_completions and codex_responses) now execute inline, and, more importantly, the inline path gets the same large-context floor and hard cap the worker path got. In the measurement quoted in the PR: at a 30k-token probe, the inline path previously allowed 90 seconds where the worker allowed 600; now both compute 600, and with the hard cap configured at 300 both compute 300.
If you run unattended automation, this is the change to know about: two code paths that should have had identical patience, one of which was 6.7× less patient and could wedge without a timeout ever firing.
Which knob wins
When several values could apply, the precedence is explicit and worth learning once:
- Explicit config beats every implicit floor.
providers.<id>.stale_timeout_seconds(orproviders.<id>.models.<model>.stale_timeout_seconds) is the operator’s deadline: it wins over the context-size tiers and the reasoning floor, and it can therefore shorten patience for a hung stream on purpose. - The 180s default is the only value that gets scaled and floored. Leave it alone and you get context scaling plus the model floor for free.
HERMES_STREAM_STALE_TIMEOUTis a user value. It is honoured as-is, never capped by the run budget.- Local endpoints get finite ceilings, not infinities.
agent.local_stream_stale_timeout(900s) applies when a local endpoint is detected at implicit defaults — the former “never time out” behaviour meant a wedged local server hung forever. HERMES_STREAM_STALE_GIVEUP(default 5, 0 disables) is the circuit breaker: after that many consecutive stale kills with no completed response, each new call aborts immediately with an actionable error instead of waiting out the timeout again. It resets on any completed response, a/modelswitch, fallback activation, or a fresh turn.
A working example for a self-hosted reasoning model that legitimately thinks for minutes:
providers:
local-reasoning:
stale_timeout_seconds: 900 # explicit: never scaled, never capped
and for a cloud reasoning provider you trust, the opposite move — leave it implicit so the floor and context tiers protect you:
agent:
run_budget_seconds: 1800 # wrap-up notice at 80%, stale caps at half the remainder
How to tell what killed your call
- A cut-off with a retry that succeeds is usually the stale detector firing during a long thinking phase — check whether your model is in the floor table, and whether you are past 50k tokens of context.
- Five consecutive kills and then an immediate abort is the giveup breaker, not the provider: the error text names the streak. Switch models or start a new session; the counter resets on the next completed response.
- A run that ignores its own budget was the streaming-path gap — a symptom worth re-testing rather than working around with a bigger timeout.
- A cron job that hangs on its second provider call matches the inline-execution fix in this batch.
What shipped, and where
All three fixes are in PR #121555, merged 2026-09-25 — after the v0.21.5 tag was cut, so they are on main only and arrive with the next release or a hermes update that tracks main. If you are tuning timeouts right now, the layering above already works today; it is the two implementation gaps that the fix closes.
Related reading on this site: the silent disconnect post covers the transport-level version of the same problem — connections that look alive and are not — and the gateway loop watchdog guide covers the third layer, the one that watches the loop rather than a single call. If your problem is not “it hangs” but “it costs”, the cost tracking writeup is the one to read next.
What to take away
Silence needs a budget at every layer, and the budget has to be honest about what it is measuring. For reasoning models the honest default is much longer than for chat models; for an unattended run the honest default is the run’s own deadline. The pattern to carry into your own systems is the same as here: tier your timeouts by what the silence means — no event yet, no content yet, no answer yet, task has run too long — and make sure every code path that can wait uses the same tiers. A path that quietly waits 90 seconds where its sibling waits 600 is not a tuning difference; it is a bug with a stopwatch.