The model no longer forgets what it thought: reasoning replays across turns, reloads and compaction

You have seen this: a debugging session runs for half an hour, you bring it back with --resume, and the model behaves like it just met you. The conclusions it had already reached, the two directions it had just ruled out — gone. You explain the context again from scratch. Worse, you ask the same question about the same error a third time and get a different diagnosis each time.
Most people put that down to the model having a bad memory. One very concrete cause is that Hermes was not carrying back everything the model had thought. Reasoning content has to be echoed back on the next turn on many providers, and Hermes only did that for a handful of routes. Everywhere else it was dropped the moment it left the model, and compaction and session branching actively deleted reasoning from earlier turns.
Changes merged in the current window turn replay from an exception into the default.
Why reasoning has to travel with the conversation
First the plain-English version, so the rest is not all jargon. A reasoning model produces a “thinking” block before it answers. It can be readable text, or an encrypted or signed blob — providers name these differently: reasoning_content, reasoning_details, a signed thinking block, Gemini’s thoughtSignature, and so on.
Three properties matter:
- Some providers treat it as part of the conversation history: leave the previous turn’s thinking out of the next request and quality drops noticeably, or the request fails with a 400 once tool calls are involved.
- Some providers encrypt or sign it, so it only works when echoed back verbatim — reconstructing it yourself produces nothing useful.
- Some providers run a strict request schema where one extra field is a hard error, so for those routes you must not send a byte more.
That is why replay was never “just include everything”. It is per-route dispatch: each route knows what it reads and what it must never be sent.
How it used to behave
Before this change, the behaviour was best described as “a few routes were taken care of”:
reasoning_contentonly went back to DeepSeek, Kimi, MiMo and Ollama Cloud.reasoning_detailsonly went back to openrouter.ai.- Signed Claude thinking was dropped on Bedrock, Vertex and Foundry.
- Gemini text-turn
thoughtSignaturewas never captured at all. - Compaction removed earlier-turn reasoning from canonical history.
The result is the two experiences above: a model that re-guesses after --resume, and a session whose quality drops off a cliff right after compaction.
How it works now: one route table plus a private carrier record
One route table. Each route now maps to the carrier fields it can read (_REASONING_CARRIER_ROUTES in message_sanitization):
| Route | Fields it gets |
|---|---|
| OpenRouter, Kilo, Vercel AI Gateway, Nous Portal | all three: reasoning_content, reasoning, reasoning_details (signed and encrypted blocks verbatim) |
| Novita, MiniMax, Tencent | reasoning_content + reasoning_details |
| Local servers | reasoning_content + reasoning |
| Everything else | reasoning_content |
| Strict schemas (Mistral, Groq, Cerebras, SambaNova, Hunyuan) | none |
| Must-echo tier (DeepSeek, Kimi, MiMo and friends) | a " " placeholder only, to stop the 400 on tool turns with no stored reasoning — it is not reasoning itself |
The vendor keep-thinking flags come along too: GLM’s clear_thinking: false, DashScope’s preserve_thinking, Kimi’s keep: "all".
A private carrier record. Fields like Gemini’s thoughtSignature and Copilot’s reasoning_text / reasoning_opaque / thought_signature do not belong to the standard shape — there was nowhere to put them. They are now stored as a private hermes.native_assistant record (implemented in agent/reasoning_carriers.py, with StreamCarriers accumulating them from streamed deltas), and only their own route ever receives that record. Other routes never see the field, so they never get a 400 for it. Because the record lives in the session, it survives gateway turns and --resume.
Bedrock Converse gets its own handling. It requests thinking for Claude and keeps a per-model replay policy (signed / strip / unsigned), and stores reasoning as Anthropic-shaped reasoning_details so a turn reloaded from state.db replays as well. That is precisely the line in the v0.21.7 release body: signed thinking replays after hermes chat --resume and on the next gateway turn.
When a route refuses: record it, retry once
Even a careful route table goes stale — what a provider accepts today it may reject tomorrow. So there is a fallback:
When a request returns a 4xx whose message names a replayed field (only the unknown-field phrasing counts), Hermes records that verdict per (provider, host, model), persists it in the session’s model config so --resume keeps the memory, then retries once without that field.
Three constraints keep this from becoming a liability:
- The must-echo tier can never record
reasoning_content, because that field is required. - History is never mutated — the field is omitted from the request only.
- It retries exactly once, so it cannot loop on a failure.
Compaction and branching stop eating reasoning
The other half is retention:
- Compaction (
compress()and the salvage pass) no longer removes reasoning text or encrypted items from retained turns. Only checkpoints shadowed by a newer one are dropped. - CLI
/branchcopiesreasoning_content, so a new branch carries its own history. - The preflight estimate and the compaction tail walk price opaque carriers identically, so compaction does not loop trying to account for them.
What it changes for you
- A resumed session behaves like the session you left. Long debugging sessions picked up with
--resume, or continued across gateway turns, keep the conclusions they had already reached. - Compaction keeps quality steadier. After the context is compressed, the model still sees its own earlier thinking instead of just the conclusions.
- Claude users on Bedrock, Vertex and Foundry. Signed thinking is no longer dropped, and resumed sessions are no longer rejected wholesale.
- Gemini and Copilot users. Signatures and opaque fields that were never captured before now travel with the conversation.
Where to look when it misbehaves
# does the session carry a carrier record
hermes sessions show <session-id> | head -40
# watch which reasoning fields a request carries (debug logging)
hermes logs --follow | grep -i reasoning
# where session state lives, to confirm reasoning is kept per turn
ls -l "$HERMES_HOME/state.db"
If you see a route log an unknown-field rejection and then retry, that is the fallback doing its job — it usually recovers on the next turn. If a route keeps failing the same way, it is worth an issue: the route table needs updating.
For the other reasoning knobs, see our reasoning effort guide. The operational side of resuming is in the session resume guide, and the background on compaction is in the compression lean-tail guide.