Hermes 'Forgot' 798 Messages After a Crash — the Session Amnesia Bug, Explained and Fixed

Monday morning. You open Telegram to pick up the thread you left with Hermes last night — and it starts making small talk about a topic that should have been closed three days ago. “Did that PR get merged?”
That’s not the model being confused, and it’s not your imagination. It’s exactly what happened to Teknium, the lead behind Hermes Agent, in early August 2026: after a gateway crash and restart, his Telegram DM silently resumed a 2.7-day-old context, and the 798 real messages exchanged in between seemingly vanished. Nothing was lost — Hermes just couldn’t find them.
What happened: how one message sent a session back in time
The official tracking issue (#82616) includes production database evidence. The incident unfolded in four steps:
| When | What happened |
|---|---|
| Aug 6, 16:18 | An inbound message triggers a recovery path that quietly creates a new session row with no identity |
| Aug 6 → Aug 8 | All 798 real messages land in this orphan row; in-memory mapping keeps the conversation looking perfectly normal — the fork is invisible |
| Aug 8, 17:03 | The gateway crashes and restarts; the in-memory “chat → session” mapping is discarded as stale |
| Aug 9, 09:40 | The next message arrives: Hermes resolves the chat by session key, and the only row carrying the key is the stale Aug 3 session — so it resumes a 3-day-old context and chats about a PR merged days earlier |
The database evidence shows two rows: the Aug 3 “original” (full session key, but ended_at empty and no new messages), and the Aug 6 orphan (session_key, chat_id all NULL — but all 798 messages hanging off it).
In plain words: Hermes quietly “forked” a nameless copy of the session, every message went into the copy, a restart wiped the in-memory address book, and when Hermes looked for the session again it only recognized the row with a name tag — so it went back in time.
Root cause: three failures stacked, all of them silent
The deep-dive (full code-level analysis in the issue’s comments) converged on a single chain:
- Session-rotation DB writes failed, and the failures were swallowed. Hermes’s idle auto-reset ends the old session and creates a new one — both writes failed: one logged at debug level, the other a bare
printto the console. Nobody noticed. - Yet the routing table had already switched to the new session. The old session became a “zombie” (never marked ended, still holding the session key), while the new session didn’t exist on disk at all.
- A lazy writer materialized the ghost row. A later background write (e.g. token-usage accounting) found the target row missing and created it on the fly — but with every identity field (
session_key,chat_id, …) empty. The orphan was born, and no code path ever re-stamps identity on it. - After the crash, restart resolution couldn’t see the orphan. It looks up sessions by key and by chat info — the orphan matches neither, so the only candidate is the zombie “original.” It wins, and the old context comes back.
There’s a fascinating twist: the logs kept saying possible FTS write corruption and disk=0 (disk read-back of 0 rows), so everyone first suspected database corruption. The investigation proved the read path never touches the full-text index — the real issue was session-ID routing: the writer followed a reroute map, the reader queried the old ID, and got 0 rows. The database was healthy the entire time, which is exactly why not a single message was lost.
Nothing was lost: the 798 messages live in a “hidden” row
The most reassuring part of this whole incident: all 798 messages are intact — they’re just sitting in a row with no identity tag, invisible to the resolver. For affected users, the issue documents a manual recovery procedure (below).
The fix: prevention + treatment, merged to main 2026-08-09
The fix ships as two PRs — one so it never happens again, one for the damage that already exists:
#82633 — write-side hardening (prevention)
- Identity written atomically: session key, chat ID, and origin are now part of the row-creation INSERT itself (
ON CONFLICTbackfills via COALESCE), instead of a best-effort post-creation UPDATE. Identity and row now live or die together. - Every refresh is a repair opportunity: the gateway refreshes session info every turn; if a row is missing, it now inserts the full identity instead of silently no-op’ing.
- Recency-aware resolution: after a crash/restart, candidate sessions are ranked by
last_activity_atwith message-bearing rows first, and a fresh ID is never minted while a keyed row exists. - Errors stop being silent: end/create write failures are logged at WARNING with the routing consequence spelled out; transcript-read exceptions no longer silently return an empty list.
- Also fixes #12857: session resets no longer lose the parent session ID (lineage).
The validation was brutal: 11 regression tests run against unfixed main produced 6 failures; after the fix, all pass. An end-to-end incident replay (3-day zombie + 798-message orphan on a real database) resolves back to the live conversation.
#82712 — hermes sessions repair-routing (treatment)
A new command that “rescues” orphaned sessions, designed around a fail-closed principle:
hermes sessions repair-routing # dry-run first: report findings, change nothing
hermes sessions repair-routing --apply # actually repair, after confirmation
- Detection: scans gateway session rows with no key but real messages (branch/delegate/tool rows are unkeyed by design and excluded, so no false positives).
- Evidence-gated: it only acts when the evidence is unambiguous — either recorded lineage (
parent_session_idpointing at a keyed row of the same source), or exactly one keyed predecessor that fell quiet within 15 minutes of the orphan’s start. Two candidate predecessors, or two orphans claiming one predecessor, are refused with a reason — mis-adopting would splice one person’s conversation into another person’s chat, so it refuses to guess. - The repair: stamps the orphan with the predecessor’s identity via COALESCE (never overwrites an existing value), records lineage, and retires the predecessor with
end_reason='superseded_by_repair'— deliberately not the normal reset reason, so the next restart can’t drift back.
Are you affected?
Check the profile before panicking:
- Gateway sessions only: Telegram, Discord, Slack, WhatsApp, and other messaging-platform sessions. Pure CLI users are structurally immune — the CLI has none of this session-routing machinery.
- Highest risk: long-lived sessions, anyone who restarts/updates/crashes the gateway, and multimodal-heavy conversations.
- Not a one-off: the same install shows 5 orphan-session incidents since June (42, 34, 5, 798, and 2 messages) — this was just the largest, and it happened to the lead developer.
- Typical symptom: after a gateway restart, Hermes talks about old topics, doesn’t remember recent days, and references things long finished.
Updating and recovering your data
Important: the fix is on main (merged 2026-08-09) but is not in any release yet — the latest release is still v0.20.0 (2026-08-03). Long-time users are advised to update, via one of two paths:
- Wait for the next release, or
- Run from
mainfor immediate coverage (see the install guide and the update command reference).
Already affected and want your conversation back:
- Preferred: update to a build containing the fix, then run
hermes sessions repair-routing(dry-run first, then--apply). - Manual procedure (from the official issue, when the tool isn’t available):
- Stop the gateway;
- Back up first:
cp state.db state.db.bak; - Copy the stale “original” row’s
session_key,chat_id,chat_type,origin_jsonetc. onto the orphan row (fill blanks only, never overwrite); - Set
ended_atandend_reason='superseded_by_repair'on the stale row; - Restart the gateway and let resolution find the real session.
Back up
state.dbbefore any manual step, and make sure the gateway is fully stopped. When in doubt, wait for the fix release and userepair-routing— it’s safer than hand-editing.
What this incident teaches us
The scariest part isn’t the bug itself — it’s the silence. The session fork happened quietly in the background; in-memory continuity kept up the illusion that everything was fine, until one restart exposed it. Two takeaways for anyone running a long-lived AI assistant: first, check session continuity after restarts and updates; second, Hermes data rarely disappears — a “lost” conversation is usually a routing problem, not a data problem.
To go deeper on session management, see the hermes sessions command reference; for the latest major release, check the v0.20.0 release notes.