Your bot answers all day in a busy chat. Then, somewhere past message 500, it stops. The process is alive, the port is listening, hermes gateway status says everything is fine — but every new message fails. You open the log and find one line that has nothing to do with chatting: RecursionError: maximum recursion depth exceeded, with the traceback ending in setup_logging. Restart the gateway and it works again. Same time tomorrow, same failure.

This is not a memory leak, not a flaky model, and not something you misconfigured. It is Python’s output stream: two proxy layers stacking on top of each other, until after a few hundred rounds the chain is deep enough to blow the interpreter’s recursion limit.

The symptom: not slow — failing

Worth pinning down before you go looking, because the symptoms point at the wrong subsystem:

What you see	What is actually happening
Gateway process alive, status says running	The process is fine; each agent build is what throws
RecursionError in setup_logging, over and over	An attribute lookup walks a very long proxy chain
Restart fixes it instantly	The chain was reset — and starts growing again
Longer uptime makes it worse	It scales with turns/agent builds, not with time or load

The key word is long-running. Restart your gateway daily, or only use Hermes from a terminal now and then, and you will probably never see this. Run it as a service with Telegram, Discord, Slack, cron jobs and background review turned on, and it is a slow fuse.

What is on that stream

Two designs, each reasonable on its own.

The first is _install_safe_stdio(). Every time an agent is built, it wraps sys.stdout and sys.stderr in a _SafeWriter so that a broken pipe or an encoding error while printing progress cannot take the whole agent down. Because it runs on every build, it has to recognise “already wrapped” — and the old check was narrow: if the stream is already a _SafeWriter, skip.

The second is the thread-routing proxy _ThreadRoutingStream, installed by background review when it needs to silence output for a while. When thread_scoped_silence() finishes, it installs a router again.

The two checks disagree:

• _install_safe_stdio() only knew _SafeWriter. Seeing _ThreadRoutingStream on top, it decided “not mine” and added another _SafeWriter.
• thread_scoped_silence() then no longer saw a router, and installed a fresh one over that wrapper.

So every “build an agent, silence a scope” cycle added two links to each stream:

_ThreadRoutingStream( _SafeWriter( _ThreadRoutingStream( _SafeWriter( ... ) ) ) )

Each link forwards unknown attributes to the next one with __getattr__. One hop is nothing. But with hundreds of links, a single attribute probe has to traverse the whole chain — and after roughly 500 cycles, the line_buffering probe inside setup_logging hits the recursion limit and raises RecursionError. From then on every turn fails until the gateway restarts and the chain is dropped.

One detail that makes this easy to underestimate: the trigger is not “many messages”. It is agent builds. A turn that spawns subagents, runs background review and goes through several tool iterations builds far more agents than you would guess from the message count.

The fix: one isinstance check

The merged change is tiny and lives in agent/process_bootstrap.py:

def _install_safe_stdio() -> None:
    """Wrap stdout/stderr so best-effort console output cannot crash the agent."""
    for stream_name in ("stdout", "stderr"):
        stream = getattr(sys, stream_name, None)
        # The routing proxy already absorbs these errors; wrapping it stacks a new proxy per build.
        if stream is not None and not isinstance(stream, (_SafeWriter, _ThreadRoutingStream)):
            setattr(sys, stream_name, _SafeWriter(stream))

The guard now recognises the whole family instead of one member. The router already absorbs the two errors _SafeWriter exists for (OSError and ValueError, handled in its own forwarding path and in isatty()), so it does not need a second protective shell around it.

There is no cleanup pass and no coordination protocol between the two installers: check the type once, and the chain stays flat forever. It applies to every gateway surface — Telegram, Discord, Slack, Desktop, the API server — because they all go through the same agent bootstrap.

What to do now

1. Confirm it is this bug. Don’t guess, count:

hermes logs errors --since 7d | grep -c RecursionError
hermes logs errors --since 2d --level ERROR

A count that grows over days, resets to zero on restart and climbs again is the signature. Then check how many gateways you are running:

hermes gateway status
hermes gateway list

2. Get service back. Restarting interrupts in-flight turns, but it is the only immediate recovery:

hermes gateway restart
# several profiles side by side
hermes gateway restart --all

3. Know when the fix lands. The change merged to main on October 19, 2026 — after v0.22.2 shipped on October 15 — so the current release does not contain it. Your next hermes update to a build that includes the commit picks it up; restart the gateway afterwards. If you run from a source checkout tracking main, pull and restart.

4. Treat a gateway as a long-lived service. This bug is cumulative, not load-driven: it appears in processes that run for weeks and build tens of thousands of agents. Give the service a supervision policy that restarts it on failure (systemd Restart=on-failure, launchd KeepAlive), so a class of bug like this cannot leave you silent overnight. Hermes has its own watchdog and heartbeat machinery worth pairing with that — see the gateway loop watchdog guide and silent-disconnect heartbeat detection.

Why this one is worth remembering

Two lessons, neither specific to Hermes.

Wrappers on a hot path must be idempotent across the whole family, not one member. Any “wrap it if it isn’t wrapped” logic with a narrow type check silently becomes “wrap it on every call”. Unit tests will not catch it, because a test builds one agent; only a long-running process exposes it, and long-running processes are the hardest thing to reproduce.

Error messages lie about their layer. RecursionError reads like runaway recursion in business logic, but it came from an attribute-forwarding chain in the I/O layer. When the top of the traceback has nothing to do with what the code is doing, suspect accumulated infrastructure state before you start reading the business logic.

If you are debugging gateway behaviour, check your commands against the hermes gateway reference first, and use cron doctor on the scheduled side — cron jobs are usually the first thing to complain when a gateway goes unhealthy.

Summary

• Symptom: a long-running gateway fails every turn after a few hundred builds, RecursionError from setup_logging, restart clears it.
• Cause: _install_safe_stdio() and thread_scoped_silence() did not recognise each other’s proxy, so each cycle added two links per stream; ~500 cycles later an attribute probe exceeds the recursion limit.
• Fix: one isinstance check covering the whole proxy family. Merged to main, after v0.22.2.
• Today: confirm with hermes logs errors, recover with hermes gateway restart, wait for the next update — and supervise the service.