Local models that think but never speak: where those six identical requests came from

You are running a local model on a laptop. The hardware is modest but workable. On some task it thinks for a long while, the status line flickers through its reasoning state, and then — nothing. No answer, no error, just a notice that the model produced no reply this time, even after retries. You open the log and see six requests of identical size, as if something got stuck in place.
The model is not stupid and your machine is not too slow. This is a contradiction inside the recovery path: Hermes kept trying to remind the model to speak, but that reminder was deleted by a sanitizer before it ever left the process. Six retries, six byte-identical requests, six identical decisions by the model.
The fix is on main, but not in v0.23.1 — it waits for the next release. Three things are worth understanding: why a thinking model produces reasoning with no content, why the reminder vanished, and what to expect now.
A reply with reasoning and no content
Start with the shape of the output. Many local reasoning models (the Qwen family, some Ollama models, anything with a reasoning channel in LM Studio) split a reply into two parts:
- the reasoning part —
reasoning_content, or a<think>block some implementations put straight into the content; - the content part — the answer a human reads.
Normally both exist. Sometimes the model spends its whole budget reasoning and leaves content empty: content: "". On screen that looks like “it thought about it and said nothing.”
Hermes has a recovery ladder for exactly this, in order:
- if the previous turn had usable content, fall back to it;
- after a tool call, nudge the model once to continue;
- when there is reasoning but no content, send a “write the answer out” reminder — up to twice;
- otherwise, enter the normal empty-response retries.
The bug lived in step 3.
Why the reminder vanished
The old approach appended an assistant prefill row saying, in effect, “you reasoned only,” so the model would look mid-thought and continue.
Reasonable on its own. But the same codebase has another rule: the send-time sanitizer drops assistant turns that contain reasoning but no visible content, precisely so those empty turns do not waste context.
The two rules collided. The prefill row was dropped before the request went out, so the outgoing request was byte-identical to the one that had just failed: same messages, same length, same hash. Facing the same input, the model did the same thing — reasoned and said nothing. Six times, then give up.
The measurement in the report is blunt:
| Before | After | |
|---|---|---|
| Main-model requests | 6 | 2 |
| Distinct request bodies (sha of messages) | 1, e39cd0d3bcfd × 6 |
2 |
| Final reply | “no reply this time, even after retries” | a real answer |
| Rows written to history | synthetic empty rows added | history unchanged |
The report came from a user on Discord: local LM Studio with a Qwen variant, heartbeats plainly showing “Thinking-only response — prefilling to continue (1/2)(2/2)”, then three “Empty response” retries and the no-reply notice, with six requests of identical input size.
The fix: the hint lives on the outgoing copy only
The approach is clean, and one sentence covers it: recovery hints exist only on the outbound copy, never in history.
Concretely:
- the reminder (“you only reasoned — answer now”) and the post-tool nudge are appended to the outgoing copy of the last user message or tool result;
- stored history, the earlier cache prefix and the SQLite rows stay untouched, so a retry no longer blows up the prompt cache;
- the hint is cleared when a response arrives, at finalization, and at the next turn;
- the post-tool nudge no longer writes synthetic empty assistant/user rows into live history;
- if a tool call actually landed, the thinking-retry budget resets without depending on a prefill row.
One more change makes diagnosis easier. The retry warning used to say “no content or reasoning” even when the model had clearly reasoned — which sends people hunting in LM Studio settings. It now says reasoning but no visible content.
The code lives in agent/turn_empty_response.py, which is exactly the ladder described above: check for prior content, check whether a tool ran, check for structured reasoning fields — including the Ollama case where <think> sits inside the content — and only then fall through to the normal retries.
What this changes for you
On a local model, the direct win is time and heat: six identical requests become two, and the second one usually produces the answer. On a metered provider this is real money — nobody gives you a discount for resending the same request.
The second win is clean history. The old retries pushed synthetic empty rows into the conversation; in long sessions those accumulate, and they interfere with compaction and cache hits. Now history only records what actually happened.
The third win is a shorter debugging path. The log no longer says something both true and useless. reasoning but no visible content tells you the model is thinking and not writing; you adjust output budget or prompting instead of suspecting the toolchain.
Two related settings are worth knowing: give local models enough output budget (or lower the reasoning effort a notch) if this keeps happening, and if long sessions keep brushing against these edges, our earlier pieces on how the lean tail gets compressed and how the context window label is chosen cover the same family of context-management problems.
When it can still happen
To be honest about scope: the fix turns “always six” into “usually two,” but it does not change the underlying fact that a model can still reason without writing. What was fixed is how we ask it again, not why it stalls. The latter depends on the model, the quantization, the output budget and the prompt.
So when you see these logs:
- if one reminder is enough to recover, the fix is working — carry on;
- if it happens on the same prompt every time, change the prompt or the output budget;
- if there is no reasoning either (the log says no content or reasoning), that is a different problem: the model or the transport returned nothing at all.
The fix was salvaged from a community PR (original authorship kept) with a log-wording correction, and merged into main on November 10, after the v0.23.1 cut, so it is not in any release yet — track main to use it now. Full notes on the v0.23.1 release page, and the other half of this patch in the catalog cap write-up.