Teaching CI to read Hermes: the stream-json event stream instead of screen-scraping


You hooked a Hermes task into CI: it reads the repository, runs the checks, writes a summary. The first run went well, so you shipped it. The second run stalled halfway through the turn and exited — and your pipeline showed green, because the wrapper only looked at exit code 0 and “stopped partway” is also 0. Then you went digging through the logs to work out which tools it called and how many tokens it burned, and found the output was written for a human to read, not for a program to parse.

The root cause is that one-shot output was always a terminal UI, never an interface. hermes chat now has a flag that turns it into an event stream.

One flag: output written for a program

hermes chat -q "Summarize this repository" --format stream-json

Every stdout line is one JSON object (JSONL), while diagnostics and the session_id: line stay on stderr — so stdout goes straight into a parser. The hard constraints, up front:

  • It requires -q / --query (or --query-file) — this format exists for one-shot runs.
  • It implies quiet mode: no banner, no spinner, no tool previews, because “pretty” is no longer the point.
  • It rejects --tui (asking explicitly for both exits 2).
  • Every event carries timestamp in Unix epoch milliseconds.

The design is inspired by Muse Code’s muse exec --json: turn one unattended agent run into a contract a tool can consume.

Five event types, one table

Event type Key fields
system subtype: "init", model, session_id
text text — a streamed assistant text delta
tool_use name, plus input when the arguments are available
tool_result name, output (capped at 5000 chars), duration_ms, is_error
result session_id, exit_code, text, tokens (input / output / total / cache_read / cache_write), duration_ms, and error when the turn failed

Once a conversation starts, its terminal record is always result — including when you interrupt with Ctrl-C, where it arrives carrying exit_code: 130. So treat that record as the completion signal rather than “the process ended”: the process exit code matches its exit_code.

The exit code is the completion signal

A one-shot run (-Q, chat --oneshot, or a query with non-TTY stdio) now reports the turn’s outcome in its exit code, identically on the quiet and non-quiet paths:

  • 0 — the turn completed;
  • 1 — it failed, stopped partway (partial), hit the iteration budget, or never ran at all (credentials / agent init failed);
  • 130 — it was interrupted;
  • 75 (EX_TEMPFAIL) — a Kanban dispatcher-spawned worker failed only because the provider was rate-limited, overloaded, returning 5xx, timing out, or the account hit a billing/quota wall, so the dispatcher requeues instead of counting a failure.

That last one deserves its own sentence: if you run unattended task queues with Hermes, handle 75 and 1 differently, or you will permanently mark “retry later” work as failed.

Consuming it in CI

The cheapest usage is to pull the final envelope with jq:

hermes chat -q "Run the repository health check and report failures" --format stream-json > run.jsonl
jq -r 'select(.type=="result") | [.exit_code, .tokens.total, .duration_ms] | @tsv' run.jsonl

To see which tools ran and how long each took:

jq -r 'select(.type=="tool_result") | [.name, .duration_ms, .is_error] | @tsv' run.jsonl

And to hand the assistant’s final answer downstream:

jq -r 'select(.type=="result") | .text' run.jsonl

In GitHub Actions, the standard shape is to wrap the step so failures are visible:

- name: Hermes repository check
  run: |
    hermes chat -q "Review this repo and write findings to REPORT.md" \
      --format stream-json > hermes-run.jsonl
    jq -r 'select(.type=="result") | .exit_code' hermes-run.jsonl | grep -qx 0

If your consumer is Python, read it line by line — JSONL means a mid-stream failure never costs you the events you already captured:

import json, subprocess

p = subprocess.run(
    ["hermes", "chat", "-q", "Summarize the diff", "--format", "stream-json"],
    capture_output=True, text=True,
)
final = None
for line in p.stdout.splitlines():
    ev = json.loads(line)
    if ev["type"] == "result":
        final = ev
print(final["exit_code"], final["tokens"]["total"])

The flags that make CI runs reproducible

Two isolation switches pair best with this format:

  • --ignore-user-config — ignore ~/.hermes/config.yaml and use built-in defaults (credentials in .env still load). Built for isolated CI runs, reproducible bug reports and third-party integrations.
  • --ignore-rules — skip the automatic injection of AGENTS.md, SOUL.md, .cursorrules, persistent memory and preloaded skills. Combined with the flag above, you get a fully isolated run.

Two more that shape the turn without touching the event format: --pass-session-id puts the session ID into the system prompt, and --checkpoints takes filesystem checkpoints before destructive file changes. --worktree runs the whole thing in an isolated git worktree, which is the right shape for “review this repo and open a PR”.

Four details you will trip over

  1. tool_result.output is capped at 5000 characters. For full tool output, read the session record (hermes sessions) or have the task write a file — do not expect the event stream to carry everything.
  2. text is a delta; result.text is the final answer. Accumulate text events to follow the stream, read result.text for “what this turn answered” — do not mix the two.
  3. Never parse stderr. The session_id: line and diagnostics belong to the human-readable channel; treating them as an interface breaks on the next patch release.
  4. It requires -q and refuses --tui. Both are hard validations: get them wrong and you exit 2 rather than receiving a half-working output format.

The short version

Wiring an unattended agent into a pipeline was never about “can it run” — it is about whether a machine can decide the run succeeded. --format stream-json closes that gap: the event stream makes the process observable, result.exit_code makes the outcome decidable, and tokens plus duration_ms make the cost accountable. To see which release shipped it and what else changed in the same window, read what v0.21.4 actually shipped; for the pre-flight on a CI machine, see hermes doctor and pinning skills into every session, which is how you make a CI agent carry the same playbook every time.