Hermes Gateway Runtime Readiness Checks Replace Binary Liveness

Before PR #62650, the Hermes gateway had one health check: GET /health returned {"status": "ok"} if the process was alive. That was it. A server could be running with a corrupt state database, a full disk, or a misconfigured model - and the liveness probe would still report ok.

PR #62645 by teknium1 (merged July 11) adds authenticated runtime readiness checks that report real subsystem state without mutating anything. The public /health endpoint stays unchanged - cheap, no authentication, binary answer. A new /health/detailed endpoint requires authentication and returns the state of eight subsystems.

What gets checked

The new readiness module lives in gateway/readiness.py (117 lines) with a matching test suite in tests/gateway/test_readiness.py (103 lines). The API server integration is 36 lines in gateway/platforms/api_server.py. Total: 316 lines added across 6 files, 5 lines removed.

The collect_runtime_readiness() function probes eight subsystems. Each returns a status (ok or degraded) with structured detail. Here is the breakdown:

The background queue metrics (active_api_runs, process_completions, active_delegations) are informational - they return counts without a degraded/ok judgment. The other five checks produce a binary status.

SQLite probe: read-only by design

The state database check uses a read-only connection to avoid competing with normal writers:

uri = f"file:{path.as_posix()}?mode=ro"
with sqlite3.connect(uri, uri=True, timeout=1.0) as conn:
    conn.execute("PRAGMA query_only = ON")
    conn.execute("SELECT name FROM sqlite_master LIMIT 1").fetchone()

This matters for production deployments. A health endpoint polled every few seconds must never take a write lock. The mode=ro URI parameter and PRAGMA query_only are defense-in-depth - either alone would block writes, but both together guard against a misconfigured connection string. The timeout=1.0 prevents the probe from blocking indefinitely if something else holds a lock.

If the database file does not exist yet (a fresh install), the probe returns ok with detail "not initialized" rather than degraded. A missing file is expected state for a new deployment; only an unreadable or corrupt file that does exist triggers degraded status.

What the endpoint does not expose

The readiness contract explicitly limits what gets returned. The collect_runtime_readiness() docstring states:

probes expose status and counts only: never config values,
credentials, paths, commands, queue payloads, or exception messages.

The config check confirms the YAML parses - it does not return any key values. The model check returns ok or degraded - it does not return the model name. Exception messages are replaced with type names (type(exc).__name__). Platform credential states are never exposed.

This is in contrast to health endpoints that return full stack traces or environment details. The design treats /health/detailed as an authenticated but still external-facing surface.

Gateway platform connectivity

The gateway check inspects the runtime status object's platform list and reports how many platforms are connected:

connected = sum(
    1 for value in platforms.values()
    if isinstance(value, dict)
    and str(value.get("state") or value.get("status") or "").lower()
    in {"connected", "running", "ok"}
)

A platform in any other state (disconnected, error, reconnecting) does not count as connected. The response includes connected_platforms and total platforms so operators can see 3/5 connected without SSHing in.

Active API run retention correctness

The active run count uses lifecycle states rather than a simple counter. Retained completed streams - runs that finished but whose output has not been collected - are excluded from the active count. This prevents a long-running Hermes instance from reporting hundreds of "active" runs when most are retained but idle.

Test coverage

The PR's validation table reports 201 targeted gateway tests passing, plus live HTTP end-to-end verification:

Why this matters for VPS deployments

Hermes Agent is often run on a VPS with no SSH access during routine operation. Before this PR, the only operational signal was process liveness - a binary yes/no. A full disk, a corrupt state database, or a gateway stuck in a degraded state were all invisible to the operator.

The readiness endpoint provides enough information to decide whether a restart resolved an issue without opening a terminal. It does not replace monitoring - 90% disk usage is a coarse threshold, and queue depth counts without rate-of-change data tell you very little. But it replaces the worst case: a server reporting ok while silently failing to do work.

The module is self-contained (117 lines of Python with no framework dependency) and the read-only SQLite pattern is reusable for any health check that touches the state database.

[^1]: teknium1. "feat(gateway): add truthful runtime readiness checks (PR #62645)." NousResearch/hermes-agent. July 11, 2026.