station-v0.82's new orchestration router keeps a result cache on disk
(WS/route_cache.json, 24h TTL) so a repeated extractive question -- classify,
extract, detect, summarize -- is answered without a second model call. The
cache lookup is the FIRST thing route_executor.execute() does, before it
decides local vs cloud and therefore before it ever calls the runner. The
runner is station_llm.complete(), which is where the entire egress-governance
chain lives: the module manifest destination check, the PII/PHI classifier,
the operator's egress policy engine, and the signed egress receipt.
So a cache hit returns the stored answer without any of that running. Two
consequences:
- It breaks station_llm.complete()'s own stated invariant. That function's
docstring says "Every call writes a receipt regardless of provider
response," and the module docstring says a receipt is written to
WS/receipts/egress/ and appended to the audit chain so a rebuilder can
reconstruct the sequence. On a cache hit, no receipt is written anywhere
and nothing is appended to the audit log -- the call leaves no trace at
all, even though the caller received real content back.
- An operator tightening their egress policy has no effect for up to 24
hours. Adding a deny rule (or a redact rule, or restricting a provider)
only affects calls that actually reach the policy engine. Any input
already in the cache keeps returning its pre-change answer until the TTL
expires, with the new rule never consulted.
The cache key makes this broad rather than narrow. cache_key() is a sha256
over exactly two things -- the task_type string and the concatenated message
text -- with no module id, no policy version, no provider, and no workflow
id. One entry is therefore shared by every caller on the station: a module,
a workflow "model" node, the /api/model/offload endpoint, and the
railcall_offload MCP tool all read and write the same store. Whichever caller
asks a given question first fixes the answer every other caller gets for the
next 24 hours, regardless of what governance would have said about the later
callers. (The same key narrowness means an expect_json=true caller can be
served a cached free-text answer that the JSON validation never ran against,
because expect_json is not part of the key either.)
Proven by an A/B on the same input with the same deny policy in force, where
the only variable is whether the cache is warm:
cold cache -> real station_llm.complete() runs -> the call is DENIED by the
operator's policy and a signed egress receipt is written
warm cache -> cache hit -> the answer is returned, complete() is never
called, and zero receipts are written
Reproduction steps:
- Extract a clean station-v0.82 tarball.
- Write WS/egress_policy.json with a single rule:
{"version": 1, "default": "allow",
"rules": [{"name": "credit cards", "if": {"has_category": "card_like"},
"then": "denied"}]}
(this is egress_policy.py's own documented example rule, unmodified).
- With a COLD cache, call route_executor.execute() with task_type="classify"
and a message containing card-like digits, leaving runner unset so the real
station_llm.complete() is used. Note the result and count the files in
WS/receipts/egress/.
- In a fresh workspace with the SAME policy, first seed the cache with one
successful answer for that identical input (an earlier allowed call, or any
other caller on the station), then repeat the step-3 call verbatim.
Expected: step 4 reaches the policy engine exactly as step 3 did and is
refused, or at minimum is re-validated against the current policy and
receipted either way.
Actual: step 3 returns denied:true with one signed egress receipt on disk.
Step 4 returns served_from:"cache" with the earlier answer, denied absent, and
zero receipts written -- the classifier, the policy engine and the manifest
check never ran.
Root cause: route_executor.py execute() performs the cache lookup as step 1
and returns early on a hit, while every governance control lives inside the
runner it would otherwise call (station_llm.complete). Governance is
downstream of the cache, so nothing upstream of the model call can be
enforced on a cached answer. Compounding it, cache_key() covers only
(task_type, message text), so entries are shared across callers and are not
invalidated by a policy change.
Suggested fix: bind the cache to the governance decision rather than sitting
in front of it. Concretely: include the caller's module id and the current
policy version hash (station_llm._policy_snapshot() already computes one) in
cache_key(), so a policy change invalidates affected entries and one caller's
entry cannot serve another's request; and emit a receipt on a cache hit --
recording it honestly as served-from-cache with a pointer to the provenance
run that produced it -- so the audit trail keeps the documented
one-receipt-per-call property.