The measured off-model fraction is biased upward by construction: it counts
receipts where nothing executed as off-model units, and it cannot see the
receipt family that records the station's model calls. A station that refused
every write and executed nothing reports 100% over a growing corpus, and no
amount of real model traffic can lower it.
Counting non-executions. approval_airlock.make_receipt seals"routing": _routing_block(cmd) unconditionally — result_status is never
consulted, and units_total is hardcoded to 1. Every receipt gets one, including
the receipts that exist precisely because nothing ran:
preview_command "pending_approval" (routes/commands.py:149)
preview_command "failed_with_receipt" validation failed (66)
preview_command "blocked_by_policy" recipient allowlist (87)
execute_command "blocked_by_policy" GLOBAL FREEZE active (214)
execute_command "blocked_by_policy" no approval bound (228)
execute_command "blocked_by_policy" approval already used (247)
execute_command "blocked_by_policy" daily rate limit (281)
Each is sealed as one unit of off-model local execution with
execution_class in ("pending", "blocked", "failed"). A write command's ordinary
lifecycle mints at least two receipts (preview, then execute), so one governed
action contributes two units of "compute this machine did without a model".
Not counting model calls. The aggregator globs three roots:
receipts/runs/.json, receipts/capoff/.json, receipts/*.json
station_llm._persist_receipt writes the sealed record of every governed model
call to receipts/egress/ (station_llm.py:66, 135). That directory is not among
the roots, and egress receipts carry no routing block to find in any case.
Adding a sealed egress receipt to the workspace moves the reported fraction by
exactly nothing.
Mixed units. The two dialects being summed into one ratio measure different
things. route_planner.summarize documents its units as "model-eligible units
that ran LOCAL, i.e. what routing actually avoided ... It is NOT 'every step
priced as a model call'". approval_airlock defines one unit as one receipt,
model-eligible or not — a validation failure is not work that routing avoided.
The aggregator adds units_total/units from both into a single numerator and
denominator.
Reproduction steps:
- Extract the station-v1.0.0 tarball to a clean directory; put workbench/ on
sys.path and load modules/sami666-railcall-essentials/handlers/handler.py
with __rc_helpers__ = {"WS": <scratch workspace>}.
- Seal five receipts with approval_airlock.make_receipt for a real send command
using statuses "pending_approval", "blocked_by_policy" (x3) and
"failed_with_receipt", writing them to <ws>/receipts/. Nothing has executed.
- Call the module's _off_model(ws).
- Write a sealed egress receipt to <ws>/receipts/egress/ and call _off_model
again.
Expected: a corpus measuring what routing avoided contains no units for
receipts whose execution_class is pending/blocked/failed, and the receipt family
recording model calls is inside the measurement, not outside it.
Actual:
step 3 {"receipts_measured": 5, "units_total": 5, "units_off_model": 5, "fraction_pct": 100}
(five receipts, zero executions, five off-model units)
step 4 {"receipts_measured": 5, "units_total": 5, "units_off_model": 5, "fraction_pct": 100}
(unchanged — the model call is invisible to the metric)
Root cause: the routing verdict is emitted from the receipt builder, which runs
on every governed decision, rather than from the execution path, which runs only
when something executes; and the aggregator's root list enumerates the local
receipt directories without the egress one.
Suggested fix: emit the routing block only when the command actually executed
(execution_class == "executed", i.e. the two make_receipt calls that passoutput=), so refusals stay sealed governance decisions without becoming
measured compute. Add receipts/egress/*.json to the aggregator's roots and give
egress receipts a routing block reporting units_off_model=0, so the denominator
includes the station's model traffic. If the dag and command dialects are to be
summed, state one unit definition and make both sides emit it.