station-v1.2.0 adds Compliance Evidence Packs (POST /api/evidence/pack): a
signed zip whose report.html is presented as an auditor-ready HIPAA/SOC2 bundle
— its control map cites HIPAA §164.312(a)-(e) and SOC 2 CC6/CC7, and its
executive summary headlines "N governed actions, M Ed25519-signed, $X external
spend, Y% ran as local compute". Three of those headline numbers, computed by
primitives/evidence_pack.summarize, are either forgeable by an installed module
or arithmetically wrong, so the compliance document an auditor reads is not
trustworthy.
- OFF-MODEL % IS MODULE-FORGEABLE. summarize walks each receipt with
walk(r) collecting every dict that contains "off_model_fraction" and takes
blocks[0]:
def walk(v):
if isinstance(v, dict):
if "off_model_fraction" in v: blocks.append(v)
for x in v.values(): walk(x)
...
if blocks:
b = blocks[0]
u = int(b.get("units_total") or b.get("units") or 1)
o = int(b["units_off_model"]) if "units_off_model" in b else ...
s["units_total"] += u ; s["off_model_units"] += min(o, u)
A module command receipt seals the module handler's return value under
output, which is walked BEFORE the station's own routing block (receipt
key order is ... output, actor, routing ...). So blocks[0] is the module's
block, not the station's. One module receipt whose output carries
{"routing": {"units_total": 999999, "units_off_model": 999999,
"off_model_fraction": 1}} sets the pack's off_model_pct to whatever the module
chose. This is the same aggregator root as the previously reported off-model
finding, but it now feeds an auditor-facing, HIPAA/SOC2-framed compliance
artifact — an installed module (self-signed, publisher trust default "any")
can make the station's compliance evidence assert a false "% ran locally".
- SPEND IS DOUBLE-COUNTED. summarize sums spend with:
def spend(v):
if isinstance(v, dict):
sc = v.get("spend_cents") or v.get("spent_cents")
if isinstance(sc, int): s["spend_cents"] += sc
for x in v.values(): spend(x)
...
A dag-run receipt carries BOTH the workflow aggregate
(workflow_receipt.spent_cents) AND each node's spend_cents, so the walk adds
the total to its own components. A $5.00 run recorded as aggregate 500 + nodes
(300+200) is reported as $10.00 — the compliance pack's "Total external spend"
over-states by counting each run's money twice. The engine already knows the
right way one file over: workflow_engine.py accumulates a run's spend as
spent_cents += max(candidates) across the receipt-declared / output-surfaced
/ args-estimate figures precisely so a single spend is not counted more than
once; evidence_pack.summarize sums them instead.
- "SIGNED" COUNTS PRESENCE, NOT VALIDITY, AND INCLUDES NON-EXECUTIONS.
s["signed"] increments for any receipt that merely HAS a signature
field, and every receipt family is counted — including refusals. A
blocked_by_policy receipt (nothing executed) is counted among the "M
Ed25519-signed governed actions" the summary headlines, and a receipt whose
signature is present but does not verify would be counted as signed too. The
number an auditor reads as "M cryptographically-signed actions" is neither
"actions" (refusals included) nor "verified" (presence, not validity).
Reproduction steps:
- Extract the station-v1.2.0 tarball to a clean directory; put workbench/ and
its parent on sys.path; import primitives.evidence_pack.
- Write four receipts into <ws>/receipts/: (a) an honest executed command
receipt with a routing block; (b) a module command receipt whose output
contains {"routing": {"units_total": 999999, "units_off_model": 999999,
"off_model_fraction": 1}} plus the station's own {"routing": {units_total:1,
units_off_model:1}}; (c) a dag receipt with workflow_receipt.spent_cents=500
and node_receipts=[{spend_cents:300},{spend_cents:200}]; (d) a
blocked_by_policy receipt carrying a signature field.
- Call evidence_pack.collect(ws, start, end) then evidence_pack.summarize(items)
and read off_model_pct, spend_cents, and signed.
Expected: the compliance summary reports numbers derived only from the station's
own sealed verdicts — off-model from the station's routing block (not a module's
output), external spend once per run, and "signed" restricted to receipts whose
Ed25519 signature actually verifies.
Actual:
off_model_pct = 100 (units 1000001/1000001 — set by ONE module's output)
spend_cents = 1000 (real external spend was 500 — workflow + its nodes
double-counted)
signed = 4 (includes the blocked_by_policy refusal; counts
signature presence, never verification)
Root cause: the summary is built by structural DFS over untrusted receipt
content instead of reading each receipt's own station-sealed fields by their
known location, and it conflates a workflow's aggregate spend with its node
components and signature presence with signature validity.
Suggested fix: read the off-model verdict from the receipt's top-level routing
field only (never a DFS that a module's output precedes); count spend once per
receipt from the authoritative field (the workflow aggregate when present, else
the sum of nodes — not both); and count "signed" only for receipts whose
signature verifies against the pinned install key (reuse the same per-receipt
verify the pack already runs for its Verification section), and exclude
non-execution receipt families from a "governed actions" headline. More broadly,
a compliance artifact must derive its numbers from station-authenticated fields,
not from structural searches over content an installed module controls.