The station markets a control that "a model unit the router flags as sensitive
(email / phone / SSN / card) may NEVER be sent to a cloud model" (Studio ->
Settings -> Live Execution; execution_policy.py:87 "PII: email/phone/SSN/card";
routing_forbids_cloud's docstring calls it "the real guarantee behind 'PII never
leaves local'"). For an actual Social Security NUMBER the guarantee does not
hold. Every SSN detector on the default (no-Presidio) path keys off the literal
word "SSN" or a phone/card shape that a 9-digit 3-2-4 SSN does not match, so a
real SSN in a prompt is classified non-sensitive and sent to the cloud model
unredacted.
Two independent controls both claim to cover SSNs, and both miss the number:
- ROUTING. execution_policy.routing_forbids_cloud(sensitive=...) blocks a cloud
route when the unit is "sensitive". That flag comes from
route_planner._looks_sensitive, which reuses cost_router._PRIVATE_PATTERNS:
email | \d{3}[-.\s]?\d{3}[-.\s]?\d{4} (a 3-3-4 PHONE shape)
| \bssn\b | \bpassport\b + a 13-19 digit "card-like" run
"123-45-6789" is 3-2-4 (not the 3-3-4 phone shape) and 9 digits (not a
13-19 digit card run), and carries no "ssn" word, so _looks_sensitive returns
False and sensitive_stays_local never fires.
- EGRESS. station_llm.guard_egress_messages — the guard Studio chat, the
Workflow Builder compose and MCP compose all call — classifies via
egress_classifier.probe_messages, whose SSN detector is
_SSN_WORD_RE = re.compile(r"\bssn\b", IGNORECASE): the word only. Its own
docstring says "ssn_reference: matches the literal word 'SSN' — not a real
[SSN detector]". When probe returns no counts the guard returns
("allow", messages, ...) with reason "no sensitive identifiers detected", and
the raw prompt is forwarded to the provider — even with an egress policy
configured that redacts on the ssn_reference category.
Presidio (real NER) would catch the number, but it is an opt-in dependency
(egress_classifier._presidio_enabled); on a default install both paths above are
the regex ones, so nothing catches an SSN number. The routing detector's own
comment even defers to "the real PII firewall (Presidio + egress policy) [that]
still runs at the station_llm egress boundary" — but that boundary uses the same
word-only egress_classifier, so the fallback has the identical blind spot.
The same blind spot covers other real PHI/PII an operator would expect a "PII
stays local" control to hold back: a bare account/SSN with no dashes
("123456789"), a name + date of birth + street address, an MRN — none match the
email/phone/card/word patterns, so all reach the cloud model.
The reach is every model-touching surface, not just chat. Workflow model nodes
and agent nodes both call station_llm.complete (the same egress guard) and are
routed by the same route_planner._looks_sensitive sensitivity flag
(workflow_engine._run_model_node raises only when route.policy_blocked, which
depends on that flag), so an SSN flowing from an upstream node into a model/agent
node reaches the cloud too.
Reproduction steps:
- Extract the station-v1.1.0 tarball to a clean directory; put workbench/ and
its parent on sys.path; import route_planner, primitives.egress_classifier,
and station_llm.
- Write an egress policy to <ws>/egress_policy.json that redacts the
ssn_reference and email categories, and point station_llm.WS at <ws> (so this
is enforce mode, not the no-policy dev carve-out).
- For the text "Please summarize this client note: 123-45-6789 is behind on
payments." call route_planner._looks_sensitive(text),
egress_classifier.probe_messages([{role:user, content:text}]), and
station_llm.guard_egress_messages([...], "openai").
- Repeat with "Client SSN 123-45-6789 — verify identity." (the word present)
and with an email as controls.
Expected: an SSN number is flagged sensitive and kept local / redacted, matching
the "email/phone/SSN/card never sent to a cloud model" control the operator
enabled.
Actual:
"123-45-6789" (no label) -> _looks_sensitive False; probe {}; guard "allow"
(raw SSN forwarded to the provider)
"123456789" (no label) -> _looks_sensitive False; probe {}; guard "allow"
"Client SSN 123-45-6789" -> _looks_sensitive True; probe {ssn_reference:1};
guard "redacted" (only because the WORD is there)
email control -> flagged + redacted, as expected
Root cause: the SSN class is detected by a keyword ("\bssn\b") rather than by the
value pattern, on both the routing and the egress paths, and the value-based
detector (Presidio) that would close the gap is opt-in and off by default — while
the operator-facing control names SSN as covered without that caveat.
Suggested fix: detect SSNs by value on the default path — add an SSN value regex
(e.g. \b\d{3}-\d{2}-\d{4}\b, plus a 9-digit variant gated to reduce false
positives) to egress_classifier and to cost_router._PRIVATE_PATTERNS, so both the
routing and egress controls fire on the number, not just the word. phi_guard
already scrubs SSN-shaped values by pattern for receipts; reuse that pattern set
at the egress boundary. If the intended posture is "regex is best-effort, enable
Presidio for real coverage", the Settings copy should say so instead of naming
SSN as a control the default configuration enforces.