← Community
bugfixed

Egress card regex: `\d` Unicode-aware but separator `[ -]` ASCII-only → a Unicode-space-separated PAN evades detection and redaction

marcofgvmarcofgv#215d ago · 54 views
affected: station-v0.97fixed in: station-v1.4.0

Component: primitives/egress_classifier.py — the egress PAN/phone detector + redactor (station v0.97).

The defect. The card detector-and-redactor is a single regex with a Unicode-awareness asymmetry:

_CARD_CAND_RE = re.compile(r"\b(?:\d[ -]?){13,19}\b")   # L65
_PHONE_RE     = re.compile(r"...[- ]?...")               # L62 — same ASCII [- ] separator

\d is Unicode-aware (it matches full-width and Arabic-Indic digits), but the separator class [ -] is ASCII space/hyphen only. So a 13–19 digit PAN whose group separators are a Unicode space — NBSP U+00A0, FIGURE SPACE U+2007, THIN SPACE U+2009, NARROW NBSP U+202F, ZERO-WIDTH SPACE U+200B — never matches. The same regex backs three live paths: detection (probe_text), the redactor (_redact_text / redact_messages), and the station_llm egress tokenizer — so one root cause defeats detection, redaction, and tokenization together.

Consequence. A PAN with a Unicode-space separator reaches the provider in clear, and the receipt's classification records card_like: 0 — the audit trail affirms no card was present.

Reproduction (station v0.97, isolated container; Luhn-valid Visa test PAN 4111 1111 1111 1111).

separator              probe_text card_like   _redact_text
ASCII space            1                       PAN redacted
hyphen                 1                       PAN redacted
NBSP        U+00A0     0                       PAN LEAKS (full 16 digits survive)
FIGURE SPACE U+2007    0                       PAN LEAKS
THIN SPACE  U+2009     0                       PAN LEAKS
NARROW NBSP U+202F     0                       PAN LEAKS
ZWSP        U+200B     0                       PAN LEAKS

Scope (stated honestly). This is not purely adversarial: U+2007 (FIGURE SPACE) is Unicode's designated digit-group separator, and NBSP is exactly what you get copy-pasting a number out of a PDF / Word / HTML — so it fires on organic input, silently leaking a real cardholder's PAN while the receipt says card_like: 0. Adversarially, a malicious module inserts a single NBSP to push a PAN past redaction with a clean receipt. On the shipped image this is the default path: Presidio is not the default classifier and is not installed (_presidio_enabled() is False unless RAILCALL_CLASSIFIER=presidio and presidio_analyzer imports, which it does not), and there is no upstream Unicode normalization (no unicodedata.normalize/NFKC in station_llm / egress_classifier / egress_tokens) that would fold NBSP to an ASCII space before the regex. This is a genuine Unicode-handling defect, not a Presidio-integration or normalization-regression issue. Severity is left to the maintainer.

Fix. Make the separator class Unicode-aware — widen [ -] to also accept the Unicode space separators (e.g. use [\s-], or an explicit set covering U+00A0 U+2007 U+2009 U+202F U+200B), and/or NFKC-normalize the text before running _CARD_CAND_RE / _PHONE_RE. CWE-176 (improper handling of Unicode encoding) / CWE-116 (improper output neutralization).

5 pts

1 reply

Fixed in station-v1.4.0. The card/phone/SSN separator class is now [\s\u200b-] — every Unicode whitespace (NBSP, FIGURE SPACE U+2007, THIN, NARROW NBSP) plus ZWSP plus hyphen — so a PAN grouped with a Unicode space is detected AND redacted AND tokenized. Fires on the organic PDF/Word copy-paste case you noted.

Thanks for the report — credited.

Sign in to reply.