← Community
bugfixed

Global freeze halts only effect nodes; a workflow http (or model) node makes its outbound request while the station is frozen (v0.99)

ShwetaShweta#119d ago · 38 views
affected: station-v0.99fixed in: station-v1.4.0

Reproduction steps:

  1. Freeze the station (the 2am kill-switch): set_freeze(True), i.e. the flag

WS/freeze.json {"frozen": true}. Freeze is implemented by policy_gate returning
{"decision":"block"} for every action while frozen.

  1. Run (live) a workflow containing an http node that POSTs to an allowed domain

(an author-configured outbound send, action_class external_send) and, for
contrast, an effect node.

  1. Run repro_global_freeze_skips_http_model_nodes_v099.py against a clean v0.99

extraction. It drives the REAL run_workflow with a policy_gate that hard-blocks
(exactly what studio_server.policy_gate returns when frozen) and records calls:
HTTP POST node : outbound POST executed=True ; policy_gate consulted=False
EFFECT node : policy_gate consulted=True ; policy_block -> BLOCKED

Expected:
Per the freeze design comment (studio_server.py:4151): "GLOBAL FREEZE — the 2am
kill-switch. One flag halts EVERY live-effect path ... freeze stops OUTBOUND
effect." A mutating http node (POST/PUT/DELETE — an outbound send that leaves the
station) and a model node (an outbound LLM call — data egress + spend) are
outbound effects and must be halted while frozen.

Actual:
Freeze is enforced at runtime by run_workflow's Tier-2 fail-closed policy backstop
(workflow_engine.py:910), which is guarded to a single node kind:

if policy_gate is not None and kind == "effect" and allow_live_effects:
_pdec = policy_gate(_pi.provider, _pi.verb, _pi.action_class, _pamt)
if _pdec.get("decision") == "block":
raise PolicyBlocked(...)

kind = n.get("type", "effect"). An http node (type=="http") and a model node
(type=="model") are DIFFERENT kinds, so this backstop — the only place the live
policy (and therefore freeze) is consulted at runtime — never runs for them. The
http node is executed directly (workflow_engine.py:1447 -> workflow_http.apply_http_plan)
with no policy_gate/freeze check; the workflow-run route (routes/dispatch_workflow.py)
has no upstream freeze gate either, and live_http/live_model are independent run
flags. The repro shows, with the station frozen, the effect node correctly BLOCKED
while the http POST node fires its outbound send and the gate is never asked. So the
kill-switch an operator hits during an incident does NOT stop a live workflow's
http/model nodes from sending data outbound. (Proven for the http node; the model
node shares the identical kind == "effect" exclusion.)

Suggested fix:
Do not gate the runtime freeze/live-policy backstop on kind == "effect". Either
(a) extend the backstop to every OUTBOUND kind — http with a mutating method
(action_class external_send) and model — consulting policy_gate/freeze before the
node runs, or (b) add a single authoritative short-circuit at the TOP of
run_workflow: if freeze_state()["frozen"] and this is a live (non-dry-run) run,
refuse the whole run and roll back, so no node kind can slip an outbound effect
past the kill-switch. A kill-switch that only covers one of several outbound node
kinds is not a kill-switch.

5 pts

1 reply

Fixed in station-v1.4.0. The runtime freeze/live-policy backstop is no longer gated to kind=='effect' — it now also consults policy_gate for http nodes (when live_http) and model nodes, raising PolicyBlocked on a block decision. Freeze halts every outbound path it claims to; unfrozen runs are unaffected.

Thanks for the report — credited.

Sign in to reply.