The new command-receipt routing block decides model-vs-local from a hardcoded
provider allowlist, and that allowlist does not contain any provider string this
station actually calls a model through. The first built-in command it is applied
to is a cloud LLM call, and it is sealed as local compute — the exact rounding-up
the function's own comment forbids.
approval_airlock.py (station-v1.0.0):
_MODEL_PROVIDERS = {"openai", "anthropic", "llm", "station_llm", "local_model"}
def _routing_block(cmd):
off = 0 if (cmd and str(cmd.get("provider") or "").lower()
in _MODEL_PROVIDERS) else 1
return {"executor": "cloud_llm" if off == 0 else "local_code",
"units_total": 1, "units_off_model": off,
"off_model_fraction": off}
The station's model transport is Groq: routes/llm.py posts to
https://api.groq.com/openai/v1/chat/completions with GROQ_MODEL defaulting to
llama-3.3-70b-versatile, and station_llm.guard_egress_messages is called with
the provider string "groq" throughout. "groq" is not in _MODEL_PROVIDERS, so
even a command that named its real provider honestly would still be classified
off-model.
The built-in command whose execution IS that call carries provider "railcall":
command_registry.COMMANDS
{"id": "workflow.build", "provider": "railcall", "mode": "read", ...}
routes/handlers_meta._h_build
spec = compose_engine.compose_spec(inputs.get("messages", []),
catalog(), _guarded_raw)
# _guarded_raw -> routes/llm.groq_raw(msgs) -> POST to the model endpoint
So every "Build a workflow from the conversation" — which sends the operator's
conversation to a hosted 70B model — seals a receipt reading
executor="local_code", units_off_model=1, off_model_fraction=1.
This is also the only sealed record that call produces. _h_build calls groq_raw
directly rather than station_llm.complete(), so no egress receipt is minted; the
one artifact on disk describing the call asserts it never touched a model.
Because essentials 0.2.0 aggregates these blocks into the quotable off-model
percentage, the misclassification lands directly in the number the program
publishes as measured rather than asserted.
Reproduction steps:
- Extract the station-v1.0.0 tarball to a clean directory and put
workbench/ on sys.path.
- Import approval_airlock and command_registry, then call
approval_airlock._routing_block(command_registry.CMD_BY_ID["workflow.build"]).
- Read routes/handlers_meta._h_build and routes/llm.groq_raw to confirm the
command's handler performs a hosted model call.
- Call approval_airlock.make_receipt with that command and inspect
receipt["routing"].
Expected: a command whose execution is a model call reports
executor="cloud_llm", units_off_model=0 — the honesty rule stated in the
comment directly above _MODEL_PROVIDERS.
Actual:
_MODEL_PROVIDERS = ['anthropic','llm','local_model','openai','station_llm']
'groq' in the set = False
workflow.build routing = {"executor": "local_code", "units_total": 1,
"units_off_model": 1, "off_model_fraction": 1}
provider='groq' = {"executor": "local_code", ..., "units_off_model": 1}
Root cause: the classifier keys off a free-text provider label that describes
the command's INTEGRATION, not its execution. "railcall" means "no third-party
integration required", which is true of workflow.build and says nothing about
whether the handler consults a model. The allowlist was then populated with
provider names that do not appear in the command registry at all, so it has no
true positives on the built-in surface and one known false negative.
Suggested fix: do not derive this from a label. Have the model transport report
it: set a per-call flag when routes/llm._model_post / station_llm.complete
actually fire during a command's execution, and have _routing_block read that
flag rather than cmd["provider"]. If a label-based classifier is kept as a
fallback, it must fail toward "model" for anything unrecognised rather than
toward "local", and it must at minimum include "groq" plus the command ids whose
handlers call the model directly (workflow.build).