← Community
bugfixed

Team Job retry changes run_id and invalidates an already-approved quorum request

DaveDave#319d ago · 101 views
fixed in: station-v0.74

Station v0.71 does not preserve workflow run identity when a Team Job is retried after quorum approval.

This causes the retry to generate a new action_hash, so an approval that was already granted for the original execution is not recognized. Station creates a new approval request instead of resuming the approved run.

Confirmed with an end-to-end Team Job → HTTP DAG → Team gate reproduction using a harmless stub effect.

Initial run:

job_id = job_e43768fe25d9ee29dc32502d
run_id = wfrun_1e8c12c5db4fcf2991b97b66
request_id = env_4ac4615d89d10e0dddce67543d6d0b8a
action_hash = sha256:25f715f696fd20869a45145bff206de3eeec18758d36b40f354c6af8deb7ef4f
outcome = AWAITING_TEAM_APPROVAL

The request was then approved successfully. The persisted approval remained valid and stored against the original action_hash.

Retrying the same job through the Team Job path produced:

job_id = job_e43768fe25d9ee29dc32502d
run_id = wfrun_bdabbafbf7e65cdd8e868120
request_id = env_bdb6870846b92ea61fcc956c17263654
action_hash = sha256:7283de59e98642cfc94d710dbea8ce6afc3d1ca5774290ed2ed5586d1dac6ce9
outcome = AWAITING_TEAM_APPROVAL

The external effect was not executed and a new approval request was created.

Observed:

run_id changed = yes
action_hash changed = yes
old approval reused = no
new request created = yes
effect executed = no

Control test:

Calling run_workflow() again with the original run_id caused the existing approved request to be recognized immediately. The workflow returned COMPLETED and the effect executed.

This confirms the approval persistence/signature itself is working. The failure is caused by execution identity not being preserved across the Team Job retry.

Root cause:

routes/team.py::job_runner() sends the workflow ID/context to the HTTP DAG route without a stable run_id.

routes/dispatch_workflow.py then calls run_workflow() without a run_id.

workflow_engine.py consequently generates a new run ID:

run_id = run_id or ("wfrun_" + token)

The Team approval action_hash includes run_id. Approval lookup is performed against that exact action_hash, so the new run produces a different hash and cannot match the already-approved request.

Expected:

A Team Job that enters AWAITING_TEAM_APPROVAL should preserve a stable execution identity. After the approval is granted, resuming/retrying that same execution should produce the same action_hash, recognize the existing approval, and continue to the external effect without requesting approval again.

Suggested fix:

Generate/persist a stable run_id or occurrence_id for the Team Job and propagate it through:

Team Job
→ handle_job_offer()
→ job_runner()
→ HTTP DAG route
→ run_workflow()

The ID should also be returned in the job result/receipt so the approved execution can be correlated and resumed reliably.

A regression test should cover:

first run → pending
approve request
resume same job/run
same run_id
same action_hash
existing approval recognized
effect executes

Confirmed against Station v0.71 runtime and source. A stable-run-id control test reproduced the expected successful behavior.

3 pts

1 reply

Verified: a job retry re-ran with a fresh random run_id, and since a quorum approval binds to run_id + plan digest, the retry invalidated the approval and forced re-approval. The job now derives a STABLE run_id from the job_id and threads it through job_runner → dag/run → run_workflow, so a retried job reuses the run and an existing approval still applies. Confirmed against the code and fixed on the v0.74 batch (verified + regression-tested); ships in station-v0.74. Thanks Dave.

Sign in to reply.