You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 381ce3b
Browse filesBrowse the repository at this point in the historyBrowse files
authored
feat(workflows): compose workflows with a unified execution tree (#4764)
* feat(workflows): compose workflows with a unified execution tree
Keep invocation results and workflow bindings on the same execution occurrence. Isolate fan-out contexts and resume persisted expansions through one executor.
Assisted-by: OpenCode (model: gpt-6-astra, autonomous)
* fix(workflows): complete composition cleanup phase one
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
* refactor(workflows): centralize execution tree access
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
* fix(workflows): narrow composition call failures
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
* fix(workflows): replay execution projections
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
* fix(workflows): qualify execution events
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
* fix(workflows): share execution snapshots
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
* docs(workflows): document composition resume
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
* fix(workflows): preserve replay projections
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
* fix(workflows): reject inconsistent execution trees
Assisted-by: OpenCode (model: deepseek-v4.1-flash, autonomous)
* fix(workflows): narrow workflow target resolution failures
Assisted-by: OpenCode (model: deepseek-v4.1-flash, autonomous)
* refactor(workflows): replay fan-out through the live path
Assisted-by: OpenCode (model: deepseek-v4.1-flash, autonomous)
* fix(workflows): match main events for unknown step types
Assisted-by: OpenCode (model: deepseek-v4.1-flash, autonomous)
* fix(workflows): omit unprojected fan-out item results
Preserve the item traversal's projection decision for missing step types. Cover sequential and parallel execution, inherited aliases, unnamed templates, and resume after reinstalling the implementation.
Assisted-by: OpenCode (model: gpt-6-astra, autonomous)
* fix(workflows): preserve composed fan-out aliases
Keep fan-out item aliases in the enclosing workflow context without exposing child workflow internals in root results. Normalize unnamed templates to the item ID for result aliases and lifecycle events, and recognize generated item aliases during fan-in validation.
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
* Refactor condition for raising ValueError on execution
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(workflows): validate composition checkpoint boundaries
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
* fix(workflows): make fan-out item aliases reporting-only
Fan-out item aliases (fan:template:N) are not valid fan-in wait_for targets.
In a loop iteration >= 1 the authored alias is never refreshed, so a fan-in
that waited on it silently joined the previous iteration's values. Joins now
use the fan-out step ID and its ordered output.results instead.
Revert the fan_out_aliases validation allowance so wait_for again requires
declared step IDs (matching main), and reject ':' wait_for entries at runtime
in FanInStep.execute() so unvalidated runs fail loudly rather than joining a
stale or empty item result.
Assisted-by: OpenCode (model: deepseek-v4.1-flash, autonomous)
* fix(workflows): stop logging after checkpoint failure; treat unknown child steps as call failures
RunState.append_log() now checks _checkpoint_failed under the state lock
(lock order _lock -> _log_lock), so no concurrent fan-out worker can append
a completion or nested start event after another worker's checkpoint failed.
This restores the documented guarantee that a checkpoint failure prevents
further writes.
Remove the error-text classification of unavailable child implementations at
the workflow-call boundary. A missing implementation stays terminal where the
step occurs (root and fan-out, as on main); at a workflow call it is a reported
child failure, whether detected at bind time or after binding, so call-level
continue_on_error treats both consistently and step-controlled error strings
are no longer interpreted.
Assisted-by: GitHub Copilot (model: claude-opus-5.5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test(workflows): cover resume snapshot check with native YAML scalars
Add a resume regression for a workflow snapshot containing `.nan`
(max_concurrency sequential fallback) and reordered keys, and cover the
previously untested rejection of a changed workflow snapshot. Both pass on
the existing comparison: PyYAML decodes `.nan` to a shared object, so the
snapshots compare equal without a production change.
Assisted-by: GitHub Copilot (model: claude-opus-5.5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Refactor executor run with error handling
Refactor execution handling to ensure active step is updated even if an error occurs during execution.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(workflows): report qualified current_step_id; keep typed gate messages; halt fan-out on worker exceptions
current_step_id again uses the occurrence ID that events and step_results
use, as on main: a paused or failed root fan-out item reports `fan:item:0`
and a later loop iteration `loop:body:1`. Inside a workflow call, the ID is
relative to the called workflow. walk_execution() now derives this ID with
the executor's qualification rules, so the post-run reconciliation (kept for
exception paths) and the gate payload's step_id report the same value.
Gate `message` keeps its typed template result (for example 42 stays an
int). Only values the JSON checkpoint cannot persist, such as native YAML
dates, are stored as text; presentation still stringifies.
A fan-out worker exception now sets the halt flag before re-raising, so
earlier items finishing in order no longer dispatch further items after a
later item has already raised.
Assisted-by: GitHub Copilot CLI (model: claude-opus-5.5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): persist the active step before it executes
The executor updated current_step_id and current_step_index only in memory
and wrote them with the next checkpoint, after the step had finished. While a
long step ran, and after a hard crash, state.json still named the previous
step, so `workflow status` and the run payload reported the wrong occurrence.
main saves before each step starts.
Checkpoint the active occurrence before it executes, so root steps, later
loop iterations, and fan-out items are persisted under their qualified IDs
while they run. This adds one state write per started step, as on main; the
checkpoint-count tests are updated accordingly.
Assisted-by: OpenCode (model: claude-opus-5.5, autonomous)
* fix(workflows): persist workflow calls before announcing them
1b0a3a4 checkpointed the active occurrence before step_started, but only
on the shared step path. Workflow calls return to Execution.workflow()
before that path and emitted step_started without a checkpoint. Until
binding was saved, and on resume of an already-bound call until the first
child step, state.json named the previous step, and a crash or checkpoint
failure could leave a start event for the call that no checkpoint
reflected.
Move the start sequence (set current_step_id, checkpoint, then log and
callback) into one Execution.start() helper used by both paths, so the
rule lives in one place. The ID update and checkpoint now happen under one
lock hold. Tests cover top-level, nested, and resumed bound calls, and a
failed start checkpoint that must leave no start event; all three failed
before this change.
Assisted-by: OpenCode (model: claude-opus-5.5, autonomous)
* fix(workflows): let interrupts during checkpoints pause the run
RunState.save() caught BaseException around the checkpoint write, so a
KeyboardInterrupt during any checkpoint became CheckpointError, marked the
state as failed to checkpoint, and bypassed the KeyboardInterrupt handler
in execute()/resume(). The run stayed "running" and could not be resumed.
On main the interrupt reaches that handler and the run pauses. Introduced
in 7aa424d.
Convert only Exception to CheckpointError. Write failures still stop
further writes; interrupts propagate unchanged, and the pause re-saves the
tree from memory. Tests interrupt a checkpoint at the root, inside a
workflow call, and during resume; all failed before this change.
Assisted-by: OpenCode (model: claude-opus-5.5, autonomous)
* fix(workflows): replay the result of aborted fan-out items
6792bea made fan-out items publish only the result their traversal
projected, so missing step implementations publish nothing. The replay of
an already-aborted occurrence returned without projecting its stored
result, so after a sibling item resumed, the aborted item appeared as {}
in the fan-out's ordered results.
Project the stored result before returning "aborted", as the live run did
when the occurrence aborted. Nothing is re-executed and no events are
emitted. The aborted-sibling regression now checks the item output for a
step, an if container, and a workflow call template; all three failed
before this change.
Assisted-by: OpenCode (model: claude-opus-5.5, autonomous)
* fix(workflows): hide partial fan-out results from resumed items
A halted fan-out checkpoints its partial results in its record for
reporting. On resume that record is projected before the items restart, so
re-executed and not-yet-started items saw steps.<fan>.output.results with
a stale prefix, including the paused item's own earlier output. Items in a
live run, and on main, never see results: the engine adds them only after
the items finish. Present since 7aa424d.
Remove the engine-added results from the fan-out's own entry in the
item-local snapshot. The parent context and checkpoint keep them.
Add an invariant test: a resumed item must see exactly the context it has
in a run that never paused, for step, if-container, and workflow-call
templates, sequentially and in parallel. It found only this difference;
the four affected cases failed before this change.
Assisted-by: OpenCode (model: claude-opus-5.5, autonomous)
* fix(workflows): validate occurrence lifecycle and report interrupted scopes
Persist fan-out templates as YAML, reject invalid resume offsets and tree states, and check only changed nodes during lifecycle transitions. Preserve original step exceptions during best-effort cleanup and derive interrupted bound-call status from the run. Cover regressions and document the contract.
Assisted-by: OpenCode (model: gpt-6-sol, autonomous)
* test(workflows): exercise legacy migration checkpoint failure
Assisted-by: GitHub Copilot (model: GPT-5.6 Luna, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): discard a resumed call's previous attempt status
A workflow call that paused kept its recorded `status: paused` result
while blocked. When resume re-entered the call and a later child step
raised, the call stayed blocked with that stale result, so status
reported the scope as paused even though the run had failed.
Re-entering a blocked, bound call now returns it to its children phase
and drops the previous attempt's result, outcome, and error. Its
children hold the authoritative progress; an exception leaves no
recorded call status, so the scope reflects the run status. Leaf steps
keep their result because finish replaces it on re-execution.
Assisted-by: GitHub Copilot (model: Claude Opus 5.5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs(workflows): describe current_step_id as a compatibility view
The public reference and the architecture document promised that
`current_step_id` always identifies the active execution-tree leaf. It
does not: it is updated as occurrences are entered, but completion does
not reconcile it, so a parallel fan-out can leave it naming an item that
has already finished while another item remains active.
Apply the reviewer's suggested wording: document `current_step_id` as a
compatibility occurrence-ID view rather than an exact active-step set, and
identify the tree's per-occurrence `active` flags as authoritative.
Assisted-by: OpenCode (model: deepseek-v4.1-flash, autonomous)
* fix(workflows): discard stale status of every re-entered blocked call
A workflow call whose binding failed (for example, an uninstalled
target) is blocked with a failed result but no binding. The re-entry
cleanup from 53a434d only ran for bound calls, so a retry that then
bound successfully carried the old result into the children phase. If
the child was interrupted, status reported the scope as failed while
the run was paused; after an exception, the call kept the stale binding
error.
A call's recorded result summarizes its previous attempt, so begin now
drops result, outcome, and error for every blocked workflow call. A
bound call continues at its children; an unbound call returns to ready
and binds again. Leaf steps keep their record until finish replaces it,
because it describes that same step: an interrupted resumed gate still
reports its prompt in status, as on main.
The unbound-call regression fails before this change (scope reported
failed instead of paused; stale error retained). The gate test guards
against extending the reset to leaf steps.
Assisted-by: OpenCode (model: claude-opus-5.5, autonomous)
* docs(workflows): state interrupt handling of the setup checkpoint
The RunState.save() comment said a KeyboardInterrupt always reaches the
pause path of execute()/resume(). That holds during step execution, but
the setup save before execution is outside that handler: an interrupt
after it replaces state.json leaves the run `running`. Describe that
accurately.
Also drop comparisons with the main branch from code comments, a test
comment, and the architecture document; once merged they no longer
describe anything. No behavior change.
Assisted-by: OpenCode (model: claude-opus-5.5, autonomous)
* fix(workflows): stop gate interrupt test leaking into later tests
test_interrupted_resumed_gate_keeps_its_prompt_in_status patched execute
on the shared STEP_REGISTRY["gate"] instance. Undoing that patch leaves the
original bound method as an instance attribute, which shadows later
class-level patches of GateStep.execute. In the full suite this made
test_resume_preserves_original_workflow_dir stay paused after resume.
Patch the GateStep class instead so teardown restores it cleanly.
Assisted-by: OpenCode (model: claude-opus-5.5, autonomous)
* test(workflows): pin and document the fan-out step_template removal
The rewrite stopped publishing a fan-out's raw `step_template` in its
step output, but nothing checked it end to end: the only `step_template`
assertions call FanOutStep.execute() directly, before the engine strips
the field. The public workflow reference did not mention it either.
Add an engine-level test that runs a fan-out, pauses, and resumes. It
checks that the template stays in the private execution tree and is
absent from the live, replayed, and persisted fan-out output, from a
fan-in aggregate, and from expressions. It fails when the engine
publishes the template again.
Add a migration note to docs/reference/workflows.md describing the
removed field and where to read the template instead.
Assisted-by: OpenCode (model: claude-opus-5.5, autonomous)
* docs(workflows): clarify rebind failure recovery
Document that retrying a previously bound workflow call discards its stale result, outcome, and error while preserving the binding and child subtree. Also describe the ready state used when an initial binding failed.\n\nAssisted-by: GitHub Copilot (model: GPT-5.6 Luna, autonomous)\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Markus <markus@example.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Markus Wondrak <markuswondrak@gmail.com>
|`-i` / `--input`| Updated input values as `key=value` (repeatable) |
76
76
|`--json`| Emit the resume outcome as a single JSON object |
77
77
78
-
Resumes a paused or failed workflow run from the exact step where it stopped. Useful after responding to a gate step or fixing an issue that caused a failure.
79
-
80
-
Supplied `--input` values are merged over the run's stored inputs and re-validated against the workflow's input types, then the blocked step is re-run with the updated values. This lets a run continue with information that only became available after it paused, or with a corrected value after a failure:
78
+
Resumes a paused or failed workflow run from its persisted execution state. A
79
+
`running` run is not resumable. Resume replays completed work into its
80
+
expression contexts without running step implementations again, then continues
81
+
at the unfinished occurrence. This lets a run continue with information that
82
+
only became available after it paused, or with a corrected value after a
83
+
failure.
84
+
85
+
Supplied `--input` values are merged over the run's stored root inputs and
86
+
re-validated against the workflow's input types. Unknown root input names are
87
+
ignored. Updated values affect only unfinished work; completed steps and calls
incomplete calls through their original mappings: newly mapped values override
745
+
the prior binding, while values not mapped again retain their bound values.
746
+
Completed calls retain their results. Failed output evaluation retries only
747
+
finalization, without repeating completed child commands.
748
+
749
+
Snapshots are stored as YAML strings inside the private JSON execution tree,
750
+
preserving YAML scalar types. Inputs and results remain JSON values. The
751
+
snapshot freezes workflow definitions and expansions, not files under
752
+
`context.workflow_dir`or step implementations; missing resources can still
753
+
cause ordinary step failures. Legacy runs without a tree enter through their
754
+
saved root index, then use tree-backed resume. Resume is available only for
755
+
runs in the `paused` or `failed` state.
756
+
757
+
`current_step_index`is the root-sequence index; `current_step_id` is a
758
+
compatibility occurrence-ID view updated as steps are entered. It is not an
759
+
exact active-step set: during parallel fan-out it may name an item that has
760
+
already completed. Qualified values include `fan:item:0` for a fan-out item
761
+
and `loop:body:1` for a later loop iteration.
762
+
Inside a workflow call, IDs are relative to the called workflow. Structured
763
+
run/status output includes `workflow_scopes`
764
+
summaries when calls exist and reports an active nested gate with its
765
+
`scope_path`. Private log events add `workflow_id` and `execution_path` to the
766
+
qualified `step_id`. A step emits completion after its checkpoint; containers
767
+
emit before their children, while replayed completed occurrences emit no step
768
+
events.
592
769
593
770
> **Security note:** a `shell` step runs a local command with **your** privileges. There is no capability sandbox — `requires` is an advisory pre-condition block (spec-kit version, integrations), not a runtime gate, so it does **not** restrict what a step can do. In particular there is no `requires.permissions` capability gate: it is rejected by validation precisely because it would imply a sandbox that does not exist. Review any catalog or downloaded workflow before running it, and use a `gate` step to require explicit approval before sensitive or destructive shell commands.
594
771
@@ -935,7 +1112,9 @@ Each workflow run persists its state at `.specify/workflows/runs/<run_id>/`:
935
1112
- `inputs.json`— resolved input values
936
1113
- `log.jsonl`— step-by-step execution log
937
1114
938
-
This enables `specify workflow resume` to continue from the exact step where a run was paused (e.g., at a gate) or failed.
1115
+
This enables `specify workflow resume` to replay completed occurrences and
1116
+
continue at the unfinished occurrence where a run paused (for example, at a
0 commit comments