feat(workflows): opt-in output_format: json exposes parsed shell stdout as output.data#2963
Open
doquanghuy wants to merge 1 commit into
Open
feat(workflows): opt-in output_format: json exposes parsed shell stdout as output.data#2963doquanghuy wants to merge 1 commit into
doquanghuy wants to merge 1 commit into
Conversation
…ut as output.data No step that runs external code could hand a typed value to a later step, so e.g. a fan-out could never consume a runtime-computed collection. With output_format: json declared, stdout is parsed and exposed under output.data (raw keys unchanged); a parse failure fails the step with a clear error. Without the key, behavior is unchanged. Reference implementation for the proposal in github#2962. Addresses github#2962
Contributor
Author
|
@mnriem when you have a moment, would appreciate your thoughts on the direction here — the issue lists the alternatives considered, and I'm happy to rework toward whichever shape fits Spec Kit best. |
Contributor
There was a problem hiding this comment.
⚠️ Not ready to approve
The newly added tests use shell echo quoting that is not portable to Windows cmd.exe, which will likely break the existing Windows CI matrix.
Pull request overview
Adds an opt-in structured output mode for shell workflow steps, enabling later steps to consume typed data derived from shell stdout while preserving the existing raw stdout/stderr/exit_code outputs for backward compatibility.
Changes:
- Add
output_format: jsonhandling toShellStep.execute()to parsestdoutand expose it asoutput.data, failing the step on JSON parse errors. - Extend
ShellStep.validate()to reject unknownoutput_formatvalues. - Add unit tests covering JSON parsing success/failure, backward compatibility (no
datakey without the flag), and validation behavior.
File summaries
| File | Description |
|---|---|
| src/specify_cli/workflows/steps/shell/init.py | Implements output_format: json parsing into output.data and validates allowed formats. |
| tests/test_workflows.py | Adds TestShellStep coverage for JSON structured output and validation. |
Copilot's findings
- Files reviewed: 2/2 changed files
- Comments generated: 3
Note
Your feedback helps us improve the quality of this feature.
Please use 👍 or 👎 to tell us whether this assessment is correct.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+948
to
+954
| step = ShellStep() | ||
| ctx = StepContext(project_root=str(tmp_path)) | ||
| config = { | ||
| "id": "emit", | ||
| "run": "echo '{\"items\": [1, 2]}'", | ||
| "output_format": "json", | ||
| } |
Comment on lines
+964
to
+970
| step = ShellStep() | ||
| ctx = StepContext(project_root=str(tmp_path)) | ||
| config = { | ||
| "id": "emit", | ||
| "run": "echo not-json", | ||
| "output_format": "json", | ||
| } |
Comment on lines
+979
to
+982
| step = ShellStep() | ||
| ctx = StepContext(project_root=str(tmp_path)) | ||
| config = {"id": "emit", "run": "echo '{\"items\": []}'"} | ||
| result = step.execute(config, ctx) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Reference implementation for #2962 — for discussion, direction welcome.
Adds an opt-in
output_format: jsonto shell steps: when declared, stdout is parsed and exposed underoutput.data(the rawstdout/stderr/exit_codekeys are unchanged, so there is no merge/clobber ambiguity), letting later steps consume typed values — e.g. a fan-out'sitems: "{{ steps.emit.output.data.items }}". A parse failure fails the step with a clear error (declaring the format is a contract; silence would hide wiring bugs). Without the key, behavior is byte-identical — fully backward-compatible.validate()rejects unknown formats.The issue lists
output_file:and a declared named-outputs:schema as alternatives — happy to rework toward either if you prefer that direction.Coordination: aware of #2443 touching the same file; this diff is isolated to the structured-output addition and I'm happy to rebase in whichever order suits.
Testing
uv sync && uv run pytest— full suite 3729 passedTestShellStep: data exposed on valid JSON; invalid stdout fails the step; no-flag backcompat (nodatakey); validate rejects unknown formats — the three behavior tests are red against currentmain, green with the change (verified both directions)uvx ruff check src/— cleanuv run specify --helpAI Disclosure
Code, tests, and this description were authored with AI assistance (Claude); verified by running the repo's test suite and ruff locally in both red and green directions.