You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Analysis Date: 2024-11-24 Focus Area: CLI Feedback Quality Strategy Type: Custom Custom Area: Yes - This focus area addresses the specific challenge of providing clear, actionable feedback to users (both human developers and AI agents) when working with agentic workflows. Given that gh-aw is a tool for creating AI-driven workflows, exceptional error messages and user guidance are critical for adoption and debugging.
Executive Summary
The gh-aw CLI demonstrates strong adoption of structured console formatting (87% formatted output) and maintains consistent messaging patterns across 754 formatted output locations. However, analysis reveals significant opportunities for improvement in error message helpfulness and command documentation. Only 7.1% of error messages (44 out of 613) provide actionable suggestions or examples, leaving 92.9% of errors as basic technical messages that may confuse users. Additionally, critical commands like compile, list, run, and status lack Short/Long descriptions and examples in their help text, creating gaps in user guidance.
The impact is particularly acute for this tool because users are learning to work with agentic workflows—a novel paradigm. When compilation fails or workflows behave unexpectedly, users need clear guidance on what went wrong and how to fix it. Improving CLI feedback quality will reduce support burden, accelerate user onboarding, and improve the overall developer experience.
Full Analysis Report
Focus Area: CLI Feedback Quality
Current State Assessment
The gh-aw CLI has established good foundations for user feedback through the pkg/console package, which provides formatting utilities for success, error, warning, info, and progress messages. The codebase shows strong discipline in using these utilities, with 87% of output properly formatted.
Metrics Collected:
Metric
Value
Status
Total Error Sites
613
ℹ️ Baseline
Helpful Errors (with suggestions/examples)
44 (7.1%)
❌ Low
Formatted Output Adoption
87.0%
✅ High
Commands with Help Text
5/10 (50%)
⚠️ Medium
Commands with Examples
2/10 (20%)
❌ Low
Progress Indicators
331
✅ Good
Success Messages
84
✅ Good
Warning Messages
208
✅ Good
Findings
Strengths
Strong Console Formatting Adoption: 87% of output uses structured console formatting (754 formatted vs 112 plain messages)
Consistent Progress Indicators: 331 progress messages keep users informed during long operations
Existing Helper Infrastructure: The console.FormatErrorWithSuggestions() function demonstrates awareness of good error message design
Good Success/Warning Coverage: 84 success messages and 208 warnings provide clear status feedback
Only 44 out of 613 errors provide suggestions or examples
569 errors are basic technical messages like "failed to create directory: %w"
Users receive minimal guidance on how to resolve issues
Incomplete Command Help Text:
compile command: No Short/Long description or examples
list, run, status, remove commands: Missing help text
Only add command has comprehensive help with 3 examples
Generic Error Messages:
Common pattern: fmt.Errorf("failed to X: %w", err) provides technical error but no context
Example: "failed to create .github/workflows directory: %w" doesn't explain why or how to fix
Missing examples of correct usage in error output
Inconsistent Example Provision:
Only 37 error messages include usage examples
Users must consult documentation separately when errors occur
No inline "did you mean?" suggestions for common mistakes
Detailed Analysis
Error Message Categories
Category 1: Basic Technical Errors (92.9%)
// Current pattern - minimal contextreturnfmt.Errorf("failed to create .github/workflows directory: %w", err)
returnfmt.Errorf("jq not found in PATH")
returnfmt.Errorf("workflow file '%s' already exists. Use --force to overwrite", destFile)
Category 2: Helpful Errors with Suggestions (7.1%)
// Good pattern - provides suggestionsreturnerrors.New(console.FormatErrorWithSuggestions(
fmt.Sprintf("workflow '%s' not found in local .github/workflows", fileOrWorkflowName),
suggestions,
))
Command Help Quality Analysis
Command
Has Short
Has Long
Has Example
Quality Score
add
✅ Yes
✅ Yes
✅ Yes (3)
🟢 Excellent
compile
❌ No
❌ No
❌ No
🔴 Poor
init
✅ Yes
✅ Yes
❌ No
🟡 Fair
list
❌ No
❌ No
❌ No
🔴 Poor
pr
✅ Yes
✅ Yes
❌ No
🟡 Fair
remove
❌ No
❌ No
❌ No
🔴 Poor
run
❌ No
❌ No
❌ No
🔴 Poor
status
❌ No
❌ No
❌ No
🔴 Poor
trial
✅ Yes
✅ Yes
✅ Yes (1)
🟢 Good
update
✅ Yes
✅ Yes
❌ No
🟡 Fair
The compile command is particularly concerning as it's one of the most frequently used commands and currently has no help text.
User Impact Scenarios
Scenario 1: New User Compiling First Workflow
User runs: gh aw compile my-workflow.md
Error: "failed to read file: no such file or directory"
User confused: Is the file path wrong? Should it be relative? Absolute? From repo root?
Better error: "Workflow file not found: my-workflow.md. Expected path: .github/workflows/my-workflow.md. Example: gh aw compile example-workflow"
Scenario 2: Workflow Compilation Failure
Error: "generated lock file is not valid YAML: %w"
User stuck: What's wrong with the YAML? Which line? What's the syntax error?
Better error: Include YAML validation details, line number, and example of correct syntax
Scenario 3: Security Scanner Failure
Error: "zizmor security scan failed: %w"
User unclear: What failed? Can I see the scan output? Is this a workflow issue or tool issue?
Better error: Include scanner output summary and suggest --verbose for details
🤖 Tasks for Copilot Agent
NOTE TO PLANNER AGENT: The following tasks are designed for GitHub Copilot agent execution. Please split these into individual work items for Claude to process.
Improvement Tasks
The following code regions and tasks should be processed by the Copilot agent. Each section is marked for easy identification by the planner agent.
Task 1: Add Comprehensive Help Text to Core Commands
Priority: High Estimated Effort: Medium Focus Area: CLI Feedback Quality
Description:
Add Short descriptions, Long descriptions, and usage Examples to commands that currently lack them: compile, list, run, status, and remove. These are core commands that users frequently use, and they need clear guidance on usage patterns.
Acceptance Criteria:
All five commands have Short descriptions (1 sentence summary)
All five commands have Long descriptions (detailed explanation)
Each command has at least 2 usage examples showing common patterns
Examples demonstrate both basic and advanced usage
Help text follows the pattern established by the add command
Add comprehensive Cobra command help text to the five core commands that currently lack it. Follow this pattern:
For each command file (compile_command.go, list_command.go, run_command.go, status_command.go, remove_command.go):
1. Add a Short description that summarizes the command in one sentence
2. Add a Long description that explains:
- What the command does
- When to use it
- Important flags or options
- How it fits into the workflow compilation process
3. Add at least 2 Examples showing:
- Basic usage (most common case)
- Advanced usage with flags
- Common patterns specific to that command
Example structure (adapt for each command):
```go
Short: "Compile agentic workflow markdown files to GitHub Actions YAML",
Long: `Compile converts agentic workflow markdown files (.md) into GitHub Actions workflow files (.lock.yml). The compiler validates frontmatter, resolves imports, processes tool configurations, and generates the final YAML.By default, compiles all markdown files in .github/workflows/. You can compile specific workflows by name or use flags for validation-only mode, security scanning, or purging orphaned lock files.`,
Example: ` # Compile all workflows gh aw compile # Compile specific workflow gh aw compile my-workflow # Compile with security scanning gh aw compile --actionlint --zizmor --poutine # Validate without generating lock files gh aw compile --no-emit`,
Reference the add command (add_command.go) for the quality standard to match.
---
#### Task 2: Enhance Error Messages with Contextual Suggestions
**Priority**: High
**Estimated Effort**: Large
**Focus Area**: CLI Feedback Quality
**Description:**
Improve the 92.9% of error messages that currently lack helpful suggestions. Focus on the most common error scenarios: file not found, directory creation failures, validation errors, and tool execution failures. Transform generic "failed to X" messages into actionable guidance with examples and next steps.
**Acceptance Criteria:**
- [ ] File not found errors include expected path and example command
- [ ] Directory creation errors explain permissions and provide workaround
- [ ] Validation errors include specific problem description and fix suggestion
- [ ] Tool execution errors include debug flags and where to find logs
- [ ] At least 50 high-frequency errors improved (targeting 15-20% helpful error rate)
- [ ] All improved errors use `console.FormatErrorWithSuggestions()` where applicable
**Code Region:** `pkg/cli/*.go` (focus on compile_command.go, commands.go, mcp_config_file.go, jq.go)
```markdown
Enhance error messages in the CLI package to provide actionable guidance. Focus on these common error patterns:
1. **File Not Found Errors**
Current: `"workflow file '%s' not found"`
Improve to: Use `console.FormatErrorWithSuggestions()` with:
- Expected file location
- Example correct path
- Suggestion to check spelling or use `gh aw list`
2. **Directory Creation Errors**
Current: `"failed to create .github/workflows directory: %w"`
Improve to: Include:
- Explanation of why this directory is needed
- Permission requirements
- Suggestion to check repository permissions or run with appropriate access
3. **Validation Errors**
Current: `"generated lock file is not valid YAML: %w"`
Improve to: Include:
- Specific line number or section with error
- Common causes (indentation, special characters, quotes)
- Link to YAML validator or example of correct syntax
4. **Tool Execution Errors**
Current: `"jq not found in PATH"`, `"zizmor security scan failed: %w"`
Improve to: Include:
- Installation instructions or link
- Suggestion to use `--verbose` for details
- Alternative approaches if tool is optional
5. **Flag Conflict Errors**
Current: `"--purge flag can only be used when compiling all markdown files"`
Improve to: Include:
- Explanation of why the conflict exists
- Example of correct usage
- Alternative flags that might achieve the user's goal
Example transformation:
```go
// Before
return fmt.Errorf("failed to create .github/workflows directory: %w", err)
// After
return errors.New(console.FormatErrorWithSuggestions(
fmt.Sprintf("failed to create .github/workflows directory: %v", err),
[]string{
"Ensure you have write permissions in the repository",
"Try running from the repository root directory",
"Check if .github/ directory exists and is writable",
},
))
Focus on errors in:
compile_command.go (compilation failures)
commands.go (workflow resolution and file operations)
mcp_config_file.go (configuration file handling)
jq.go (jq tool execution)
---
#### Task 3: Add Progress Indicators for Long-Running Operations
**Priority**: Medium
**Estimated Effort**: Medium
**Focus Area**: CLI Feedback Quality
**Description:**
Enhance user feedback during long-running operations by adding structured progress indicators. The `compile` command already has some progress logging, but operations like security scanning, bulk compilation, and workflow validation would benefit from clearer progress indication showing which step is active and how many steps remain.
**Acceptance Criteria:**
- [ ] Compilation shows "Step X of Y: [operation name]" format
- [ ] Security scanning (actionlint, zizmor, poutine) shows progress for each scanner
- [ ] Bulk operations show "Processing file N of M: [filename]"
- [ ] Progress messages use `console.FormatProgressMessage()` consistently
- [ ] Long operations (>5 seconds) have at least one progress update mid-operation
- [ ] Final summary shows total time and success/failure counts
**Code Region:** `pkg/cli/compile_command.go`
```markdown
Add structured progress indicators to the compile command workflow. Create a clear progression showing:
1. **Compilation Phases**
- Step 1/5: Reading workflow files
- Step 2/5: Validating frontmatter
- Step 3/5: Resolving imports
- Step 4/5: Generating YAML
- Step 5/5: Running security scans (if applicable)
2. **Bulk Compilation**
When compiling multiple files, show:
- "Processing workflow 3 of 12: example-workflow.md"
- "Compiled successfully: 8/12, Failed: 1/12, Skipped: 3/12"
3. **Security Scanning**
For each scanner, show:
- "Running actionlint... " → "✓ actionlint passed"
- "Running zizmor... " → "✓ zizmor passed" or "⚠ zizmor found 2 warnings"
- "Running poutine... " → "✓ poutine passed"
4. **Summary Output**
At completion, show:
Implementation approach:
- Create a progress tracker struct to maintain state
- Use `console.FormatProgressMessage()` for all progress output
- Add completion checkmarks (✓/✗) for each phase
- Include timing information for operations >2 seconds
- Ensure progress messages only appear in verbose mode or for multi-file operations
Reference the existing progress indicators at line ~87-89 in compile_command.go as a starting point.
Priority: Medium Estimated Effort: Small Focus Area: CLI Feedback Quality
Description:
Document the error message patterns and best practices for contributors. Create a style guide that explains when to use different console formatting functions, how to structure helpful error messages, and provides before/after examples. This will help maintain consistency as the codebase grows and new contributors add features.
Acceptance Criteria:
New markdown file created at .github/instructions/error-message-style.instructions.md
Includes sections: Overview, Error Categories, Formatting Functions, Examples, Anti-patterns
Provides 5+ before/after examples
Links to existing good examples in the codebase
Explains when to use FormatErrorWithSuggestions vs basic fmt.Errorf
Documents the console formatting utilities and their use cases
Code Region:.github/instructions/ (new file)
Create a new error message style guide at `.github/instructions/error-message-style.instructions.md` with the following content:
# Error Message Style Guide for gh-aw CLI## Overview
This guide establishes patterns for clear, actionable error messages in the gh-aw CLI. Good error messages help users understand what went wrong and how to fix it, reducing support burden and improving developer experience.
## Error Message Categories### 1. User Input Errors
When the user provides invalid input, explain what's wrong and show correct usage.
**Pattern**: Specific problem + Correct format + Example
```goreturn errors.New(console.FormatErrorWithSuggestions(
fmt.Sprintf("invalid workflow name: '%s' contains special characters", name),
[]string{
"Workflow names can only contain: a-z, A-Z, 0-9, hyphens, underscores",
"Example: gh aw compile my-workflow-name",
},
))
2. File/Path Errors
Include expected path, current working directory context, and navigation help.
Pattern: What's missing + Where we looked + How to fix
returnerrors.New(console.FormatErrorWithSuggestions(
fmt.Sprintf("workflow file not found: %s", filename),
[]string{
fmt.Sprintf("Expected location: .github/workflows/%s", filename),
"List available workflows: gh aw list",
"Ensure you're in the repository root directory",
},
))
3. Tool/Dependency Errors
Explain what tool is missing, why it's needed, and how to install it.
Pattern: Tool name + Purpose + Installation
returnerrors.New(console.FormatErrorWithSuggestions(
"jq not found in PATH",
[]string{
"jq is required for filtering workflow logs",
"Install: brew install jq (macOS) or apt-get install jq (Linux)",
"Documentation: (redacted)",
},
))
4. Validation Errors
Include specific location of problem, explanation, and how to fix.
Pattern: Specific issue + Location + Fix
returnerrors.New(console.FormatErrorWithSuggestions(
"frontmatter validation failed: unknown field 'engnie'",
[]string{
"Did you mean: 'engine'?",
"Valid fields: on, permissions, engine, tools, safe-outputs",
"See schema documentation: (redacted)",
},
))
5. Operation Failures
Wrap technical errors with user-friendly context and next steps.
Pattern: What failed + Why + Debug options
returnfmt.Errorf("%w\n\nTroubleshooting:\n • Run with --verbose for detailed logs\n • Check file permissions in .github/workflows/\n • Verify GitHub Actions is enabled in repository settings", err)
Use for informational output during normal operation.
FormatSuccessMessage(msg string)
Use for completion confirmations.
FormatProgressMessage(msg string)
Use for progress updates during long operations.
Before/After Examples
Example 1: File Operations
Before: return fmt.Errorf("failed to create directory: %w", err)
After:
returnerrors.New(console.FormatErrorWithSuggestions(
fmt.Sprintf("failed to create .github/workflows directory: %v", err),
[]string{
"Ensure you have write permissions in the repository",
"Check if .github/ directory exists: ls -la .github/",
"Try running from repository root",
},
))
Example 2: Invalid Flags
Before: return fmt.Errorf("--purge flag can only be used when compiling all markdown files")
After:
returnerrors.New(console.FormatErrorWithSuggestions(
"--purge flag can only be used when compiling all workflows",
[]string{
"Remove specific workflow names from the command",
"Correct usage: gh aw compile --purge",
"To compile one workflow: gh aw compile my-workflow (without --purge)",
},
))
returnerrors.New(console.FormatErrorWithSuggestions(
fmt.Sprintf("zizmor security scan failed: %v", err),
[]string{
"Run with --verbose to see detailed scan output",
"Common issues: GitHub Actions syntax errors, missing required fields",
"Skip security scans: gh aw compile (without --zizmor flag)",
},
))
Anti-patterns to Avoid
❌ Generic wrapped errors without context
returnfmt.Errorf("operation failed: %w", err)
❌ Technical jargon without explanation
returnerrors.New("YAML unmarshal error at line 42")
❌ Errors without actionable next steps
returnerrors.New("validation failed")
❌ Assuming user knowledge
returnerrors.New("check your frontmatter") // What should they check?
Good Patterns to Follow
✅ Specific, contextual, actionable
returnerrors.New(console.FormatErrorWithSuggestions(
"frontmatter validation failed: 'on' field is required",
[]string{
"Add workflow trigger: on: issues",
"Example frontmatter: (redacted)",
},
))
✅ Include examples of correct usage
✅ Link to documentation when relevant
✅ Suggest debug flags for complex issues
✅ Use consistent terminology from user documentation
Testing Error Messages
When adding new error messages, test them by:
Triggering the error condition locally
Verifying the output is clear and actionable
Following your own suggestions to ensure they work
Having a colleague review the message for clarity
References
Good examples in the codebase:
pkg/cli/commands.go - workflow resolution errors
pkg/cli/resolver.go - file not found with suggestions
---
#### Task 5: Add Usage Examples to Console Formatting Documentation
**Priority**: Low
**Estimated Effort**: Small
**Focus Area**: CLI Feedback Quality
**Description:**
The `pkg/console` package provides formatting utilities but lacks comprehensive usage examples in its documentation. Add godoc comments with examples showing how to use each formatting function, when to choose one over another, and common patterns for combining them with error handling.
**Acceptance Criteria:**
- [ ] Each public function in `pkg/console` has godoc with usage example
- [ ] Examples show common patterns (e.g., wrapping errors, multi-line suggestions)
- [ ] Documentation explains when to use each formatting function
- [ ] Includes example of FormatErrorWithSuggestions with 3+ suggestions
- [ ] Shows integration with fmt.Fprintln(os.Stderr, ...) pattern
**Code Region:** `pkg/console/*.go`
```markdown
Add comprehensive godoc comments to the console formatting package. For each public function, include:
1. **FormatErrorMessage**
```go
// FormatErrorMessage formats an error message with appropriate styling for terminal output.
// Use this for simple error output without suggestions.
//
// Example:
// fmt.Fprintln(os.Stderr, console.FormatErrorMessage("Compilation failed"))
//
// For errors that need actionable suggestions, use FormatErrorWithSuggestions instead.
func FormatErrorMessage(msg string) string {
FormatErrorWithSuggestions
// FormatErrorWithSuggestions formats an error message with a list of actionable suggestions.// This is the preferred function for user-facing errors as it guides users toward resolution.//// Example:// err := console.FormatErrorWithSuggestions(// "workflow file not found: my-workflow.md",// []string{// "Check the filename spelling",// "Ensure the file is in .github/workflows/",// "List available workflows: gh aw list",// },// )// return errors.New(err)//// The formatted output includes:// - The main error message// - A "Suggestions:" header// - Each suggestion as a bullet pointfuncFormatErrorWithSuggestions(msgstring, suggestions []string) string {
FormatWarningMessage
// FormatWarningMessage formats a warning message for non-critical issues.// Use this for issues that don't prevent operation but warrant user attention.//// Example:// fmt.Fprintln(os.Stderr, console.FormatWarningMessage("Workflow uses deprecated syntax"))//// Warnings are styled differently than errors to indicate lower severity.funcFormatWarningMessage(msgstring) string {
FormatInfoMessage, FormatSuccessMessage, FormatProgressMessage
Add similar documentation for each showing:
When to use the function
Example usage with fmt.Fprintln(os.Stderr, ...)
How it differs from other formatting functions
FormatError (CompilerError struct)
// FormatError formats a CompilerError with file location, error message, and optional code context.// This is used primarily by the compiler to show validation errors with line numbers.//// Example:// err := console.FormatError(console.CompilerError{// File: "my-workflow.md",// Line: 15,// Message: "unknown field 'engnie' (did you mean 'engine'?)",// Context: " engnie: copilot",// })// fmt.Fprintln(os.Stderr, err)funcFormatError(errCompilerError) string {
Add a package-level documentation comment explaining:
Purpose of the console package
When to use formatted vs plain output
Pattern: fmt.Fprintln(os.Stderr, console.Format...) for CLI output
Reference to error-message-style.instructions.md for best practices
---
## 📊 Historical Context
<details>
<summary><b>Previous Focus Areas</b></summary>
| Date | Focus Area | Type | Custom | Key Outcomes |
|------|------------|------|--------|--------------|
| 2024-11-24 | CLI Feedback Quality | Custom | Y | First analysis - identified 92.9% of errors lack helpful suggestions, created 5 improvement tasks |
</details>
---
## 🎯 Recommendations
### Immediate Actions (This Week)
1. **Add help text to compile command** - Priority: High
This is the most-used command and currently has no help text, creating confusion for new users.
2. **Enhance file-not-found errors** - Priority: High
These are the most common errors users encounter and have the biggest impact on first-time experience.
### Short-term Actions (This Month)
1. **Improve validation error messages** - Priority: Medium
Add specific field names, line numbers, and fix suggestions to frontmatter validation errors.
2. **Add progress indicators to bulk operations** - Priority: Medium
Help users understand progress during multi-file compilation and security scanning.
### Long-term Actions (This Quarter)
1. **Create error message style guide** - Priority: Low
Document best practices for future contributors to maintain consistency.
2. **Audit all 613 error messages** - Priority: Low
Systematically review and enhance remaining error messages to achieve 50%+ helpful error rate.
---
## 📈 Success Metrics
Track these metrics to measure improvement in **CLI Feedback Quality**:
- **Helpful Error Rate**: 7.1% → 25% (target: improve 110 error messages)
- **Commands with Help Text**: 50% (5/10) → 100% (10/10)
- **Commands with Examples**: 20% (2/10) → 80% (8/10)
- **Format Adoption Rate**: 87% → 95% (eliminate remaining plain output)
- **User Support Requests**: Baseline TBD → Reduce by 30% (track via issues/discussions)
---
## Next Steps
1. Review and prioritize the tasks above
2. Assign tasks to Copilot agent via planner agent
3. Track progress on improvement items using issue labels: `cli-feedback-quality`
4. Re-evaluate this focus area in **1 month** to measure impact
5. Next analysis: **2024-11-25** - Focus area will be selected based on diversity algorithm (aim for another custom area or strategic reuse)
---
*Generated by Repository Quality Improvement Agent*
*Focus Area Strategy: Custom (60% target rate)*
*Next run will prioritize: Standard category (30% probability) or Custom area (60% probability)*
> AI generated by [Repository Quality Improvement Agent](https://github.com/githubnext/gh-aw/actions/runs/19635568676)
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
🎯 Repository Quality Improvement Report - CLI Feedback Quality
Analysis Date: 2024-11-24
Focus Area: CLI Feedback Quality
Strategy Type: Custom
Custom Area: Yes - This focus area addresses the specific challenge of providing clear, actionable feedback to users (both human developers and AI agents) when working with agentic workflows. Given that gh-aw is a tool for creating AI-driven workflows, exceptional error messages and user guidance are critical for adoption and debugging.
Executive Summary
The gh-aw CLI demonstrates strong adoption of structured console formatting (87% formatted output) and maintains consistent messaging patterns across 754 formatted output locations. However, analysis reveals significant opportunities for improvement in error message helpfulness and command documentation. Only 7.1% of error messages (44 out of 613) provide actionable suggestions or examples, leaving 92.9% of errors as basic technical messages that may confuse users. Additionally, critical commands like
compile,list,run, andstatuslack Short/Long descriptions and examples in their help text, creating gaps in user guidance.The impact is particularly acute for this tool because users are learning to work with agentic workflows—a novel paradigm. When compilation fails or workflows behave unexpectedly, users need clear guidance on what went wrong and how to fix it. Improving CLI feedback quality will reduce support burden, accelerate user onboarding, and improve the overall developer experience.
Full Analysis Report
Focus Area: CLI Feedback Quality
Current State Assessment
The gh-aw CLI has established good foundations for user feedback through the
pkg/consolepackage, which provides formatting utilities for success, error, warning, info, and progress messages. The codebase shows strong discipline in using these utilities, with 87% of output properly formatted.Metrics Collected:
Findings
Strengths
console.FormatErrorWithSuggestions()function demonstrates awareness of good error message designAreas for Improvement
Low Helpful Error Rate (7.1%):
Incomplete Command Help Text:
compilecommand: No Short/Long description or exampleslist,run,status,removecommands: Missing help textaddcommand has comprehensive help with 3 examplesGeneric Error Messages:
fmt.Errorf("failed to X: %w", err)provides technical error but no context"failed to create .github/workflows directory: %w"doesn't explain why or how to fixInconsistent Example Provision:
Detailed Analysis
Error Message Categories
Category 1: Basic Technical Errors (92.9%)
Category 2: Helpful Errors with Suggestions (7.1%)
Command Help Quality Analysis
The
compilecommand is particularly concerning as it's one of the most frequently used commands and currently has no help text.User Impact Scenarios
Scenario 1: New User Compiling First Workflow
gh aw compile my-workflow.md"failed to read file: no such file or directory"Scenario 2: Workflow Compilation Failure
"generated lock file is not valid YAML: %w"Scenario 3: Security Scanner Failure
"zizmor security scan failed: %w"--verbosefor details🤖 Tasks for Copilot Agent
NOTE TO PLANNER AGENT: The following tasks are designed for GitHub Copilot agent execution. Please split these into individual work items for Claude to process.
Improvement Tasks
The following code regions and tasks should be processed by the Copilot agent. Each section is marked for easy identification by the planner agent.
Task 1: Add Comprehensive Help Text to Core Commands
Priority: High
Estimated Effort: Medium
Focus Area: CLI Feedback Quality
Description:
Add Short descriptions, Long descriptions, and usage Examples to commands that currently lack them:
compile,list,run,status, andremove. These are core commands that users frequently use, and they need clear guidance on usage patterns.Acceptance Criteria:
addcommandCode Region:
pkg/cli/*_command.go(compile_command.go, list_command.go, run_command.go, status_command.go, remove_command.go)Reference the
addcommand (add_command.go) for the quality standard to match.Focus on errors in:
✓ Compilation completed in 12.3s
Summary:
Total workflows: 12
Successfully compiled: 10
Failed: 2
Skipped (up-to-date): 0
Task 4: Create Error Message Style Guide Documentation
Priority: Medium
Estimated Effort: Small
Focus Area: CLI Feedback Quality
Description:
Document the error message patterns and best practices for contributors. Create a style guide that explains when to use different console formatting functions, how to structure helpful error messages, and provides before/after examples. This will help maintain consistency as the codebase grows and new contributors add features.
Acceptance Criteria:
.github/instructions/error-message-style.instructions.mdCode Region:
.github/instructions/(new file)2. File/Path Errors
Include expected path, current working directory context, and navigation help.
Pattern: What's missing + Where we looked + How to fix
3. Tool/Dependency Errors
Explain what tool is missing, why it's needed, and how to install it.
Pattern: Tool name + Purpose + Installation
4. Validation Errors
Include specific location of problem, explanation, and how to fix.
Pattern: Specific issue + Location + Fix
5. Operation Failures
Wrap technical errors with user-friendly context and next steps.
Pattern: What failed + Why + Debug options
Console Formatting Functions
FormatErrorMessage(msg string)
Use for simple error output without suggestions.
FormatErrorWithSuggestions(msg string, suggestions []string)
Use this for most user-facing errors. Provides actionable next steps.
FormatWarningMessage(msg string)
Use for non-critical issues that don't stop execution.
FormatInfoMessage(msg string)
Use for informational output during normal operation.
FormatSuccessMessage(msg string)
Use for completion confirmations.
FormatProgressMessage(msg string)
Use for progress updates during long operations.
Before/After Examples
Example 1: File Operations
Before:
return fmt.Errorf("failed to create directory: %w", err)After:
Example 2: Invalid Flags
Before:
return fmt.Errorf("--purge flag can only be used when compiling all markdown files")After:
Example 3: Tool Execution
Before:
return fmt.Errorf("zizmor security scan failed: %w", err)After:
Anti-patterns to Avoid
❌ Generic wrapped errors without context
❌ Technical jargon without explanation
❌ Errors without actionable next steps
❌ Assuming user knowledge
Good Patterns to Follow
✅ Specific, contextual, actionable
✅ Include examples of correct usage
✅ Link to documentation when relevant
✅ Suggest debug flags for complex issues
✅ Use consistent terminology from user documentation
Testing Error Messages
When adding new error messages, test them by:
References
Good examples in the codebase:
pkg/cli/commands.go- workflow resolution errorspkg/cli/resolver.go- file not found with suggestionspkg/workflow/resolve.go- import resolution errorsAdd similar documentation for each showing:
Add a package-level documentation comment explaining:
Beta Was this translation helpful? Give feedback.
All reactions