Fix /create-issue workflow: parsing, error handling, speed, and testing by radical · Pull Request #15972 · microsoft/aspire

radical · 2026-04-08T19:59:02Z

Description

Follow-up to #15780 (merged as PR #15973 for the initial restore fix). This PR contains the remaining fixes, improvements, and test coverage for the /create-issue workflow.

Fixes

Positional + flag parsing bug: /create-issue TestName --force-new was rejected because the parser didn't allow positional test names when flags were present. Fixed in create-failing-test-issue.js.
Silent build failure in list-only mode: When the resolver step failed (e.g., build error), the workflow silently posted "No test failures found" instead of reporting the error. Added RESOLVE_OUTCOME check.
Speed optimization: Replaced restore.sh (~2.5 min) with actions/setup-dotnet (~5s) by isolating CreateFailingTestIssue from the Arcade SDK. Added Directory.Build.props/.targets/Packages.props under the tool directory and replaced ProjectReference to Aspire.TestTools with shared source links.

New features

workflow_dispatch trigger: Enables testing from any branch with inputs for test_query, source_url, workflow, force_new, and pr_number.
formatListResponse() helper: Extracted list-only response formatting into a testable JS function.

Test coverage

4 new workflow tests: positional+flags parsing, resolver failure error handling, test names from result, empty results
CI skip pattern added for create-failing-test-issue.*

Files changed

.github/workflows/create-failing-test-issue.yml — workflow fixes + speed optimization
.github/workflows/create-failing-test-issue.js — parser fix + formatListResponse
tools/CreateFailingTestIssue/ — Arcade isolation (Directory.Build.props/targets/Packages.props, shared source)
tests/Infrastructure.Tests/WorkflowScripts/ — new tests + harness update
eng/testing/github-ci-trigger-patterns.txt — CI skip pattern

E2E validation

All 7 /create-issue variants tested successfully via workflow_dispatch and PR comments during development.

Microsoft Reviewers: Open in CodeFlow

github-actions · 2026-04-08T19:59:18Z

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 15972

Or

Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 15972"

radical · 2026-04-08T20:07:54Z

/create-issue

github-actions · 2026-04-08T20:10:07Z

No test failures were found. Use --url to point to a specific workflow run.

📋 /create-issue — Usage

Creates or updates a failing-test issue from CI failures.

/create-issue <test-name>
/create-issue <test-name> <pr|run|job-url>
/create-issue --test "<test-name>"
/create-issue --test "<test-name>" --url <pr|run|job-url>
/create-issue --test "<test-name>" --force-new

radical · 2026-04-08T20:11:43Z

/create-issue --url https://github.com/microsoft/aspire/actions/runs/24155707576

github-actions · 2026-04-08T20:13:56Z

No test failures were found. Use --url to point to a specific workflow run.

📋 /create-issue — Usage

Creates or updates a failing-test issue from CI failures.

/create-issue <test-name>
/create-issue <test-name> <pr|run|job-url>
/create-issue --test "<test-name>"
/create-issue --test "<test-name>" --url <pr|run|job-url>
/create-issue --test "<test-name>" --force-new

radical · 2026-04-08T20:48:58Z

/create-issue

github-actions · 2026-04-08T20:54:21Z

No test failures were found. Use --url to point to a specific workflow run.

📋 /create-issue — Usage

Creates or updates a failing-test issue from CI failures.

/create-issue <test-name>
/create-issue <test-name> <pr|run|job-url>
/create-issue --test "<test-name>"
/create-issue --test "<test-name>" --url <pr|run|job-url>
/create-issue --test "<test-name>" --force-new

- Remove --no-restore from dotnet build (fix NETSDK1004) - Add workflow_dispatch trigger for testing from any branch - Make workflow steps handle both issue_comment and workflow_dispatch - Intentionally break two tests for end-to-end validation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

radical · 2026-04-08T21:26:14Z

/create-issue

github-actions · 2026-04-08T21:29:06Z

Failed tests found on this PR:

/create-issue Aspire.Hosting.Tests.ModelNameTests.ValidateName_Empty_Error
/create-issue Aspire.Hosting.Tests.ValueSnapshotTests.IsValueSet_FalseBeforeAnySet

📋 /create-issue — Usage

Creates or updates a failing-test issue from CI failures.

/create-issue <test-name>
/create-issue <test-name> <pr|run|job-url>
/create-issue --test "<test-name>"
/create-issue --test "<test-name>" --url <pr|run|job-url>
/create-issue --test "<test-name>" --force-new

radical · 2026-04-08T21:29:52Z

/create-issue Aspire.Hosting.Tests.ValueSnapshotTests.IsValueSet_FalseBeforeAnySet

github-actions · 2026-04-08T21:32:28Z

✅ Created failing-test issue #15978: #15978

To disable this test on your PR, comment:

/disable-test Aspire.Hosting.Tests.ValueSnapshotTests.IsValueSet_FalseBeforeAnySet https://github.com/microsoft/aspire/issues/15978

radical · 2026-04-08T21:32:49Z

/create-issue --test "Aspire.Hosting.Tests.ModelNameTests.ValidateName_Empty_Error"

github-actions · 2026-04-08T21:36:06Z

✅ Created failing-test issue #15979: #15979

To disable this test on your PR, comment:

/disable-test Aspire.Hosting.Tests.ModelNameTests.ValidateName_Empty_Error https://github.com/microsoft/aspire/issues/15979

radical · 2026-04-08T21:36:25Z

/create-issue Aspire.Hosting.Tests.ValueSnapshotTests.IsValueSet_FalseBeforeAnySet

github-actions · 2026-04-08T21:48:24Z

✅ Updated failing-test issue #15978: #15978

To disable this test on your PR, comment:

/disable-test Aspire.Hosting.Tests.ValueSnapshotTests.IsValueSet_FalseBeforeAnySet https://github.com/microsoft/aspire/issues/15978

radical · 2026-04-08T21:48:52Z

/create-issue Aspire.Hosting.Tests.ValueSnapshotTests.IsValueSet_FalseBeforeAnySet --force-new

github-actions · 2026-04-08T21:50:13Z

@radical ❌ Unknown argument 'Aspire.Hosting.Tests.ValueSnapshotTests.IsValueSet_FalseBeforeAnySet'. Supported arguments are --test, --url, --workflow, and --force-new.

The parser rejected positional test names when any flag was present, e.g. '/create-issue TestName --force-new' would fail with 'Unknown argument'. Now non-flag tokens are accepted as positional test names in flag mode. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

radical · 2026-04-08T21:52:59Z

/create-issue --test "Aspire.Hosting.Tests.ValueSnapshotTests.IsValueSet_FalseBeforeAnySet" --force-new

radical · 2026-04-08T21:53:05Z

/create-issue --test "Aspire.Hosting.Tests.ModelNameTests.ValidateName_Empty_Error" --url https://github.com/microsoft/aspire/actions/runs/24158509554

github-actions · 2026-04-08T22:12:47Z

✅ Created failing-test issue #15982: #15982

To disable this test on your PR, comment:

/disable-test Aspire.Hosting.Tests.ValueSnapshotTests.IsValueSet_FalseBeforeAnySet https://github.com/microsoft/aspire/issues/15982

github-actions · 2026-04-08T22:20:13Z

✅ Reopened failing-test issue #15979: #15979

To disable this test on your PR, comment:

/disable-test Aspire.Hosting.Tests.ModelNameTests.ValidateName_Empty_Error https://github.com/microsoft/aspire/issues/15979

github-actions · 2026-04-08T22:26:02Z

✅ Reopened failing-test issue #15982: #15982

To disable this test on your PR, comment:

/disable-test Aspire.Hosting.Tests.ValueSnapshotTests.IsValueSet_FalseBeforeAnySet https://github.com/microsoft/aspire/issues/15982

All /create-issue variants tested successfully: - List-only (no args) - Positional test name - --test flag - Update existing issue - --force-new - --url with explicit run URL - workflow_dispatch with pr_number Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Extract the list-only response logic from inline YAML into a testable formatListResponse() function in the JS helper. This covers the bug where a resolver build failure silently fell through to 'No test failures found' instead of reporting the error. New tests: - FormatListResponseReturnsErrorWhenResolverFailed - FormatListResponseReturnsTestNamesFromResult - FormatListResponseReturnsNoFailuresWhenResultIsEmpty Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Isolate CreateFailingTestIssue from the repo's Arcade SDK by adding Directory.Build.props/targets/Packages.props that cut inheritance. Replace ProjectReference to Aspire.TestTools with shared source links. Replace restore.sh (~2.5min) with setup-dotnet action (~5s). The tool builds in ~4s with a plain dotnet SDK, no Arcade needed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

# Conflicts: # .github/workflows/create-failing-test-issue.yml

github-actions · 2026-04-08T22:47:34Z

No test failures were found. Use --url to point to a specific workflow run.

📋 /create-issue — Usage

Creates or updates a failing-test issue from CI failures.

/create-issue <test-name>
/create-issue <test-name> <pr|run|job-url>
/create-issue --test "<test-name>"
/create-issue --test "<test-name>" --url <pr|run|job-url>
/create-issue --test "<test-name>" --force-new

github-actions · 2026-04-08T22:54:02Z

Failed tests found on this PR:

/create-issue Aspire.Cli.EndToEnd.Tests.JavaScriptPublishTests.AllPublishMethodsBuildDockerImages
/create-issue Aspire.Cli.EndToEnd.Tests.StartStopTests.AddPackageInteractiveWhileAppHostRunningDetached
/create-issue Aspire.Cli.EndToEnd.Tests.StartStopTests.AddPackageWhileAppHostRunningDetached
/create-issue Aspire.Cli.EndToEnd.Tests.StartStopTests.CreateStartAndStopAspireProject
/create-issue Aspire.Cli.EndToEnd.Tests.StartStopTests.StopWithNoRunningAppHostExitsSuccessfully

📋 /create-issue — Usage

Creates or updates a failing-test issue from CI failures.

/create-issue <test-name>
/create-issue <test-name> <pr|run|job-url>
/create-issue --test "<test-name>"
/create-issue --test "<test-name>" --url <pr|run|job-url>
/create-issue --test "<test-name>" --force-new

github-actions · 2026-04-08T22:57:02Z

✅ Created failing-test issue #15983: #15983

To disable this test on your PR, comment:

/disable-test Aspire.Cli.EndToEnd.Tests.StartStopTests.StopWithNoRunningAppHostExitsSuccessfully https://github.com/microsoft/aspire/issues/15983

Replace Node 20 github-script@v7 with github-script@v8 to avoid the upcoming Node 20 deprecation on GitHub Actions runners. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-04-08T23:17:00Z

Failed tests found on this PR:

/create-issue Aspire.Cli.EndToEnd.Tests.JavaScriptPublishTests.AllPublishMethodsBuildDockerImages
/create-issue Aspire.Cli.EndToEnd.Tests.StartStopTests.AddPackageInteractiveWhileAppHostRunningDetached
/create-issue Aspire.Cli.EndToEnd.Tests.StartStopTests.AddPackageWhileAppHostRunningDetached
/create-issue Aspire.Cli.EndToEnd.Tests.StartStopTests.CreateStartAndStopAspireProject
/create-issue Aspire.Cli.EndToEnd.Tests.StartStopTests.StopWithNoRunningAppHostExitsSuccessfully

📋 /create-issue — Usage

Creates or updates a failing-test issue from CI failures.

/create-issue <test-name>
/create-issue <test-name> <pr|run|job-url>
/create-issue --test "<test-name>"
/create-issue --test "<test-name>" --url <pr|run|job-url>
/create-issue --test "<test-name>" --force-new

github-actions · 2026-04-08T23:18:02Z

✅ Created failing-test issue #15985: #15985

To disable this test on your PR, comment:

/disable-test Aspire.Cli.EndToEnd.Tests.StartStopTests.CreateStartAndStopAspireProject https://github.com/microsoft/aspire/issues/15985

github-actions · 2026-04-09T00:09:01Z

Re-running the failed jobs in the CI workflow for this pull request because 1 job was identified as retry-safe transient failures in the CI run attempt.
GitHub was asked to rerun all failed jobs for that attempt, and the rerun is being tracked in the rerun attempt.
The job links below point to the failed attempt jobs that matched the retry-safe transient failure rules.

Tests / Hosting-5 / Hosting-5 (windows-latest) - Job-level runner or infrastructure failure matched the transient allowlist.

radical · 2026-04-09T04:43:05Z

Closed in favor of #15988

radical mentioned this pull request Apr 8, 2026

Fix create-failing-test-issue: remove --no-restore, add workflow_dispatch #15973

Merged

radical force-pushed the fix-create-issue branch from 8eb6fec to 6f700dd Compare April 8, 2026 20:41

radical force-pushed the fix-create-issue branch from 6f700dd to 7dc7e66 Compare April 8, 2026 20:56

radical force-pushed the fix-create-issue branch from 7dc7e66 to b38c474 Compare April 8, 2026 21:01

radical and others added 2 commits April 8, 2026 18:27

radical changed the title ~~Fix /create-issue workflow restore failure~~ Fix /create-issue workflow: parsing, error handling, speed, and testing Apr 8, 2026

Merge remote-tracking branch 'origin/main' into fix-create-issue

bf5d7ab

# Conflicts: # .github/workflows/create-failing-test-issue.yml

Update actions/github-script to v8 (Node 22)

f4c87b4

Replace Node 20 github-script@v7 with github-script@v8 to avoid the upcoming Node 20 deprecation on GitHub Actions runners. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

radical closed this Apr 9, 2026

Conversation

radical commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Fixes

New features

Test coverage

Files changed

E2E validation

Microsoft Reviewers: Open in CodeFlow

Uh oh!

github-actions bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

radical commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

radical commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

radical commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

radical commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

radical commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

radical commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

radical commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

radical commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

radical commented Apr 8, 2026

Uh oh!

radical commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 9, 2026

Uh oh!

radical commented Apr 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

radical commented Apr 8, 2026 •

edited

Loading

github-actions bot commented Apr 8, 2026 •

edited

Loading