diff --git a/.claude/skills/code-testing-agent/SKILL.md b/.claude/skills/code-testing-agent/SKILL.md new file mode 100644 index 000000000000..e4dbe0387e89 --- /dev/null +++ b/.claude/skills/code-testing-agent/SKILL.md @@ -0,0 +1,132 @@ +--- +name: code-testing-agent +description: >- + Generates comprehensive, workable unit tests for any programming language + using a multi-agent pipeline. Use when asked to generate tests, write unit + tests, improve test coverage, add test coverage, create test files, or test a + codebase. Supports C#, TypeScript, JavaScript, Python, Go, Rust, Java, and + more. Orchestrates research, planning, and implementation phases to produce + tests that compile, pass, and follow project conventions. +--- + +# Code Testing Generation Skill + +An AI-powered skill that generates comprehensive, workable unit tests for any programming language using a coordinated multi-agent pipeline. + +## When to Use This Skill + +Use this skill when you need to: + +- Generate unit tests for an entire project or specific files +- Improve test coverage for existing codebases +- Create test files that follow project conventions +- Write tests that actually compile and pass +- Add tests for new features or untested code + +## When Not to Use + +- Running or executing existing tests (use the `run-tests` skill) +- Migrating between test frameworks (use migration skills) +- Debugging failing test logic + +## How It Works + +This skill coordinates multiple specialized agents in a **Research → Plan → Implement** pipeline: + +### Pipeline Overview + +``` +┌─────────────────────────────────────────────────────────────┐ +│ TEST GENERATOR │ +│ Coordinates the full pipeline and manages state │ +└─────────────────────┬───────────────────────────────────────┘ + │ + ┌─────────────┼─────────────┐ + ▼ ▼ ▼ +┌───────────┐ ┌───────────┐ ┌───────────────┐ +│ RESEARCHER│ │ PLANNER │ │ IMPLEMENTER │ +│ │ │ │ │ │ +│ Analyzes │ │ Creates │ │ Writes tests │ +│ codebase │→ │ phased │→ │ per phase │ +│ │ │ plan │ │ │ +└───────────┘ └───────────┘ └───────┬───────┘ + │ + ┌─────────┬───────┼───────────┐ + ▼ ▼ ▼ ▼ + ┌─────────┐ ┌───────┐ ┌───────┐ ┌───────┐ + │ BUILDER │ │TESTER │ │ FIXER │ │LINTER │ + │ │ │ │ │ │ │ │ + │ Compiles│ │ Runs │ │ Fixes │ │Formats│ + │ code │ │ tests │ │ errors│ │ code │ + └─────────┘ └───────┘ └───────┘ └───────┘ +``` + +## Step-by-Step Instructions + +### Step 1: Determine the user request + +Make sure you understand what user is asking and for what scope. +When the user does not express strong requirements for test style, coverage goals, or conventions, +source the guidelines from [unit-test-generation.prompt.md](unit-test-generation.prompt.md). This prompt provides best practices for discovering conventions, parameterization strategies, coverage goals (aim for 80%), and language-specific patterns. + +### Step 2: Invoke the Test Generator + +Start by calling the `code-testing-generator` agent with your test generation request: + +``` +Generate unit tests for [path or description of what to test], following the unit-test-generation.prompt.md guidelines +``` + +The Test Generator will manage the entire pipeline automatically. + +## State Management + +All pipeline state is stored in `.testagent/` folder: + +| File | Purpose | +| ------------------------ | ---------------------------- | +| `.testagent/research.md` | Codebase analysis results | +| `.testagent/plan.md` | Phased implementation plan | +| `.testagent/status.md` | Progress tracking (optional) | + +## Agent Reference + +| Agent | Purpose | +| -------------------------- | -------------------- | +| `code-testing-generator` | Coordinates pipeline | +| `code-testing-researcher` | Analyzes codebase | +| `code-testing-planner` | Creates test plan | +| `code-testing-implementer` | Writes test files | +| `code-testing-builder` | Compiles code | +| `code-testing-tester` | Runs tests | +| `code-testing-fixer` | Fixes errors | +| `code-testing-linter` | Formats code | + +## Requirements + +- Project must have a build/test system configured +- Testing framework should be installed (or installable) +- GitHub Actions environment with Copilot CLI/agentic workflow support, or VS Code with GitHub Copilot extension + +## Troubleshooting + +### Tests don't compile + +The `code-testing-fixer` agent will attempt to resolve compilation errors. Check `.testagent/plan.md` for the expected test structure. Check the `.claude/skills/code-testing-agent/extensions/` folder for language-specific error code references (e.g., `.claude/skills/code-testing-agent/extensions/dotnet.md` for .NET). + +### Tests fail + +Most failures in generated tests are caused by **wrong expected values in assertions**, not production code bugs: + +1. Read the actual test output +2. Read the production code to understand correct behavior +3. Fix the assertion, not the production code +4. Never mark tests `[Ignore]` or `[Skip]` just to make them pass + +### Environment-dependent tests fail + +Tests that depend on external services, network endpoints, specific ports, or precise timing will fail in CI environments. Focus on unit tests with mocked dependencies instead. + +### Build fails on full solution + +During phase implementation, build only the specific test project for speed. After all phases, run a full non-incremental workspace build to catch cross-project errors. diff --git a/.claude/skills/code-testing-agent/extensions/dotnet.md b/.claude/skills/code-testing-agent/extensions/dotnet.md new file mode 100644 index 000000000000..f737ae58bac4 --- /dev/null +++ b/.claude/skills/code-testing-agent/extensions/dotnet.md @@ -0,0 +1,68 @@ +# .NET Extension + +Language-specific guidance for .NET (C#/F#/VB) test generation. + +## Build Commands + +| Scope | Command | +|-------|---------| +| Specific test project | `dotnet build MyProject.Tests.csproj` | +| Full solution (final validation) | `dotnet build MySolution.sln --no-incremental` | +| From repo root (no .sln) | `dotnet build --no-incremental` | + +- Use `--no-restore` if dependencies are already restored +- Use `-v:q` (quiet) to reduce output noise +- Always use `--no-incremental` for the final validation build — incremental builds hide errors like CS7036 + +## Test Commands + +| Scope | Command | +|-------|---------| +| All tests (MSTest projects) | `dotnet test path/to/TestProject.csproj` | +| Filtered (MSTest) | `dotnet test path/to/TestProject.csproj --filter "FullyQualifiedName~ClassName"` | +| After build (MSTest) | `dotnet test --no-build` | +| XUnit v3 assemblies | `dotnet exec artifacts/bin/redist/Debug/TestAssembly.dll -method "*TestMethodName*"` | + +- Use `--no-build` if already built +- Use `-v:q` for quieter output +- **This repo uses both MSTest and xUnit**. For xUnit test assemblies, use `dotnet exec` on the built DLL rather than `dotnet test` + +## Lint Command + +```bash +dotnet format --include path/to/file.cs +dotnet format MySolution.sln # full solution +``` + +## Project Reference Validation + +Before writing test code, read the test project's `.csproj` to verify it has `` entries for the assemblies your tests will use. If a reference is missing, add it: + +```xml + + + +``` + +This prevents CS0234 ("namespace not found") and CS0246 ("type not found") errors. + +## Common CS Error Codes + +| Error | Meaning | Fix | +|-------|---------|-----| +| CS0234 | Namespace not found | Add `` to the source project in the test `.csproj` | +| CS0246 | Type not found | Add `using Namespace;` or add missing `` | +| CS0103 | Name not found | Check spelling, add `using` statement | +| CS1061 | Missing member | Verify method/property name matches the source code exactly | +| CS0029 | Type mismatch | Cast or change the type to match the expected signature | +| CS7036 | Missing required parameter | Read the constructor/method signature and pass all required arguments | + +## `.csproj` / `.sln` Handling + +- During phase implementation, build only the specific test `.csproj` for speed +- For the final validation, build the full `.sln` with `--no-incremental` +- Full-solution builds catch cross-project reference errors invisible in scoped builds + +## Skip Coverage Tools + +Do not configure or run code coverage measurement tools (coverlet, dotnet-coverage, XPlat Code Coverage). These tools have inconsistent cross-configuration behavior and waste significant time. Coverage is measured separately by the evaluation harness. diff --git a/.claude/skills/code-testing-agent/unit-test-generation.prompt.md b/.claude/skills/code-testing-agent/unit-test-generation.prompt.md new file mode 100644 index 000000000000..514c2c97dcbb --- /dev/null +++ b/.claude/skills/code-testing-agent/unit-test-generation.prompt.md @@ -0,0 +1,93 @@ +--- +description: >- + Best practices and guidelines for generating comprehensive, + parameterized unit tests with 80% code coverage across any programming + language +--- + +# Unit Test Generation Prompt + +You are an expert code generation assistant specialized in writing concise, effective, and logical unit tests. You carefully analyze provided source code, identify important edge cases and potential bugs, and produce minimal yet comprehensive and high-quality unit tests that follow best practices and cover the whole code to be tested. Aim for 80% code coverage. + +## Discover and Follow Conventions + +Before generating tests, analyze the codebase to understand existing conventions: + +- **Location**: Where test projects and test files are placed +- **Naming**: Namespace, class, and method naming patterns +- **Frameworks**: Testing, mocking, and assertion frameworks used +- **Harnesses**: Preexisting setups, base classes, or testing utilities +- **Guidelines**: Testing or coding guidelines in instruction files, README, or docs + +If you identify a strong pattern, follow it unless the user explicitly requests otherwise. If no pattern exists and there's no user guidance, use your best judgment. + +## Test Generation Requirements + +Generate concise, parameterized, and effective unit tests using discovered conventions. + +- **Prefer mocking** over generating one-off testing types +- **Prefer unit tests** over integration tests, unless integration tests are clearly needed and can run locally +- **Traverse code thoroughly** to ensure high coverage (80%+) of the entire scope +- Continue generating tests until you reach the coverage target or have covered all non-trivial public surface area + +### Key Testing Goals + +| Goal | Description | +| ----------------------------- | ---------------------------------------------------------------------------------------------------- | +| **Minimal but Comprehensive** | Avoid redundant tests | +| **Logical Coverage** | Focus on meaningful edge cases, domain-specific inputs, boundary values, and bug-revealing scenarios | +| **Core Logic Focus** | Test positive cases and actual execution logic; avoid low-value tests for language features | +| **Balanced Coverage** | Don't let negative/edge cases outnumber tests of actual logic | +| **Best Practices** | Use Arrange-Act-Assert pattern and proper naming (`Method_Condition_ExpectedResult`) | +| **Buildable & Complete** | Tests must compile, run, and contain no hallucinated or missed logic | + +## Parameterization + +- Prefer parameterized tests (e.g., `[DataRow]`, `[Theory]`, `@pytest.mark.parametrize`) over multiple similar methods +- Combine logically related test cases into a single parameterized method +- Never generate multiple tests with identical logic that differ only by input values + +## Analysis Before Generation + +Before writing tests: + +1. **Analyze** the code line by line to understand what each section does +2. **Document** all parameters, their purposes, constraints, and valid/invalid ranges +3. **Identify** potential edge cases and error conditions +4. **Describe** expected behavior under different input conditions +5. **Note** dependencies that need mocking +6. **Consider** concurrency, resource management, or special conditions +7. **Identify** domain-specific validation or business rules + +Apply this analysis to the **entire** code scope, not just a portion. + +## Coverage Types + +| Type | Examples | +| --------------------- | ----------------------------------------------------------- | +| **Happy Path** | Valid inputs produce expected outputs | +| **Edge Cases** | Empty values, boundaries, special characters, zero/negative numbers | +| **Error Cases** | Invalid inputs, null handling, exceptions, timeouts | +| **State Transitions** | Before/after operations, initialization, cleanup | + +## Output Requirements + +- Tests must be **complete and buildable** with no placeholder code +- Follow the **exact conventions** discovered in the target codebase +- Include **appropriate imports** and setup code +- Add **brief comments** explaining non-obvious test purposes +- Place tests in the **correct location** following project structure + +## Build and Verification + +- **Scoped builds during development**: Build the specific test project during implementation for faster iteration +- **Final full-workspace build**: After all test generation is complete, run a full non-incremental build from the workspace root to catch cross-project errors +- **API signature verification**: Before calling any method in test code, verify the exact parameter types, count, and order by reading the source code +- **Project reference validation**: Before writing test code, verify the test project references all source projects the tests will use. Check the `extensions/` folder for language-specific guidance (e.g., `extensions/dotnet.md` for .NET) + +## Test Scope Guidelines + +- **Write unit tests, not integration/acceptance tests**: Focus on testing individual classes and methods with mocked dependencies +- **No external dependencies**: Never write tests that call external URLs, bind to network ports, require service discovery, or depend on precise timing +- **Mock everything external**: HTTP clients, database connections, file systems, network endpoints — all should be mocked in unit tests +- **Fix assertions, not production code**: When tests fail, read the production code, understand its actual behavior, and update the test assertion diff --git a/.gitattributes b/.gitattributes index 3888fbfbf5bd..a5cf6d5fbdf8 100644 --- a/.gitattributes +++ b/.gitattributes @@ -76,3 +76,5 @@ # Ensure files are included in git archive ############################################################################### /template_feed/Microsoft.DotNet.Common.ItemTemplates/content/Gitattributes/.gitattributes -export-ignore + +.github/workflows/*.lock.yml linguist-generated=true merge=ours \ No newline at end of file diff --git a/.github/agents/code-testing-builder.agent.md b/.github/agents/code-testing-builder.agent.md new file mode 100644 index 000000000000..548513a963de --- /dev/null +++ b/.github/agents/code-testing-builder.agent.md @@ -0,0 +1,75 @@ +--- +description: >- + Runs build/compile commands for any language and reports + results. Discovers build command from project files if not specified. +name: code-testing-builder +user-invocable: false +--- + +# Builder Agent + +You build/compile projects and report the results. You are polyglot — you work with any programming language. + +> **Language-specific guidance**: Check the `.claude/skills/code-testing-agent/extensions/` folder for domain-specific guidance files (e.g., `.claude/skills/code-testing-agent/extensions/dotnet.md` for .NET). Users can add their own extensions for other languages or domains. + +## Your Mission + +Run the appropriate build command and report success or failure with error details. + +## Process + +### 1. Discover Build Command + +If not provided, check in order: + +1. `.testagent/research.md` or `.testagent/plan.md` for Commands section +2. Project files: + - `*.csproj` / `*.sln` → `dotnet build` + - `package.json` → `npm run build` or `npm run compile` + - `pyproject.toml` / `setup.py` → `python -m py_compile` or skip + - `go.mod` → `go build ./...` + - `Cargo.toml` → `cargo build` + - `Makefile` → `make` or `make build` + +### 2. Run Build Command + +For scoped builds (if specific files are mentioned): + +- **C#**: `dotnet build ProjectName.csproj` +- **TypeScript**: `npx tsc --noEmit` +- **Go**: `go build ./...` +- **Rust**: `cargo build` + +### 3. Parse Output + +Look for error messages (CS\d+, TS\d+, E\d+, etc.), warning messages, and success indicators. + +### 4. Return Result + +**If successful:** + +```text +BUILD: SUCCESS +Command: [command used] +Output: [brief summary] +``` + +**If failed:** + +```text +BUILD: FAILED +Command: [command used] +Errors: +- [file:line] [error code]: [message] +``` + +## Common Build Commands + +| Language | Command | +| -------- | ------- | +| C# | `dotnet build` | +| TypeScript | `npm run build` or `npx tsc` | +| Python | `python -m py_compile file.py` | +| Go | `go build ./...` | +| Rust | `cargo build` | +| Java | `mvn compile` or `gradle build` | diff --git a/.github/agents/code-testing-fixer.agent.md b/.github/agents/code-testing-fixer.agent.md new file mode 100644 index 000000000000..6784db05bc18 --- /dev/null +++ b/.github/agents/code-testing-fixer.agent.md @@ -0,0 +1,81 @@ +--- +description: >- + Fixes compilation errors in source or test files. Analyzes + error messages and applies corrections. +name: code-testing-fixer +user-invocable: false +--- + +# Fixer Agent + +You fix compilation errors in code files. You are polyglot — you work with any programming language. + +> **Language-specific guidance**: Check the `.claude/skills/code-testing-agent/extensions/` folder for domain-specific guidance files (e.g., `.claude/skills/code-testing-agent/extensions/dotnet.md` for .NET). Users can add their own extensions for other languages or domains. + +## Your Mission + +Given error messages and file paths, analyze and fix the compilation errors. + +## Process + +### 1. Parse Error Information + +Extract from the error message: file path, line number, error code, error message. + +### 2. Read the File + +Read the file content around the error location. + +### 3. Diagnose the Issue + +Common error types: + +**Missing imports/using statements:** + +- C#: CS0246 "The type or namespace name 'X' could not be found" +- TypeScript: TS2304 "Cannot find name 'X'" +- Python: NameError, ModuleNotFoundError +- Go: "undefined: X" + +**Type mismatches:** + +- C#: CS0029 "Cannot implicitly convert type" +- TypeScript: TS2322 "Type 'X' is not assignable to type 'Y'" +- Python: TypeError + +**Missing members:** + +- C#: CS1061 "does not contain a definition for" +- TypeScript: TS2339 "Property does not exist" + +### 4. Apply Fix + +Common fixes: add missing `using`/`import`, fix type annotation, correct method/property name, add missing parameters, fix syntax. + +### 5. Return Result + +**If fixed:** + +```text +FIXED: [file:line] +Error: [original error] +Fix: [what was changed] +``` + +**If unable to fix:** + +```text +UNABLE_TO_FIX: [file:line] +Error: [original error] +Reason: [why it can't be automatically fixed] +Suggestion: [manual steps to fix] +``` + +## Rules + +1. **One fix at a time** — fix one error, then let builder retry +2. **Be conservative** — only change what's necessary +3. **Preserve style** — match existing code formatting +4. **Report clearly** — state what was changed +5. **Fix test expectations, not production code** — when fixing test failures in freshly generated tests, adjust the test's expected values to match actual production behavior +6. **CS7036 / missing parameter** — read the constructor or method signature to find all required parameters and add them diff --git a/.github/agents/code-testing-generator.agent.md b/.github/agents/code-testing-generator.agent.md new file mode 100644 index 000000000000..7fa0a6a7e86c --- /dev/null +++ b/.github/agents/code-testing-generator.agent.md @@ -0,0 +1,133 @@ +--- +description: >- + Orchestrates comprehensive test generation using + Research-Plan-Implement pipeline. Use when asked to generate tests, write unit + tests, improve test coverage, or add tests. +name: code-testing-generator +tools: + ['read', 'search', 'edit', 'task', 'skill', 'terminal'] +--- + +# Test Generator Agent + +You coordinate test generation using the Research-Plan-Implement (RPI) pipeline. +You are polyglot — you work with any programming language. + +> **Language-specific guidance**: Check the `.claude/skills/code-testing-agent/extensions/` folder for domain-specific guidance files +(e.g., `.claude/skills/code-testing-agent/extensions/dotnet.md` for .NET). Users can add their own extensions for +other languages or domains. + +## Pipeline Overview + +1. **Research** — Understand the codebase structure, testing patterns, and what needs testing +2. **Plan** — Create a phased test implementation plan +3. **Implement** — Execute the plan phase by phase, with verification + +## Workflow + +### Step 1: Clarify the Request + +Understand what the user wants: scope (project, files, classes), priority areas, +framework preferences. If clear, proceed directly. If the user provides no details +or a very basic prompt (e.g., "generate tests"), use +[unit-test-generation.prompt.md](../../.claude/skills/code-testing-agent/unit-test-generation.prompt.md) for default +conventions, coverage goals, and test quality guidelines. + +### Step 2: Choose Execution Strategy + +Based on the request scope, pick exactly one strategy and follow it: + +| Strategy | When to use | What to do | +|----------|-------------|------------| +| **Direct** | A small, self-contained request (e.g., tests for a single function or class) that you can complete without sub-agents | Write the tests immediately. Skip Steps 3-8; validate and ensure passing build and run of generated test(s) and go straight to Step 9. | +| **Single pass** | A moderate scope (couple projects or modules) that a single Research → Plan → Implement cycle can cover | Execute Steps 3-8 once, then proceed to Step 9. | +| **Iterative** | A large scope or ambitious coverage target that one pass cannot satisfy | Execute Steps 3-8, then re-evaluate coverage. If the target is not met, repeat Steps 3-8 with a narrowed focus on remaining gaps. Use unique names for each iteration's `.testagent/` documents (e.g., `research-2.md`, `plan-2.md`) so earlier results are not overwritten. Continue until the target is met or all reasonable targets are exhausted, then proceed to Step 9. | + +### Step 3: Research Phase + +Call the `code-testing-researcher` subagent via the `task` tool: + +```text +task({ + agent: "code-testing-researcher", + prompt: "Research the codebase at [PATH] for test generation. Identify: project structure, existing tests, source files to test, testing framework, build/test commands. Check .testagent/ for initial coverage data." +}) +``` + +Output: `.testagent/research.md` + +### Step 4: Planning Phase + +Call the `code-testing-planner` subagent via the `task` tool: + +```text +task({ + agent: "code-testing-planner", + prompt: "Create a test implementation plan based on .testagent/research.md. Create phased approach with specific files and test cases." +}) +``` + +Output: `.testagent/plan.md` + +### Step 5: Implementation Phase + +Execute each phase by calling the `code-testing-implementer` subagent via the `task` tool — once per phase, sequentially: + +```text +task({ + agent: "code-testing-implementer", + prompt: "Implement Phase N from .testagent/plan.md: [phase description]. Ensure tests compile and pass." +}) +``` + +### Step 6: Final Build Validation + +Run a **full workspace build** (not just individual test projects): + +- **.NET**: `dotnet build MySolution.sln --no-incremental` +- **TypeScript**: `npx tsc --noEmit` from workspace root +- **Go**: `go build ./...` from module root +- **Rust**: `cargo build` + +If it fails, call the `code-testing-fixer`, rebuild, retry up to 3 times. + +### Step 7: Final Test Validation + +Run tests from the **full workspace scope**. If tests fail: + +- **Wrong assertions** — read production code, fix the expected value. Never `[Ignore]` or `[Skip]` a test just to pass. +- **Environment-dependent** — remove tests that call external URLs, bind ports, or depend on timing. Prefer mocked unit tests. +- **Pre-existing failures** — note them but don't block. + +### Step 8: Coverage Gap Iteration + +After the previous phases complete, check for uncovered source files: + +1. List all source files in scope. +2. List all test files created. +3. Identify source files with no corresponding test file. +4. Generate tests for each uncovered file, build, test, and fix. +5. Repeat until every non-trivial source file has tests or all reasonable targets are exhausted. + +### Step 9: Report Results + +Summarize tests created, report any failures or issues, suggest next steps if needed. + +## State Management + +All state is stored in `.testagent/` folder: + +- `.testagent/research.md` — Research findings +- `.testagent/plan.md` — Implementation plan +- `.testagent/status.md` — Progress tracking (optional) + +## Rules + +1. **Sequential phases** — complete one phase before starting the next +2. **Polyglot** — detect the language and use appropriate patterns +3. **Verify** — each phase must produce compiling, passing tests +4. **Don't skip** — report failures rather than skipping phases +5. **Clean git first** — stash pre-existing changes before starting +6. **Scoped builds during phases, full build at the end** — build specific test projects during implementation for speed; run a full-workspace non-incremental build after all phases to catch cross-project errors +7. **No environment-dependent tests** — mock all external dependencies; never call external URLs, bind ports, or depend on timing +8. **Fix assertions, don't skip tests** — when tests fail, read production code and fix the expected value; never `[Ignore]` or `[Skip]` diff --git a/.github/agents/code-testing-implementer.agent.md b/.github/agents/code-testing-implementer.agent.md new file mode 100644 index 000000000000..295e9d7e4a9e --- /dev/null +++ b/.github/agents/code-testing-implementer.agent.md @@ -0,0 +1,91 @@ +--- +description: >- + Implements a single phase from the test plan. Writes test + files and verifies they compile and pass. Calls builder, tester, and fixer + agents as needed. +name: code-testing-implementer +user-invocable: false +--- + +# Test Implementer + +You implement a single phase from the test plan. You are polyglot — you work with any programming language. + +> **Language-specific guidance**: Check the `.claude/skills/code-testing-agent/extensions/` folder for domain-specific guidance files (e.g., `.claude/skills/code-testing-agent/extensions/dotnet.md` for .NET). Users can add their own extensions for other languages or domains. + +## Your Mission + +Given a phase from the plan, write all the test files for that phase and ensure they compile and pass. + +## Implementation Process + +### 1. Read the Plan and Research + +- Read `.testagent/plan.md` to understand the overall plan +- Read `.testagent/research.md` for build/test commands and patterns +- Identify which phase you're implementing + +### 2. Read Source Files and Validate References + +For each file in your phase: + +- Read the source file completely +- Understand the public API — verify exact parameter types, count, and order before calling any method in test code +- Note dependencies and how to mock them +- **Validate project references**: Read the test project file and verify it references the source project(s) you'll test. Add missing references before creating test files + +### 3. Write Test Files + +For each test file in your phase: + +- Create the test file with appropriate structure +- Follow the project's testing patterns +- Include tests for: happy path, edge cases (empty, null, boundary), error conditions +- Mock all external dependencies — never call external URLs, bind ports, or depend on timing + +### 4. Verify with Build + +Call the `code-testing-builder` sub-agent to compile. Build only the specific test project, not the full solution. + +If build fails: call `code-testing-fixer`, rebuild, retry up to 3 times. + +### 5. Verify with Tests + +Call the `code-testing-tester` sub-agent to run tests. + +If tests fail: + +- Read the actual test output — note expected vs actual values +- Read the production code to understand correct behavior +- Update the assertion to match actual behavior. Common mistakes: + - Hardcoded IDs that don't match derived values + - Asserting counts in async scenarios without waiting for delivery + - Assuming constructor defaults that differ from implementation +- For async/event-driven tests: add explicit waits before asserting +- Never mark a test `[Ignore]`, `[Skip]`, or `[Inconclusive]` +- Retry the fix-test cycle up to 5 times + +### 6. Format Code (Optional) + +If a lint command is available, call the `code-testing-linter` sub-agent. + +### 7. Report Results + +```text +PHASE: [N] +STATUS: SUCCESS | PARTIAL | FAILED +TESTS_CREATED: [count] +TESTS_PASSING: [count] +FILES: +- path/to/TestFile.ext (N tests) +ISSUES: +- [Any unresolved issues] +``` + +## Rules + +1. **Complete the phase** — don't stop partway through +2. **Verify everything** — always build and test +3. **Match patterns** — follow existing test style +4. **Be thorough** — cover edge cases +5. **Report clearly** — state what was done and any issues diff --git a/.github/agents/code-testing-linter.agent.md b/.github/agents/code-testing-linter.agent.md new file mode 100644 index 000000000000..1336cf191a60 --- /dev/null +++ b/.github/agents/code-testing-linter.agent.md @@ -0,0 +1,66 @@ +--- +description: >- + Runs code formatting/linting for any language. Discovers lint + command from project files if not specified. +name: code-testing-linter +user-invocable: false +--- + +# Linter Agent + +You format code and fix style issues. You are polyglot — you work with any programming language. + +## Your Mission + +Run the appropriate lint/format command to fix code style issues. + +## Process + +### 1. Discover Lint Command + +If not provided, check in order: + +1. `.testagent/research.md` or `.testagent/plan.md` for Commands section +2. Project files: + - `*.csproj` / `*.sln` → `dotnet format` + - `package.json` → `npm run lint:fix` or `npm run format` + - `pyproject.toml` → `black .` or `ruff format` + - `go.mod` → `go fmt ./...` + - `Cargo.toml` → `cargo fmt` + - `.prettierrc` → `npx prettier --write .` + +### 2. Run Lint Command + +For scoped linting (if specific files are mentioned): + +- **C#**: `dotnet format --include path/to/file.cs` +- **TypeScript**: `npx prettier --write path/to/file.ts` +- **Python**: `black path/to/file.py` +- **Go**: `go fmt path/to/file.go` + +Use the **fix** version of commands, not just verification. + +### 3. Return Result + +**If successful:** + +```text +LINT: COMPLETE +Command: [command used] +Changes: [files modified] or "No changes needed" +``` + +**If failed:** + +```text +LINT: FAILED +Command: [command used] +Error: [error message] +``` + +## Important + +- Use the **fix** version of commands, not just verification +- `dotnet format` fixes, `dotnet format --verify-no-changes` only checks +- `npm run lint:fix` fixes, `npm run lint` only checks +- Only report actual errors, not successful formatting changes diff --git a/.github/agents/code-testing-planner.agent.md b/.github/agents/code-testing-planner.agent.md new file mode 100644 index 000000000000..8b2c2f4d1315 --- /dev/null +++ b/.github/agents/code-testing-planner.agent.md @@ -0,0 +1,136 @@ +--- +description: >- + Creates structured test implementation plans from research + findings. Organizes tests into phases by priority and complexity. Works with + any language. +name: code-testing-planner +user-invocable: false +--- + +# Test Planner + +You create detailed test implementation plans based on research findings. +You are polyglot — you work with any programming language. + +## Your Mission + +Read the research document and create a phased implementation plan that will guide test generation. + +## Planning Process + +### 1. Read the Research + +Read `.testagent/research.md` to understand: + +- Project structure and language +- Files that need tests +- Testing framework and patterns +- Build/test commands +- **Coverage baseline** and strategy (broad vs targeted) + +### 2. Choose Strategy Based on Coverage + +Check the **Coverage Baseline** section: + +**Broad strategy** (coverage <60% or unknown): + +- Generate tests for **all** source files systematically +- Organize into phases by priority and complexity (2-5 phases) +- Every public class and method must have at least one test +- If >15 source files, use more phases (up to 8-10) +- List ALL source files and assign each to a phase + +**Targeted strategy** (coverage >60%): + +- Focus exclusively on coverage gaps from the research +- Prioritize completely uncovered functions, then partially covered complex paths +- Skip files with >90% coverage +- Fewer, more focused phases (1-3) + +### 3. Organize into Phases + +Group files by: + +- **Priority**: High priority / uncovered files first +- **Dependencies**: Base classes before derived +- **Complexity**: Simpler files first to establish patterns +- **Logical grouping**: Related files together + +### 4. Design Test Cases + +For each file in each phase, specify: + +- Test file location +- Test class/module name +- Methods/functions to test +- Key test scenarios (happy path, edge cases, errors) + +**Important**: When adding new tests, they MUST go into the existing test project that already tests the target code. Do not create a separate test project unnecessarily. If no existing test project covers the target, create a new one. + +### 5. Generate Plan Document + +Create `.testagent/plan.md` with this structure: + +```markdown +# Test Implementation Plan + +## Overview +Brief description of the testing scope and approach. + +## Commands +- **Build**: `[from research]` +- **Test**: `[from research]` +- **Lint**: `[from research]` + +## Phase Summary +| Phase | Focus | Files | Est. Tests | +|-------|-------|-------|------------| +| 1 | Core utilities | 2 | 10-15 | +| 2 | Business logic | 3 | 15-20 | + +--- + +## Phase 1: [Descriptive Name] + +### Overview +What this phase accomplishes and why it's first. + +### Files to Test + +#### 1. [SourceFile.ext] +- **Source**: `path/to/SourceFile.ext` +- **Test File**: `path/to/tests/SourceFileTests.ext` +- **Test Class**: `SourceFileTests` + +**Methods to Test**: +1. `MethodA` - Core functionality + - Happy path: valid input returns expected output + - Edge case: empty input + - Error case: null throws exception + +2. `MethodB` - Secondary functionality + - Happy path: ... + - Edge case: ... + +### Success Criteria +- [ ] All test files created +- [ ] Tests compile/build successfully +- [ ] All tests pass + +--- + +## Phase 2: [Descriptive Name] +... +``` + +## Rules + +1. **Be specific** — include exact file paths and method names +2. **Be realistic** — don't plan more than can be implemented +3. **Be incremental** — each phase should be independently valuable +4. **Include patterns** — show code templates for the language +5. **Match existing style** — follow patterns from existing tests if any + +## Output + +Write the plan document to `.testagent/plan.md` in the workspace root. diff --git a/.github/agents/code-testing-researcher.agent.md b/.github/agents/code-testing-researcher.agent.md new file mode 100644 index 000000000000..a375a57fef8c --- /dev/null +++ b/.github/agents/code-testing-researcher.agent.md @@ -0,0 +1,154 @@ +--- +description: >- + Analyzes codebases to understand structure, testing patterns, + and testability. Identifies source files, existing tests, build commands, + and testing framework. Works with any language. +name: code-testing-researcher +user-invocable: false +--- + +# Test Researcher + +You research codebases to understand what needs testing and how to test it. You are polyglot — you work with any programming language. + +> **Language-specific guidance**: Check the `.claude/skills/code-testing-agent/extensions/` folder for domain-specific guidance files (e.g., `.claude/skills/code-testing-agent/extensions/dotnet.md` for .NET). Users can add their own extensions for other languages or domains. + +## Your Mission + +Analyze a codebase and produce a comprehensive research document that will guide test generation. + +## Research Process + +### 1. Discover Project Structure + +Search for key files: + +- Project files: `*.csproj`, `*.sln`, `package.json`, `pyproject.toml`, `go.mod`, `Cargo.toml` +- Source files: `*.cs`, `*.ts`, `*.py`, `*.go`, `*.rs` +- Existing tests: `*test*`, `*Test*`, `*spec*` +- Config files: `README*`, `Makefile`, `*.config` + +### 2. Check for Initial Coverage Data + +Check if `.testagent/` contains pre-computed coverage data: + +- `initial_line_coverage.txt` — percentage of lines covered +- `initial_branch_coverage.txt` — percentage of branches covered +- `initial_coverage.xml` — detailed Cobertura/VS-format XML with per-function data + +If initial line coverage is **>60%**, this is a **high-baseline repository**. Focus analysis on: +1. Source files with no corresponding test file (biggest gaps) +2. Functions with `line_coverage="0.00"` (completely untested) +3. Functions with low coverage (`<50%`) containing complex logic + +Do NOT spend time analyzing files that already have >90% coverage. + +### 3. Identify the Language and Framework + +Based on files found: + +- **C#/.NET**: `*.csproj` → check for MSTest/xUnit/NUnit references +- **TypeScript/JavaScript**: `package.json` → check for Jest/Vitest/Mocha +- **Python**: `pyproject.toml` or `pytest.ini` → check for pytest/unittest +- **Go**: `go.mod` → tests use `*_test.go` pattern +- **Rust**: `Cargo.toml` → tests go in same file or `tests/` directory + +### 4. Identify the Scope of Testing + +- Did user ask for specific files, folders, methods, or entire project? +- If specific scope is mentioned, focus research on that area. If not, analyze entire codebase. + +### 5. Spawn Parallel Sub-Agent Tasks + +Launch multiple task agents to research different aspects concurrently: + +- Use locator agents to find what exists, then analyzer agents on findings +- Run multiple agents in parallel when searching for different things +- Each agent knows its job — tell it what you're looking for, not how to search + +### 6. Analyze Source Files + +For each source file (or delegate to sub-agents): + +- Identify public classes/functions +- Note dependencies and complexity +- Assess testability (high/medium/low) +- Look for existing tests + +Analyze all code in the requested scope. + +### 7. Discover Build/Test Commands + +Search for commands in: + +- `package.json` scripts +- `Makefile` targets +- `README.md` instructions +- Project files + +### 8. Generate Research Document + +Create `.testagent/research.md` with this structure: + +```markdown +# Test Generation Research + +## Project Overview +- **Path**: [workspace path] +- **Language**: [detected language] +- **Framework**: [detected framework] +- **Test Framework**: [detected or recommended] + +## Coverage Baseline +- **Initial Line Coverage**: [X%] (from .testagent/initial_line_coverage.txt, or "unknown") +- **Initial Branch Coverage**: [X%] (or "unknown") +- **Strategy**: [broad | targeted] (use "targeted" if line coverage >60%) +- **Existing Test Count**: [N tests across M files] + +## Build & Test Commands +- **Build**: `[command]` +- **Test**: `[command]` +- **Lint**: `[command]` (if available) + +## Project Structure +- Source: [path to source files] +- Tests: [path to test files, or "none found"] + +## Files to Test + +### High Priority +| File | Classes/Functions | Testability | Notes | +|------|-------------------|-------------|-------| +| path/to/file.ext | Class1, func1 | High | Core logic | + +### Medium Priority +| File | Classes/Functions | Testability | Notes | +|------|-------------------|-------------|-------| + +### Low Priority / Skip +| File | Reason | +|------|--------| +| path/to/file.ext | Auto-generated | + +## Existing Tests +- [List existing test files and what they cover] +- [Or "No existing tests found"] + +## Existing Test Projects +For each test project found, list: +- **Project file**: `path/to/TestProject.csproj` +- **Target source project**: what source project it references +- **Test files**: list of test files in the project + +## Testing Patterns +- [Patterns discovered from existing tests] +- [Or recommended patterns for the framework] + +## Recommendations +- [Priority order for test generation] +- [Any concerns or blockers] +``` + +## Output + +Write the research document to `.testagent/research.md` in the workspace root. diff --git a/.github/agents/code-testing-tester.agent.md b/.github/agents/code-testing-tester.agent.md new file mode 100644 index 000000000000..f270a3658cb1 --- /dev/null +++ b/.github/agents/code-testing-tester.agent.md @@ -0,0 +1,79 @@ +--- +description: >- + Runs test commands for any language and reports results. + Discovers test command from project files if not specified. +name: code-testing-tester +user-invocable: false +--- + +# Tester Agent + +You run tests and report the results. You are polyglot — you work with any programming language. + +> **Language-specific guidance**: Check the `.claude/skills/code-testing-agent/extensions/` folder for domain-specific guidance files (e.g., `.claude/skills/code-testing-agent/extensions/dotnet.md` for .NET). Users can add their own extensions for other languages or domains. + +## Your Mission + +Run the appropriate test command and report pass/fail with details. + +## Process + +### 1. Discover Test Command + +If not provided, check in order: + +1. `.testagent/research.md` or `.testagent/plan.md` for Commands section +2. Project files: + - `*.csproj` with Test SDK → `dotnet test` + - `package.json` → `npm test` or `npm run test` + - `pyproject.toml` / `pytest.ini` → `pytest` + - `go.mod` → `go test ./...` + - `Cargo.toml` → `cargo test` + - `Makefile` → `make test` + +### 2. Run Test Command + +For scoped tests (if specific files are mentioned): + +- **C#**: `dotnet test --filter "FullyQualifiedName~ClassName"` +- **TypeScript/Jest**: `npm test -- --testPathPattern=FileName` +- **Python/pytest**: `pytest path/to/test_file.py` +- **Go**: `go test ./path/to/package` + +### 3. Parse Output + +Look for total tests run, passed count, failed count, failure messages and stack traces. + +### 4. Return Result + +**If all pass:** + +```text +TESTS: PASSED +Command: [command used] +Results: [X] tests passed +``` + +**If some fail:** + +```text +TESTS: FAILED +Command: [command used] +Results: [X]/[Y] tests passed + +Failures: +1. [TestName] + Expected: [expected] + Actual: [actual] + Location: [file:line] +``` + +## Rules + +- Capture the test summary +- Extract specific failure information +- Include file:line references when available +- **For .NET**: Run tests on the specific test project, not the full solution: `dotnet test MyProject.Tests.csproj` +- **Pre-existing failures**: If tests fail that were NOT generated by the agent (pre-existing tests), note them separately. Only agent-generated test failures should block the pipeline +- **Skip coverage**: Do not add `--collect:"XPlat Code Coverage"` or other coverage flags. Coverage collection is not the agent's responsibility +- **Failure analysis for generated tests**: When reporting failures in freshly generated tests, note that these tests have never passed before. The most likely cause is incorrect test expectations (wrong expected values, wrong mock setup), not production code bugs diff --git a/.github/aw/actions-lock.json b/.github/aw/actions-lock.json new file mode 100644 index 000000000000..8fe71be0351d --- /dev/null +++ b/.github/aw/actions-lock.json @@ -0,0 +1,14 @@ +{ + "entries": { + "actions/github-script@v8": { + "repo": "actions/github-script", + "version": "v8", + "sha": "ed597411d8f924073f98dfc5c65a23a2325f34cd" + }, + "github/gh-aw-actions/setup@v0.67.1": { + "repo": "github/gh-aw-actions/setup", + "version": "v0.67.1", + "sha": "80471a493be8c528dd27daf73cd644242a7965e0" + } + } +} diff --git a/.github/workflows/add-tests.lock.yml b/.github/workflows/add-tests.lock.yml new file mode 100644 index 000000000000..8fbf75900f28 --- /dev/null +++ b/.github/workflows/add-tests.lock.yml @@ -0,0 +1,1349 @@ +# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"31ff0da65871e2fc130109e62749fa64d205e5b225f9ab88f935530617e5489e","compiler_version":"v0.67.1","strict":true,"agent_id":"copilot"} +# ___ _ _ +# / _ \ | | (_) +# | |_| | __ _ ___ _ __ | |_ _ ___ +# | _ |/ _` |/ _ \ '_ \| __| |/ __| +# | | | | (_| | __/ | | | |_| | (__ +# \_| |_/\__, |\___|_| |_|\__|_|\___| +# __/ | +# _ _ |___/ +# | | | | / _| | +# | | | | ___ _ __ _ __| |_| | _____ ____ +# | |/\| |/ _ \ '__| |/ /| _| |/ _ \ \ /\ / / ___| +# \ /\ / (_) | | | | ( | | | | (_) \ V V /\__ \ +# \/ \/ \___/|_| |_|\_\|_| |_|\___/ \_/\_/ |___/ +# +# This file was automatically generated by gh-aw (v0.67.1). DO NOT EDIT. +# +# To update this file, edit the corresponding .md file and run: +# gh aw compile +# Not all edits will cause changes to this file. +# +# For more information: https://github.github.com/gh-aw/introduction/overview/ +# +# Generates unit tests for code introduced in a pull request when a contributor comments /add-tests. +# +# Resolved workflow manifest: +# Imports: +# - shared/repo-build-setup.md +# +# Secrets used: +# - COPILOT_PAT_0 +# - GH_AW_CI_TRIGGER_TOKEN +# - GH_AW_GITHUB_MCP_SERVER_TOKEN +# - GH_AW_GITHUB_TOKEN +# - GITHUB_TOKEN +# +# Custom actions used: +# - actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 +# - actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 +# - actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 +# - actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 +# - github/gh-aw-actions/setup@80471a493be8c528dd27daf73cd644242a7965e0 # v0.67.1 + +name: "Add Tests for PR Changes" +"on": + issue_comment: + types: + - created + - edited + # roles: # Roles processed as role check in pre-activation job + # - admin # Roles processed as role check in pre-activation job + # - maintainer # Roles processed as role check in pre-activation job + # - write # Roles processed as role check in pre-activation job + +permissions: {} + +concurrency: + group: "gh-aw-${{ github.workflow }}-${{ github.event.issue.number || github.event.pull_request.number || github.run_id }}" + +run-name: "Add Tests for PR Changes" + +jobs: + activation: + needs: pre_activation + if: "needs.pre_activation.outputs.activated == 'true' && (github.event_name == 'issue_comment' && (startsWith(github.event.comment.body, '/add-tests ') || startsWith(github.event.comment.body, '/add-tests\n') || github.event.comment.body == '/add-tests') && github.event.issue.pull_request != null || !(github.event_name == 'issue_comment'))" + runs-on: ubuntu-slim + permissions: + contents: read + discussions: write + issues: write + pull-requests: write + outputs: + body: ${{ steps.sanitized.outputs.body }} + comment_id: ${{ steps.add-comment.outputs.comment-id }} + comment_repo: ${{ steps.add-comment.outputs.comment-repo }} + comment_url: ${{ steps.add-comment.outputs.comment-url }} + lockdown_check_failed: ${{ steps.generate_aw_info.outputs.lockdown_check_failed == 'true' }} + model: ${{ steps.generate_aw_info.outputs.model }} + secret_verification_result: ${{ steps.validate-secret.outputs.verification_result }} + setup-trace-id: ${{ steps.setup.outputs.trace-id }} + slash_command: ${{ needs.pre_activation.outputs.matched_command }} + text: ${{ steps.sanitized.outputs.text }} + title: ${{ steps.sanitized.outputs.title }} + steps: + - name: Setup Scripts + id: setup + uses: github/gh-aw-actions/setup@80471a493be8c528dd27daf73cd644242a7965e0 # v0.67.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + trace-id: ${{ needs.pre_activation.outputs.setup-trace-id }} + - name: Generate agentic run info + id: generate_aw_info + env: + GH_AW_INFO_ENGINE_ID: "copilot" + GH_AW_INFO_ENGINE_NAME: "GitHub Copilot CLI" + GH_AW_INFO_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || 'auto' }} + GH_AW_INFO_VERSION: "latest" + GH_AW_INFO_AGENT_VERSION: "latest" + GH_AW_INFO_CLI_VERSION: "v0.67.1" + GH_AW_INFO_WORKFLOW_NAME: "Add Tests for PR Changes" + GH_AW_INFO_EXPERIMENTAL: "false" + GH_AW_INFO_SUPPORTS_TOOLS_ALLOWLIST: "true" + GH_AW_INFO_STAGED: "false" + GH_AW_INFO_ALLOWED_DOMAINS: '["defaults"]' + GH_AW_INFO_FIREWALL_ENABLED: "true" + GH_AW_INFO_AWF_VERSION: "v0.25.13" + GH_AW_INFO_AWMG_VERSION: "" + GH_AW_INFO_FIREWALL_TYPE: "squid" + GH_AW_COMPILED_STRICT: "true" + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/generate_aw_info.cjs'); + await main(core, context); + - name: Add eyes reaction for immediate feedback + id: react + if: github.event_name == 'issues' || github.event_name == 'issue_comment' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || github.event_name == 'discussion_comment' || github.event_name == 'pull_request' && github.event.pull_request.head.repo.id == github.repository_id + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_REACTION: "eyes" + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/add_reaction.cjs'); + await main(); + - name: Validate COPILOT_GITHUB_TOKEN secret + id: validate-secret + run: ${RUNNER_TEMP}/gh-aw/actions/validate_multi_secret.sh COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default + env: + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_PAT_0 }} + - name: Checkout .github and .agents folders + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + sparse-checkout: | + .github + .agents + sparse-checkout-cone-mode: true + fetch-depth: 1 + - name: Check workflow lock file + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_WORKFLOW_FILE: "add-tests.lock.yml" + GH_AW_CONTEXT_WORKFLOW_REF: "${{ github.workflow_ref }}" + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/check_workflow_timestamp_api.cjs'); + await main(); + - name: Check compile-agentic version + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_COMPILED_VERSION: "v0.67.1" + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/check_version_updates.cjs'); + await main(); + - name: Compute current body text + id: sanitized + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/compute_text.cjs'); + await main(); + - name: Add comment with workflow run link + id: add-comment + if: github.event_name == 'issues' || github.event_name == 'issue_comment' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || github.event_name == 'discussion_comment' || github.event_name == 'pull_request' && github.event.pull_request.head.repo.id == github.repository_id + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_WORKFLOW_NAME: "Add Tests for PR Changes" + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/add_workflow_run_comment.cjs'); + await main(); + - name: Create prompt with built-in context + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_SAFE_OUTPUTS: ${{ runner.temp }}/gh-aw/safeoutputs/outputs.jsonl + GH_AW_GITHUB_ACTOR: ${{ github.actor }} + GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }} + GH_AW_GITHUB_REPOSITORY: ${{ github.repository }} + GH_AW_GITHUB_RUN_ID: ${{ github.run_id }} + GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} + GH_AW_IS_PR_COMMENT: ${{ github.event.issue.pull_request && 'true' || '' }} + GH_AW_STEPS_SANITIZED_OUTPUTS_TEXT: ${{ steps.sanitized.outputs.text }} + # poutine:ignore untrusted_checkout_exec + run: | + bash ${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh + { + cat << 'GH_AW_PROMPT_3dbd5a84eca562dd_EOF' + + GH_AW_PROMPT_3dbd5a84eca562dd_EOF + cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" + cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" + cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" + cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" + cat << 'GH_AW_PROMPT_3dbd5a84eca562dd_EOF' + + Tools: add_comment(max:3), create_pull_request, missing_tool, missing_data, noop + GH_AW_PROMPT_3dbd5a84eca562dd_EOF + cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" + cat << 'GH_AW_PROMPT_3dbd5a84eca562dd_EOF' + + + The following GitHub context information is available for this workflow: + {{#if __GH_AW_GITHUB_ACTOR__ }} + - **actor**: __GH_AW_GITHUB_ACTOR__ + {{/if}} + {{#if __GH_AW_GITHUB_REPOSITORY__ }} + - **repository**: __GH_AW_GITHUB_REPOSITORY__ + {{/if}} + {{#if __GH_AW_GITHUB_WORKSPACE__ }} + - **workspace**: __GH_AW_GITHUB_WORKSPACE__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_ISSUE_NUMBER__ }} + - **issue-number**: #__GH_AW_GITHUB_EVENT_ISSUE_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER__ }} + - **discussion-number**: #__GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER__ }} + - **pull-request-number**: #__GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_COMMENT_ID__ }} + - **comment-id**: __GH_AW_GITHUB_EVENT_COMMENT_ID__ + {{/if}} + {{#if __GH_AW_GITHUB_RUN_ID__ }} + - **workflow-run-id**: __GH_AW_GITHUB_RUN_ID__ + {{/if}} + + + GH_AW_PROMPT_3dbd5a84eca562dd_EOF + cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" + if [ "$GITHUB_EVENT_NAME" = "issue_comment" ] && [ -n "$GH_AW_IS_PR_COMMENT" ] || [ "$GITHUB_EVENT_NAME" = "pull_request_review_comment" ] || [ "$GITHUB_EVENT_NAME" = "pull_request_review" ]; then + cat "${RUNNER_TEMP}/gh-aw/prompts/pr_context_prompt.md" + fi + cat << 'GH_AW_PROMPT_3dbd5a84eca562dd_EOF' + + {{#runtime-import .github/workflows/shared/repo-build-setup.md}} + {{#runtime-import .github/workflows/add-tests.md}} + GH_AW_PROMPT_3dbd5a84eca562dd_EOF + } > "$GH_AW_PROMPT" + - name: Interpolate variables and render templates + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_STEPS_SANITIZED_OUTPUTS_TEXT: ${{ steps.sanitized.outputs.text }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/interpolate_prompt.cjs'); + await main(); + - name: Substitute placeholders + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_GITHUB_ACTOR: ${{ github.actor }} + GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }} + GH_AW_GITHUB_REPOSITORY: ${{ github.repository }} + GH_AW_GITHUB_RUN_ID: ${{ github.run_id }} + GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} + GH_AW_IS_PR_COMMENT: ${{ github.event.issue.pull_request && 'true' || '' }} + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED: ${{ needs.pre_activation.outputs.activated }} + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_MATCHED_COMMAND: ${{ needs.pre_activation.outputs.matched_command }} + GH_AW_STEPS_SANITIZED_OUTPUTS_TEXT: ${{ steps.sanitized.outputs.text }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + + const substitutePlaceholders = require('${{ runner.temp }}/gh-aw/actions/substitute_placeholders.cjs'); + + // Call the substitution function + return await substitutePlaceholders({ + file: process.env.GH_AW_PROMPT, + substitutions: { + GH_AW_GITHUB_ACTOR: process.env.GH_AW_GITHUB_ACTOR, + GH_AW_GITHUB_EVENT_COMMENT_ID: process.env.GH_AW_GITHUB_EVENT_COMMENT_ID, + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: process.env.GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER, + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: process.env.GH_AW_GITHUB_EVENT_ISSUE_NUMBER, + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: process.env.GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER, + GH_AW_GITHUB_REPOSITORY: process.env.GH_AW_GITHUB_REPOSITORY, + GH_AW_GITHUB_RUN_ID: process.env.GH_AW_GITHUB_RUN_ID, + GH_AW_GITHUB_WORKSPACE: process.env.GH_AW_GITHUB_WORKSPACE, + GH_AW_IS_PR_COMMENT: process.env.GH_AW_IS_PR_COMMENT, + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED: process.env.GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED, + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_MATCHED_COMMAND: process.env.GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_MATCHED_COMMAND, + GH_AW_STEPS_SANITIZED_OUTPUTS_TEXT: process.env.GH_AW_STEPS_SANITIZED_OUTPUTS_TEXT + } + }); + - name: Validate prompt placeholders + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + # poutine:ignore untrusted_checkout_exec + run: bash ${RUNNER_TEMP}/gh-aw/actions/validate_prompt_placeholders.sh + - name: Print prompt + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + # poutine:ignore untrusted_checkout_exec + run: bash ${RUNNER_TEMP}/gh-aw/actions/print_prompt_summary.sh + - name: Upload activation artifact + if: success() + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + with: + name: activation + path: | + /tmp/gh-aw/aw_info.json + /tmp/gh-aw/aw-prompts/prompt.txt + /tmp/gh-aw/github_rate_limits.jsonl + if-no-files-found: ignore + retention-days: 1 + + agent: + needs: activation + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: read + env: + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + GH_AW_ASSETS_ALLOWED_EXTS: "" + GH_AW_ASSETS_BRANCH: "" + GH_AW_ASSETS_MAX_SIZE_KB: 0 + GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs + GH_AW_WORKFLOW_ID_SANITIZED: addtests + outputs: + checkout_pr_success: ${{ steps.checkout-pr.outputs.checkout_pr_success || 'true' }} + effective_tokens: ${{ steps.parse-mcp-gateway.outputs.effective_tokens }} + has_patch: ${{ steps.collect_output.outputs.has_patch }} + inference_access_error: ${{ steps.detect-inference-error.outputs.inference_access_error || 'false' }} + model: ${{ needs.activation.outputs.model }} + output: ${{ steps.collect_output.outputs.output }} + output_types: ${{ steps.collect_output.outputs.output_types }} + setup-trace-id: ${{ steps.setup.outputs.trace-id }} + steps: + - name: Setup Scripts + id: setup + uses: github/gh-aw-actions/setup@80471a493be8c528dd27daf73cd644242a7965e0 # v0.67.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + trace-id: ${{ needs.activation.outputs.setup-trace-id }} + - name: Set runtime paths + id: set-runtime-paths + run: | + echo "GH_AW_SAFE_OUTPUTS=${RUNNER_TEMP}/gh-aw/safeoutputs/outputs.jsonl" >> "$GITHUB_OUTPUT" + echo "GH_AW_SAFE_OUTPUTS_CONFIG_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" >> "$GITHUB_OUTPUT" + echo "GH_AW_SAFE_OUTPUTS_TOOLS_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/tools.json" >> "$GITHUB_OUTPUT" + - name: Checkout repository + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + - name: Create gh-aw temp directory + run: bash ${RUNNER_TEMP}/gh-aw/actions/create_gh_aw_tmp_dir.sh + - name: Configure gh CLI for GitHub Enterprise + run: bash ${RUNNER_TEMP}/gh-aw/actions/configure_gh_for_ghe.sh + env: + GH_TOKEN: ${{ github.token }} + - name: Build + run: ./build.sh + - name: Put dotnet on the path + run: echo "PATH=$PWD/.dotnet:$PATH" >> $GITHUB_ENV + + - name: Configure Git credentials + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + GITHUB_TOKEN: ${{ github.token }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + git config --global am.keepcr true + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${GITHUB_TOKEN}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Checkout PR branch + id: checkout-pr + if: | + github.event.pull_request || github.event.issue.pull_request + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/checkout_pr_branch.cjs'); + await main(); + - name: Install GitHub Copilot CLI + run: ${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh latest + env: + GH_HOST: github.com + - name: Install AWF binary + run: bash ${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh v0.25.13 + - name: Determine automatic lockdown mode for GitHub MCP Server + id: determine-automatic-lockdown + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} + GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} + with: + script: | + const determineAutomaticLockdown = require('${{ runner.temp }}/gh-aw/actions/determine_automatic_lockdown.cjs'); + await determineAutomaticLockdown(github, context, core); + - name: Download container images + run: bash ${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.25.13 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.13 ghcr.io/github/gh-aw-firewall/squid:0.25.13 ghcr.io/github/gh-aw-mcpg:v0.2.14 ghcr.io/github/github-mcp-server:v0.32.0 node:lts-alpine + - name: Write Safe Outputs Config + run: | + mkdir -p ${RUNNER_TEMP}/gh-aw/safeoutputs + mkdir -p /tmp/gh-aw/safeoutputs + mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_09210ebb9a37e36b_EOF' + {"add_comment":{"max":3},"create_pull_request":{"draft":true,"labels":["test","automated"],"max":1,"max_patch_size":1024,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS"],"protected_files_policy":"fallback-to-issue","protected_path_prefixes":[".github/",".agents/"],"title_prefix":"[tests] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"report_incomplete":{}} + GH_AW_SAFE_OUTPUTS_CONFIG_09210ebb9a37e36b_EOF + - name: Write Safe Outputs Tools + run: | + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_3a2de07f9cf8864f_EOF' + { + "description_suffixes": { + "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added.", + "create_pull_request": " CONSTRAINTS: Maximum 1 pull request(s) can be created. Title will be prefixed with \"[tests] \". Labels [\"test\" \"automated\"] will be automatically added. PRs will be created as drafts." + }, + "repo_params": {}, + "dynamic_tools": [] + } + GH_AW_SAFE_OUTPUTS_TOOLS_META_3a2de07f9cf8864f_EOF + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_3769314f4b153208_EOF' + { + "add_comment": { + "defaultMax": 1, + "fields": { + "body": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "item_number": { + "issueOrPRNumber": true + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, + "create_pull_request": { + "defaultMax": 1, + "fields": { + "body": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "branch": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "draft": { + "type": "boolean" + }, + "labels": { + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + }, + "title": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 128 + } + } + }, + "missing_data": { + "defaultMax": 20, + "fields": { + "alternatives": { + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "context": { + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "data_type": { + "type": "string", + "sanitize": true, + "maxLength": 128 + }, + "reason": { + "type": "string", + "sanitize": true, + "maxLength": 256 + } + } + }, + "missing_tool": { + "defaultMax": 20, + "fields": { + "alternatives": { + "type": "string", + "sanitize": true, + "maxLength": 512 + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "tool": { + "type": "string", + "sanitize": true, + "maxLength": 128 + } + } + }, + "noop": { + "defaultMax": 1, + "fields": { + "message": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + } + } + }, + "report_incomplete": { + "defaultMax": 5, + "fields": { + "details": { + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 1024 + } + } + } + } + GH_AW_SAFE_OUTPUTS_VALIDATION_3769314f4b153208_EOF + node ${RUNNER_TEMP}/gh-aw/actions/generate_safe_outputs_tools.cjs + - name: Generate Safe Outputs MCP Server Config + id: safe-outputs-config + run: | + # Generate a secure random API key (360 bits of entropy, 40+ chars) + # Mask immediately to prevent timing vulnerabilities + API_KEY=$(openssl rand -base64 45 | tr -d '/+=') + echo "::add-mask::${API_KEY}" + + PORT=3001 + + # Set outputs for next steps + { + echo "safe_outputs_api_key=${API_KEY}" + echo "safe_outputs_port=${PORT}" + } >> "$GITHUB_OUTPUT" + + echo "Safe Outputs MCP server will run on port ${PORT}" + + - name: Start Safe Outputs MCP HTTP Server + id: safe-outputs-start + env: + DEBUG: '*' + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-config.outputs.safe_outputs_port }} + GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-config.outputs.safe_outputs_api_key }} + GH_AW_SAFE_OUTPUTS_TOOLS_PATH: ${{ runner.temp }}/gh-aw/safeoutputs/tools.json + GH_AW_SAFE_OUTPUTS_CONFIG_PATH: ${{ runner.temp }}/gh-aw/safeoutputs/config.json + GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs + run: | + # Environment variables are set above to prevent template injection + export DEBUG + export GH_AW_SAFE_OUTPUTS + export GH_AW_SAFE_OUTPUTS_PORT + export GH_AW_SAFE_OUTPUTS_API_KEY + export GH_AW_SAFE_OUTPUTS_TOOLS_PATH + export GH_AW_SAFE_OUTPUTS_CONFIG_PATH + export GH_AW_MCP_LOG_DIR + + bash ${RUNNER_TEMP}/gh-aw/actions/start_safe_outputs_server.sh + + - name: Start MCP Gateway + id: start-mcp-gateway + env: + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }} + GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }} + GITHUB_MCP_GUARD_MIN_INTEGRITY: ${{ steps.determine-automatic-lockdown.outputs.min_integrity }} + GITHUB_MCP_GUARD_REPOS: ${{ steps.determine-automatic-lockdown.outputs.repos }} + GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + run: | + set -eo pipefail + mkdir -p /tmp/gh-aw/mcp-config + + # Export gateway environment variables for MCP config and gateway script + export MCP_GATEWAY_PORT="80" + export MCP_GATEWAY_DOMAIN="host.docker.internal" + MCP_GATEWAY_API_KEY=$(openssl rand -base64 45 | tr -d '/+=') + echo "::add-mask::${MCP_GATEWAY_API_KEY}" + export MCP_GATEWAY_API_KEY + export MCP_GATEWAY_PAYLOAD_DIR="/tmp/gh-aw/mcp-payloads" + mkdir -p "${MCP_GATEWAY_PAYLOAD_DIR}" + export MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD="524288" + export DEBUG="*" + + export GH_AW_ENGINE="copilot" + export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.2.14' + + mkdir -p /home/runner/.copilot + cat << GH_AW_MCP_CONFIG_efcadc66e3192ebc_EOF | bash ${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh + { + "mcpServers": { + "github": { + "type": "stdio", + "container": "ghcr.io/github/github-mcp-server:v0.32.0", + "env": { + "GITHUB_HOST": "\${GITHUB_SERVER_URL}", + "GITHUB_PERSONAL_ACCESS_TOKEN": "\${GITHUB_MCP_SERVER_TOKEN}", + "GITHUB_READ_ONLY": "1", + "GITHUB_TOOLSETS": "pull_requests,repos" + }, + "guard-policies": { + "allow-only": { + "min-integrity": "$GITHUB_MCP_GUARD_MIN_INTEGRITY", + "repos": "$GITHUB_MCP_GUARD_REPOS" + } + } + }, + "safeoutputs": { + "type": "http", + "url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT", + "headers": { + "Authorization": "\${GH_AW_SAFE_OUTPUTS_API_KEY}" + }, + "guard-policies": { + "write-sink": { + "accept": [ + "*" + ] + } + } + } + }, + "gateway": { + "port": $MCP_GATEWAY_PORT, + "domain": "${MCP_GATEWAY_DOMAIN}", + "apiKey": "${MCP_GATEWAY_API_KEY}", + "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}" + } + } + GH_AW_MCP_CONFIG_efcadc66e3192ebc_EOF + - name: Download activation artifact + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: activation + path: /tmp/gh-aw + - name: Clean git credentials + continue-on-error: true + run: bash ${RUNNER_TEMP}/gh-aw/actions/clean_git_credentials.sh + - name: Execute GitHub Copilot CLI + id: agentic_execution + # Copilot CLI tool arguments (sorted): + # --allow-tool github + # --allow-tool safeoutputs + # --allow-tool shell(cat) + # --allow-tool shell(date) + # --allow-tool shell(dotnet:*) + # --allow-tool shell(echo) + # --allow-tool shell(find) + # --allow-tool shell(git add:*) + # --allow-tool shell(git branch:*) + # --allow-tool shell(git checkout:*) + # --allow-tool shell(git commit:*) + # --allow-tool shell(git merge:*) + # --allow-tool shell(git rm:*) + # --allow-tool shell(git status) + # --allow-tool shell(git switch:*) + # --allow-tool shell(git:*) + # --allow-tool shell(grep) + # --allow-tool shell(head) + # --allow-tool shell(ls) + # --allow-tool shell(mkdir) + # --allow-tool shell(pwd) + # --allow-tool shell(sort) + # --allow-tool shell(tail) + # --allow-tool shell(uniq) + # --allow-tool shell(wc) + # --allow-tool shell(yq) + # --allow-tool write + timeout-minutes: 60 + run: | + set -o pipefail + touch /tmp/gh-aw/agent-step-summary.md + # shellcheck disable=SC1003 + sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --image-tag 0.25.13 --skip-pull --enable-api-proxy \ + -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --allow-tool github --allow-tool safeoutputs --allow-tool '\''shell(cat)'\'' --allow-tool '\''shell(date)'\'' --allow-tool '\''shell(dotnet:*)'\'' --allow-tool '\''shell(echo)'\'' --allow-tool '\''shell(find)'\'' --allow-tool '\''shell(git add:*)'\'' --allow-tool '\''shell(git branch:*)'\'' --allow-tool '\''shell(git checkout:*)'\'' --allow-tool '\''shell(git commit:*)'\'' --allow-tool '\''shell(git merge:*)'\'' --allow-tool '\''shell(git rm:*)'\'' --allow-tool '\''shell(git status)'\'' --allow-tool '\''shell(git switch:*)'\'' --allow-tool '\''shell(git:*)'\'' --allow-tool '\''shell(grep)'\'' --allow-tool '\''shell(head)'\'' --allow-tool '\''shell(ls)'\'' --allow-tool '\''shell(mkdir)'\'' --allow-tool '\''shell(pwd)'\'' --allow-tool '\''shell(sort)'\'' --allow-tool '\''shell(tail)'\'' --allow-tool '\''shell(uniq)'\'' --allow-tool '\''shell(wc)'\'' --allow-tool '\''shell(yq)'\'' --allow-tool write --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_PAT_0 }} + COPILOT_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || '' }} + GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json + GH_AW_PHASE: agent + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + GH_AW_VERSION: v0.67.1 + GITHUB_API_URL: ${{ github.api_url }} + GITHUB_AW: true + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md + GITHUB_WORKSPACE: ${{ github.workspace }} + GIT_AUTHOR_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_AUTHOR_NAME: github-actions[bot] + GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_COMMITTER_NAME: github-actions[bot] + XDG_CONFIG_HOME: /home/runner + - name: Detect inference access error + id: detect-inference-error + if: always() + continue-on-error: true + run: bash ${RUNNER_TEMP}/gh-aw/actions/detect_inference_access_error.sh + - name: Configure Git credentials + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + GITHUB_TOKEN: ${{ github.token }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + git config --global am.keepcr true + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${GITHUB_TOKEN}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Copy Copilot session state files to logs + if: always() + continue-on-error: true + run: bash ${RUNNER_TEMP}/gh-aw/actions/copy_copilot_session_state.sh + - name: Stop MCP Gateway + if: always() + continue-on-error: true + env: + MCP_GATEWAY_PORT: ${{ steps.start-mcp-gateway.outputs.gateway-port }} + MCP_GATEWAY_API_KEY: ${{ steps.start-mcp-gateway.outputs.gateway-api-key }} + GATEWAY_PID: ${{ steps.start-mcp-gateway.outputs.gateway-pid }} + run: | + bash ${RUNNER_TEMP}/gh-aw/actions/stop_mcp_gateway.sh "$GATEWAY_PID" + - name: Redact secrets in logs + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/redact_secrets.cjs'); + await main(); + env: + GH_AW_SECRET_NAMES: 'COPILOT_PAT_0,GH_AW_GITHUB_MCP_SERVER_TOKEN,GH_AW_GITHUB_TOKEN,GITHUB_TOKEN' + SECRET_COPILOT_PAT_0: ${{ secrets.COPILOT_PAT_0 }} + SECRET_GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} + SECRET_GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} + SECRET_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + - name: Append agent step summary + if: always() + run: bash ${RUNNER_TEMP}/gh-aw/actions/append_agent_step_summary.sh + - name: Copy Safe Outputs + if: always() + env: + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + run: | + mkdir -p /tmp/gh-aw + cp "$GH_AW_SAFE_OUTPUTS" /tmp/gh-aw/safeoutputs.jsonl 2>/dev/null || true + - name: Ingest agent output + id: collect_output + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + GH_AW_ALLOWED_DOMAINS: "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_API_URL: ${{ github.api_url }} + GH_AW_COMMAND: add-tests + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/collect_ndjson_output.cjs'); + await main(); + - name: Parse agent logs for step summary + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: /tmp/gh-aw/sandbox/agent/logs/ + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_copilot_log.cjs'); + await main(); + - name: Parse MCP Gateway logs for step summary + if: always() + id: parse-mcp-gateway + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_mcp_gateway_log.cjs'); + await main(); + - name: Print firewall logs + if: always() + continue-on-error: true + env: + AWF_LOGS_DIR: /tmp/gh-aw/sandbox/firewall/logs + run: | + # Fix permissions on firewall logs so they can be uploaded as artifacts + # AWF runs with sudo, creating files owned by root + sudo chmod -R a+r /tmp/gh-aw/sandbox/firewall/logs 2>/dev/null || true + # Only run awf logs summary if awf command exists (it may not be installed if workflow failed before install step) + if command -v awf &> /dev/null; then + awf logs summary | tee -a "$GITHUB_STEP_SUMMARY" + else + echo 'AWF binary not installed, skipping firewall log summary' + fi + - name: Parse token usage for step summary + if: always() + continue-on-error: true + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_token_usage.cjs'); + await main(); + - name: Write agent output placeholder if missing + if: always() + run: | + if [ ! -f /tmp/gh-aw/agent_output.json ]; then + echo '{"items":[]}' > /tmp/gh-aw/agent_output.json + fi + - name: Upload agent artifacts + if: always() + continue-on-error: true + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + with: + name: agent + path: | + /tmp/gh-aw/aw-prompts/prompt.txt + /tmp/gh-aw/sandbox/agent/logs/ + /tmp/gh-aw/redacted-urls.log + /tmp/gh-aw/mcp-logs/ + /tmp/gh-aw/agent_usage.json + /tmp/gh-aw/agent-stdio.log + /tmp/gh-aw/agent/ + /tmp/gh-aw/github_rate_limits.jsonl + /tmp/gh-aw/safeoutputs.jsonl + /tmp/gh-aw/agent_output.json + /tmp/gh-aw/aw-*.patch + /tmp/gh-aw/aw-*.bundle + if-no-files-found: ignore + - name: Upload firewall audit logs + if: always() + continue-on-error: true + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + with: + name: firewall-audit-logs + path: | + /tmp/gh-aw/sandbox/firewall/logs/ + /tmp/gh-aw/sandbox/firewall/audit/ + if-no-files-found: ignore + + conclusion: + needs: + - activation + - agent + - detection + - safe_outputs + if: always() && (needs.agent.result != 'skipped' || needs.activation.outputs.lockdown_check_failed == 'true') + runs-on: ubuntu-slim + permissions: + contents: write + discussions: write + issues: write + pull-requests: write + concurrency: + group: "gh-aw-conclusion-add-tests" + cancel-in-progress: false + outputs: + incomplete_count: ${{ steps.report_incomplete.outputs.incomplete_count }} + noop_message: ${{ steps.noop.outputs.noop_message }} + tools_reported: ${{ steps.missing_tool.outputs.tools_reported }} + total_count: ${{ steps.missing_tool.outputs.total_count }} + steps: + - name: Setup Scripts + id: setup + uses: github/gh-aw-actions/setup@80471a493be8c528dd27daf73cd644242a7965e0 # v0.67.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + trace-id: ${{ needs.activation.outputs.setup-trace-id }} + - name: Download agent output artifact + id: download-agent-output + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Setup agent output environment variable + id: setup-agent-output-env + if: steps.download-agent-output.outcome == 'success' + run: | + mkdir -p /tmp/gh-aw/ + find "/tmp/gh-aw/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT" + - name: Process No-Op Messages + id: noop + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_NOOP_MAX: "1" + GH_AW_WORKFLOW_NAME: "Add Tests for PR Changes" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_NOOP_REPORT_AS_ISSUE: "true" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_noop_message.cjs'); + await main(); + - name: Record Missing Tool + id: missing_tool + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_MISSING_TOOL_CREATE_ISSUE: "true" + GH_AW_WORKFLOW_NAME: "Add Tests for PR Changes" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/missing_tool.cjs'); + await main(); + - name: Record Incomplete + id: report_incomplete + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_REPORT_INCOMPLETE_CREATE_ISSUE: "true" + GH_AW_WORKFLOW_NAME: "Add Tests for PR Changes" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/report_incomplete_handler.cjs'); + await main(); + - name: Handle Agent Failure + id: handle_agent_failure + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "Add Tests for PR Changes" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_WORKFLOW_ID: "add-tests" + GH_AW_ENGINE_ID: "copilot" + GH_AW_SECRET_VERIFICATION_RESULT: ${{ needs.activation.outputs.secret_verification_result }} + GH_AW_CHECKOUT_PR_SUCCESS: ${{ needs.agent.outputs.checkout_pr_success }} + GH_AW_INFERENCE_ACCESS_ERROR: ${{ needs.agent.outputs.inference_access_error }} + GH_AW_CODE_PUSH_FAILURE_ERRORS: ${{ needs.safe_outputs.outputs.code_push_failure_errors }} + GH_AW_CODE_PUSH_FAILURE_COUNT: ${{ needs.safe_outputs.outputs.code_push_failure_count }} + GH_AW_LOCKDOWN_CHECK_FAILED: ${{ needs.activation.outputs.lockdown_check_failed }} + GH_AW_GROUP_REPORTS: "false" + GH_AW_FAILURE_REPORT_AS_ISSUE: "true" + GH_AW_TIMEOUT_MINUTES: "60" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_agent_failure.cjs'); + await main(); + - name: Update reaction comment with completion status + id: conclusion + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_COMMENT_ID: ${{ needs.activation.outputs.comment_id }} + GH_AW_COMMENT_REPO: ${{ needs.activation.outputs.comment_repo }} + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_WORKFLOW_NAME: "Add Tests for PR Changes" + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_DETECTION_CONCLUSION: ${{ needs.detection.outputs.detection_conclusion }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/notify_comment_error.cjs'); + await main(); + + detection: + needs: + - activation + - agent + if: > + always() && needs.agent.result != 'skipped' && (needs.agent.outputs.output_types != '' || needs.agent.outputs.has_patch == 'true') + runs-on: ubuntu-latest + permissions: + contents: read + outputs: + detection_conclusion: ${{ steps.detection_conclusion.outputs.conclusion }} + detection_success: ${{ steps.detection_conclusion.outputs.success }} + steps: + - name: Setup Scripts + id: setup + uses: github/gh-aw-actions/setup@80471a493be8c528dd27daf73cd644242a7965e0 # v0.67.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + trace-id: ${{ needs.activation.outputs.setup-trace-id }} + - name: Download agent output artifact + id: download-agent-output + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Setup agent output environment variable + id: setup-agent-output-env + if: steps.download-agent-output.outcome == 'success' + run: | + mkdir -p /tmp/gh-aw/ + find "/tmp/gh-aw/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT" + - name: Checkout repository for patch context + if: needs.agent.outputs.has_patch == 'true' + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + # --- Threat Detection --- + - name: Download container images + run: bash ${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.25.13 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.13 ghcr.io/github/gh-aw-firewall/squid:0.25.13 + - name: Check if detection needed + id: detection_guard + if: always() + env: + OUTPUT_TYPES: ${{ needs.agent.outputs.output_types }} + HAS_PATCH: ${{ needs.agent.outputs.has_patch }} + run: | + if [[ -n "$OUTPUT_TYPES" || "$HAS_PATCH" == "true" ]]; then + echo "run_detection=true" >> "$GITHUB_OUTPUT" + echo "Detection will run: output_types=$OUTPUT_TYPES, has_patch=$HAS_PATCH" + else + echo "run_detection=false" >> "$GITHUB_OUTPUT" + echo "Detection skipped: no agent outputs or patches to analyze" + fi + - name: Clear MCP configuration for detection + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + rm -f /tmp/gh-aw/mcp-config/mcp-servers.json + rm -f /home/runner/.copilot/mcp-config.json + rm -f "$GITHUB_WORKSPACE/.gemini/settings.json" + - name: Prepare threat detection files + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + mkdir -p /tmp/gh-aw/threat-detection/aw-prompts + cp /tmp/gh-aw/aw-prompts/prompt.txt /tmp/gh-aw/threat-detection/aw-prompts/prompt.txt 2>/dev/null || true + cp /tmp/gh-aw/agent_output.json /tmp/gh-aw/threat-detection/agent_output.json 2>/dev/null || true + for f in /tmp/gh-aw/aw-*.patch; do + [ -f "$f" ] && cp "$f" /tmp/gh-aw/threat-detection/ 2>/dev/null || true + done + for f in /tmp/gh-aw/aw-*.bundle; do + [ -f "$f" ] && cp "$f" /tmp/gh-aw/threat-detection/ 2>/dev/null || true + done + echo "Prepared threat detection files:" + ls -la /tmp/gh-aw/threat-detection/ 2>/dev/null || true + - name: Setup threat detection + if: always() && steps.detection_guard.outputs.run_detection == 'true' + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + WORKFLOW_NAME: "Add Tests for PR Changes" + WORKFLOW_DESCRIPTION: "Generates unit tests for code introduced in a pull request when a contributor comments /add-tests." + HAS_PATCH: ${{ needs.agent.outputs.has_patch }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/setup_threat_detection.cjs'); + await main(); + - name: Ensure threat-detection directory and log + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + mkdir -p /tmp/gh-aw/threat-detection + touch /tmp/gh-aw/threat-detection/detection.log + - name: Install GitHub Copilot CLI + run: ${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh latest + env: + GH_HOST: github.com + - name: Install AWF binary + run: bash ${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh v0.25.13 + - name: Execute GitHub Copilot CLI + if: always() && steps.detection_guard.outputs.run_detection == 'true' + id: detection_agentic_execution + # Copilot CLI tool arguments (sorted): + timeout-minutes: 20 + run: | + set -o pipefail + touch /tmp/gh-aw/agent-step-summary.md + # shellcheck disable=SC1003 + sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,github.com,host.docker.internal,telemetry.enterprise.githubcopilot.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --image-tag 0.25.13 --skip-pull --enable-api-proxy \ + -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --allow-all-tools --add-dir "${GITHUB_WORKSPACE}" --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/threat-detection/detection.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_PAT_0 }} + COPILOT_MODEL: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || '' }} + GH_AW_PHASE: detection + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_VERSION: v0.67.1 + GITHUB_API_URL: ${{ github.api_url }} + GITHUB_AW: true + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md + GITHUB_WORKSPACE: ${{ github.workspace }} + GIT_AUTHOR_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_AUTHOR_NAME: github-actions[bot] + GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_COMMITTER_NAME: github-actions[bot] + XDG_CONFIG_HOME: /home/runner + - name: Upload threat detection log + if: always() && steps.detection_guard.outputs.run_detection == 'true' + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + with: + name: detection + path: /tmp/gh-aw/threat-detection/detection.log + if-no-files-found: ignore + - name: Parse and conclude threat detection + id: detection_conclusion + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + RUN_DETECTION: ${{ steps.detection_guard.outputs.run_detection }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_threat_detection_results.cjs'); + await main(); + + pre_activation: + if: "github.event_name == 'issue_comment' && (startsWith(github.event.comment.body, '/add-tests ') || startsWith(github.event.comment.body, '/add-tests\n') || github.event.comment.body == '/add-tests') && github.event.issue.pull_request != null || !(github.event_name == 'issue_comment')" + runs-on: ubuntu-slim + outputs: + activated: ${{ steps.check_membership.outputs.is_team_member == 'true' && steps.check_command_position.outputs.command_position_ok == 'true' }} + matched_command: ${{ steps.check_command_position.outputs.matched_command }} + setup-trace-id: ${{ steps.setup.outputs.trace-id }} + steps: + - name: Setup Scripts + id: setup + uses: github/gh-aw-actions/setup@80471a493be8c528dd27daf73cd644242a7965e0 # v0.67.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + - name: Check team membership for command workflow + id: check_membership + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_REQUIRED_ROLES: "admin,maintainer,write" + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/check_membership.cjs'); + await main(); + - name: Check command position + id: check_command_position + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_COMMANDS: "[\"add-tests\"]" + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/check_command_position.cjs'); + await main(); + + safe_outputs: + needs: + - activation + - agent + - detection + if: (!cancelled()) && needs.agent.result != 'skipped' && needs.detection.result == 'success' + runs-on: ubuntu-slim + permissions: + contents: write + discussions: write + issues: write + pull-requests: write + timeout-minutes: 15 + env: + GH_AW_CALLER_WORKFLOW_ID: "${{ github.repository }}/add-tests" + GH_AW_EFFECTIVE_TOKENS: ${{ needs.agent.outputs.effective_tokens }} + GH_AW_ENGINE_ID: "copilot" + GH_AW_ENGINE_MODEL: ${{ needs.agent.outputs.model }} + GH_AW_WORKFLOW_ID: "add-tests" + GH_AW_WORKFLOW_NAME: "Add Tests for PR Changes" + outputs: + code_push_failure_count: ${{ steps.process_safe_outputs.outputs.code_push_failure_count }} + code_push_failure_errors: ${{ steps.process_safe_outputs.outputs.code_push_failure_errors }} + comment_id: ${{ steps.process_safe_outputs.outputs.comment_id }} + comment_url: ${{ steps.process_safe_outputs.outputs.comment_url }} + create_discussion_error_count: ${{ steps.process_safe_outputs.outputs.create_discussion_error_count }} + create_discussion_errors: ${{ steps.process_safe_outputs.outputs.create_discussion_errors }} + created_pr_number: ${{ steps.process_safe_outputs.outputs.created_pr_number }} + created_pr_url: ${{ steps.process_safe_outputs.outputs.created_pr_url }} + process_safe_outputs_processed_count: ${{ steps.process_safe_outputs.outputs.processed_count }} + process_safe_outputs_temporary_id_map: ${{ steps.process_safe_outputs.outputs.temporary_id_map }} + steps: + - name: Setup Scripts + id: setup + uses: github/gh-aw-actions/setup@80471a493be8c528dd27daf73cd644242a7965e0 # v0.67.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + trace-id: ${{ needs.activation.outputs.setup-trace-id }} + - name: Download agent output artifact + id: download-agent-output + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Setup agent output environment variable + id: setup-agent-output-env + if: steps.download-agent-output.outcome == 'success' + run: | + mkdir -p /tmp/gh-aw/ + find "/tmp/gh-aw/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT" + - name: Download patch artifact + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Checkout repository + if: (!cancelled()) && needs.agent.result != 'skipped' && contains(needs.agent.outputs.output_types, 'create_pull_request') + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + ref: ${{ github.base_ref || github.event.pull_request.base.ref || github.ref_name || github.event.repository.default_branch }} + token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + persist-credentials: false + fetch-depth: 1 + - name: Configure Git credentials + if: (!cancelled()) && needs.agent.result != 'skipped' && contains(needs.agent.outputs.output_types, 'create_pull_request') + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + GIT_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + git config --global am.keepcr true + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${GIT_TOKEN}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Configure GH_HOST for enterprise compatibility + id: ghes-host-config + shell: bash + run: | + # Derive GH_HOST from GITHUB_SERVER_URL so the gh CLI targets the correct + # GitHub instance (GHES/GHEC). On github.com this is a harmless no-op. + GH_HOST="${GITHUB_SERVER_URL#https://}" + GH_HOST="${GH_HOST#http://}" + echo "GH_HOST=${GH_HOST}" >> "$GITHUB_ENV" + - name: Process Safe Outputs + id: process_safe_outputs + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_ALLOWED_DOMAINS: "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_API_URL: ${{ github.api_url }} + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"max\":3},\"create_pull_request\":{\"draft\":true,\"labels\":[\"test\",\"automated\"],\"max\":1,\"max_patch_size\":1024,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"AGENTS.md\"],\"protected_files_policy\":\"fallback-to-issue\",\"protected_path_prefixes\":[\".github/\",\".agents/\"],\"title_prefix\":\"[tests] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"report_incomplete\":{}}" + GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/safe_output_handler_manager.cjs'); + await main(); + - name: Upload Safe Output Items + if: always() + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + with: + name: safe-output-items + path: /tmp/gh-aw/safe-output-items.jsonl + if-no-files-found: ignore + diff --git a/.github/workflows/add-tests.md b/.github/workflows/add-tests.md new file mode 100644 index 000000000000..fced78471561 --- /dev/null +++ b/.github/workflows/add-tests.md @@ -0,0 +1,112 @@ +--- +description: "Generates unit tests for code introduced in a pull request when a contributor comments /add-tests." + +on: + slash_command: + name: add-tests + events: [pull_request_comment] + roles: [admin, maintainer, write] + +engine: + id: copilot + env: + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_PAT_0 }} + +permissions: + contents: read + pull-requests: read + +imports: + - shared/repo-build-setup.md + +tools: + github: + toolsets: [pull_requests, repos] + edit: + bash: ["dotnet", "git", "find", "ls", "cat", "grep", "head", "tail", "wc", "mkdir"] + +safe-outputs: + create-pull-request: + title-prefix: "[tests] " + labels: [test, automated] + draft: true + max: 1 + protected-files: fallback-to-issue + add-comment: + max: 3 + +timeout-minutes: 60 +--- + +# Add Tests for PR Changes + +Generate comprehensive unit tests for the code changes introduced in pull request #${{ github.event.issue.number }}. + +## Context + +The PR comment that triggered this workflow: "${{ steps.sanitized.outputs.text }}" + +## Goal + +Analyze the pull request diff to identify source files that were added or modified, then generate unit tests that cover those changes. The resulting tests should be submitted as a new draft pull request. + +## Instructions + +### Step 1: Understand the PR Changes + +1. Use the GitHub pull requests tools to fetch the PR diff for PR #${{ github.event.issue.number }} +2. Identify all **source files** (under `src/`) that were added or modified — ignore test files, build files, docs, and config +3. For each changed source file, understand what classes, methods, or functionality was added or changed + +### Step 2: Identify Test Gaps + +1. For each changed source file, find the corresponding existing test project under `test/` +2. Check if the changed code already has test coverage +3. Focus on code that is **not yet covered** by existing tests + +### Step 3: Generate Tests + +Use the `task` tool to invoke the `code-testing-generator` agent (defined at `.github/agents/code-testing-generator.agent.md`) as a **subagent**: + +``` +task({ + agent: "code-testing-generator", + prompt: "Generate unit tests for the changed source files in this PR. Follow the Research → Plan → Implement pipeline. Use the `task` tool to invoke all sub-agents (code-testing-researcher, code-testing-planner, code-testing-implementer, etc.) as subagents." +}) +``` + +**Important**: The `code-testing-generator` agent MUST use the `task` tool to invoke its own sub-agents (researcher, planner, implementer) as subagents. Do NOT inline the sub-agent work — each sub-agent must be called via the `task` tool. + +Follow these SDK-specific conventions: + +1. Follow the Research → Plan → Implement pipeline from the skill +2. **Scope**: Only generate tests for code modified in this PR — do not attempt full-repo coverage +3. **Test framework**: This repo uses xUnit with `[Fact]`, `[Theory]`, `[InlineData]` attributes. Some projects also use MSTest +4. **Naming**: Test classes as `{Feature}Tests`, test methods as PascalCase descriptive names +5. **License header**: Every `.cs` file must start with the 3-line .NET Foundation MIT license header: + ``` + // Licensed to the .NET Foundation under one or more agreements. + // The .NET Foundation licenses this file to you under the MIT license. + // See the LICENSE file in the project root for more information. + ``` +6. **Style**: Follow existing test patterns in the repo — check adjacent test files for conventions +7. **Test projects**: When creating new test projects in `test/`, always use `$(CurrentTargetFramework)` for the `` property instead of hard-coding a specific version +8. **Build**: Use `dotnet build ` for scoped builds during development +9. **Test execution**: + - For MSTest-style projects: `dotnet test path/to/project.csproj --filter "FullyQualifiedName~TestName"` + - For XUnit test assemblies: `dotnet exec artifacts/bin/redist/Debug/TestAssembly.dll -method "*TestMethodName*"` +10. **Skip parameter**: When using the `Skip` parameter of the `[Fact]` attribute, point to a specific issue link +11. **Do not modify `.xlf` files** — if localization files need updating, note it in the PR description + +### Step 4: Validate + +1. Build the specific test project(s) you modified +2. Run the tests to verify they pass +3. If tests fail, fix assertions based on actual production code behavior — never skip or ignore tests + +### Step 5: Create the PR + +Commit all test files and create a draft pull request. The PR description should: +- Reference the original PR (#${{ github.event.issue.number }}) +- List the test files created +- Summarize what is covered by the new tests diff --git a/.github/workflows/shared/repo-build-setup.md b/.github/workflows/shared/repo-build-setup.md new file mode 100644 index 000000000000..7c01bc881820 --- /dev/null +++ b/.github/workflows/shared/repo-build-setup.md @@ -0,0 +1,36 @@ +--- +# Repo-specific build & environment setup for the add-tests workflow. +# +# This shared component is imported by add-tests.md to keep repo-specific +# configuration (build commands, checkout options) separate +# from the generic workflow logic. +# +# To adapt for your own repository: +# 1. Adjust checkout settings (submodules, LFS, fetch-depth) as needed +# 2. Replace the steps with your repo's build commands +# 3. Ensure dotnet (or your toolchain) is on PATH after the steps run + +description: "Repository-specific build setup for add-tests workflow" + +steps: + - name: Build + run: ./build.sh + + - name: Put dotnet on the path + run: echo "PATH=$PWD/.dotnet:$PATH" >> $GITHUB_ENV +--- + +# Repository Build Setup + +This file contains the repo-specific build configuration for the `/add-tests` workflow. + +## What this provides + +- **Build**: Runs the Arcade SDK build script (`build.sh`) to compile all managed code and produce the repo-local .NET SDK +- **PATH**: Adds the locally-installed .NET SDK (`.dotnet/`) to `$PATH` so the agent can invoke `dotnet` directly + +## Customization + +To use the `/add-tests` workflow in a different repository, replace this file with your own +build setup steps. The workflow only requires that the repo is built and the +appropriate toolchain is available on `$PATH` after these steps run.