Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .apm/instructions/cicd.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ integration suite runs only at merge time via GitHub Merge Queue
## Release Flow Dependencies
- **PR workflow**: Tier 1 only - ci.yml (build-and-test, Linux-only) provides fast feedback. Tier 2 does not run until enqueued.
- **Merge queue workflow**: ci.yml (Tier 1 against tentative merge ref) + ci-integration.yml (Tier 2: build -> smoke-test -> integration-tests -> release-validation). Queue auto-merges on success; ejects on failure.
- **Push/Release workflow (Linux + Windows)**: build-and-test -> integration-tests -> release-validation -> create-release -> publish-pypi -> update-homebrew (gh-aw-compat runs in parallel, informational)
- **Push/Release workflow (Linux + Windows)**: build-and-test -> integration-tests -> release-validation -> create-release -> publish-pypi (gh-aw-compat runs in parallel, informational)
- **Homebrew distribution**: `microsoft/homebrew-apm` polls the latest public APM release and updates its formula with its own `GITHUB_TOKEN`; this workflow must not send a cross-repository dispatch or hold a tap credential.
- **Push/Release workflow (macOS Intel)**: build-and-validate-macos-intel (root node: unit tests + build always + conditional integration/release-validation) -> create-release
- **Push/Release workflow (macOS ARM)**: build-and-validate-macos-arm (root node, tag/schedule/dispatch only; all phases run) -> create-release
- **Tag Triggers**: Only `v*.*.*` tags trigger full release pipeline
Expand Down
3 changes: 2 additions & 1 deletion .github/instructions/cicd.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ integration suite runs only at merge time via GitHub Merge Queue
## Release Flow Dependencies
- **PR workflow**: Tier 1 only - ci.yml (build-and-test, Linux-only) provides fast feedback. Tier 2 does not run until enqueued.
- **Merge queue workflow**: ci.yml (Tier 1 against tentative merge ref) + ci-integration.yml (Tier 2: build -> smoke-test -> integration-tests -> release-validation). Queue auto-merges on success; ejects on failure.
- **Push/Release workflow (Linux + Windows)**: build-and-test -> integration-tests -> release-validation -> create-release -> publish-pypi -> update-homebrew (gh-aw-compat runs in parallel, informational)
- **Push/Release workflow (Linux + Windows)**: build-and-test -> integration-tests -> release-validation -> create-release -> publish-pypi (gh-aw-compat runs in parallel, informational)
- **Homebrew distribution**: `microsoft/homebrew-apm` polls the latest public APM release and updates its formula with its own `GITHUB_TOKEN`; this workflow must not send a cross-repository dispatch or hold a tap credential.
- **Push/Release workflow (macOS Intel)**: build-and-validate-macos-intel (root node: unit tests + build always + conditional integration/release-validation) -> create-release
- **Push/Release workflow (macOS ARM)**: build-and-validate-macos-arm (root node, tag/schedule/dispatch only; all phases run) -> create-release
- **Tag Triggers**: Only `v*.*.*` tags trigger full release pipeline
Expand Down
104 changes: 0 additions & 104 deletions .github/workflows/build-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -740,107 +740,3 @@ jobs:
uses: pypa/gh-action-pypi-publish@release/v1
with:
verbose: true

# Update Homebrew formula (only stable releases from public repo)
update-homebrew:
name: Update Homebrew Formula
runs-on: ubuntu-latest
needs: [build-and-test, build-and-validate-macos-intel, build-and-validate-macos-arm, integration-tests, release-validation, create-release, publish-pypi]
if: github.ref_type == 'tag' && needs.create-release.outputs.is_private_repo != 'true' && needs.create-release.outputs.is_prerelease != 'true'
permissions:
contents: read

steps:
- name: Extract SHA256 checksums from GitHub release
id: checksums
run: |
# Download the SHA256 checksum files from the GitHub release
RELEASE_TAG="${{ github.ref_name }}"

echo "Downloading checksums for release $RELEASE_TAG"

# Download checksum files directly from the release
curl -L -o apm-darwin-arm64.tar.gz.sha256 \
"https://github.com/${{ github.repository }}/releases/download/$RELEASE_TAG/apm-darwin-arm64.tar.gz.sha256"
curl -L -o apm-darwin-x86_64.tar.gz.sha256 \
"https://github.com/${{ github.repository }}/releases/download/$RELEASE_TAG/apm-darwin-x86_64.tar.gz.sha256"
curl -L -o apm-linux-x86_64.tar.gz.sha256 \
"https://github.com/${{ github.repository }}/releases/download/$RELEASE_TAG/apm-linux-x86_64.tar.gz.sha256"
curl -L -o apm-linux-arm64.tar.gz.sha256 \
"https://github.com/${{ github.repository }}/releases/download/$RELEASE_TAG/apm-linux-arm64.tar.gz.sha256"

# Extract SHA256 checksums
DARWIN_ARM64_SHA=$(cat apm-darwin-arm64.tar.gz.sha256 | cut -d' ' -f1)
DARWIN_X86_64_SHA=$(cat apm-darwin-x86_64.tar.gz.sha256 | cut -d' ' -f1)
LINUX_X86_64_SHA=$(cat apm-linux-x86_64.tar.gz.sha256 | cut -d' ' -f1)
LINUX_ARM64_SHA=$(cat apm-linux-arm64.tar.gz.sha256 | cut -d' ' -f1)

echo "darwin-arm64-sha=$DARWIN_ARM64_SHA" >> $GITHUB_OUTPUT
echo "darwin-x86_64-sha=$DARWIN_X86_64_SHA" >> $GITHUB_OUTPUT
echo "linux-x86_64-sha=$LINUX_X86_64_SHA" >> $GITHUB_OUTPUT
echo "linux-arm64-sha=$LINUX_ARM64_SHA" >> $GITHUB_OUTPUT

echo "Darwin ARM64 SHA: $DARWIN_ARM64_SHA"
echo "Darwin x86_64 SHA: $DARWIN_X86_64_SHA"
echo "Linux x86_64 SHA: $LINUX_X86_64_SHA"
echo "Linux ARM64 SHA: $LINUX_ARM64_SHA"

- name: Trigger Homebrew tap repository update
uses: peter-evans/repository-dispatch@v3
with:
token: ${{ secrets.GH_PKG_PAT }}
repository: microsoft/homebrew-apm
event-type: formula-update
client-payload: |
{
"release": {
"version": "${{ github.ref_name }}",
"tag": "${{ github.ref_name }}",
"repository": "${{ github.repository }}"
},
"checksums": {
"darwin_arm64": "${{ steps.checksums.outputs.darwin-arm64-sha }}",
"darwin_x86_64": "${{ steps.checksums.outputs.darwin-x86_64-sha }}",
"linux_x86_64": "${{ steps.checksums.outputs.linux-x86_64-sha }}",
"linux_arm64": "${{ steps.checksums.outputs.linux-arm64-sha }}"
}
}

# Update Scoop bucket (only stable releases from public repo)
update-scoop:
name: Update Scoop Bucket
runs-on: ubuntu-latest
needs: [build-and-test, build-and-validate-macos-intel, build-and-validate-macos-arm, integration-tests, release-validation, create-release, publish-pypi]
# TODO: Enable once downstream repository and secrets are configured (see #88)
if: false && github.ref_type == 'tag' && needs.create-release.outputs.is_private_repo != 'true' && needs.create-release.outputs.is_prerelease != 'true'
permissions:
contents: read

steps:
- name: Extract Windows checksum from GitHub release
id: checksums
run: |
RELEASE_TAG="${{ github.ref_name }}"
curl -L -o apm-windows-x86_64.zip.sha256 \
"https://github.com/${{ github.repository }}/releases/download/$RELEASE_TAG/apm-windows-x86_64.zip.sha256"
WINDOWS_X86_64_SHA=$(cat apm-windows-x86_64.zip.sha256 | cut -d' ' -f1)
echo "windows-x86_64-sha=$WINDOWS_X86_64_SHA" >> $GITHUB_OUTPUT
echo "Windows x86_64 SHA: $WINDOWS_X86_64_SHA"

- name: Trigger Scoop bucket repository update
uses: peter-evans/repository-dispatch@v3
with:
token: ${{ secrets.GH_PKG_PAT }}
repository: microsoft/scoop-apm
event-type: bucket-update
client-payload: |
{
"release": {
"version": "${{ github.ref_name }}",
"tag": "${{ github.ref_name }}",
"repository": "${{ github.repository }}"
},
"checksums": {
"windows_x86_64": "${{ steps.checksums.outputs.windows-x86_64-sha }}"
}
}
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Fixed

- Fresh checkouts with declared consumer targets no longer remain
Expand Down
2 changes: 1 addition & 1 deletion apm.lock.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -683,7 +683,7 @@ local_deployed_file_hashes:
.github/agents/spec-tag-architect.agent.md: sha256:82907265c5e7cf1ac61ad96866fa7c5683b69c8f09b7a4c5f3cc241acc9568ca
.github/agents/supply-chain-security-expert.agent.md: sha256:8fb8cc426d6af17ba084a28b3f026c2b475b62e3ca63ed2f88b83bd823f877af
.github/instructions/changelog.instructions.md: sha256:1e51ec4c74e847967962bd279dc4c6e582c5d3578490b3c28d5f3acd3e05f73e
.github/instructions/cicd.instructions.md: sha256:9c0fafc74f743aa97e5adba2168d66c9e3a327b135065e3b804bdbb5f04cda5d
.github/instructions/cicd.instructions.md: sha256:08d87b7d635761cb41deb8fc71d5d83f54678de463db484afb16d2d4f8713ecb
.github/instructions/cli.instructions.md: sha256:8e39e8d5047ce88575cb02f87c2bcede584dfef258bd86f7466c7badf136541a
.github/instructions/doc-sync.instructions.md: sha256:bb3816254f8df6bffc6faacd556871f36903e9d7f348982f1e2de0339384c696
.github/instructions/encoding.instructions.md: sha256:93db7377dc896f6efecf2c5d8c5d89255a555562f468d034d64c42edd5cf46d5
Expand Down
11 changes: 3 additions & 8 deletions docs/src/content/docs/contributing/integration-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,6 @@ environment end-to-end; for local iteration prefer the direct
3. **E2E golden scenario tests** (using built binaries)
4. Create GitHub Release
5. Publish to PyPI
6. Update Homebrew Formula

**Manual workflow dispatch:**
- Test builds (uploads as workflow artifacts)
Expand Down Expand Up @@ -159,10 +158,11 @@ The workflow ensures quality gates at each step:
3. **release-validation** job - Final shipped-binary validation
4. **create-release** job - GitHub release creation
5. **publish-pypi** job - PyPI package publication
6. **update-homebrew** job - Homebrew formula update

Each stage must succeed before proceeding to the next, ensuring only fully validated releases reach users.

The [`microsoft/homebrew-apm`](https://github.com/microsoft/homebrew-apm) tap updates independently: it polls the latest APM release and commits formula updates with its own repository-scoped `GITHUB_TOKEN`. The release pipeline does not hold a cross-repository Homebrew credential.

### Test Matrix

Promotion integration tests run on:
Expand Down Expand Up @@ -196,7 +196,7 @@ Promotion integration tests run on:
- ✅ Parameter substitution works correctly
- ✅ MCP integration functions (GitHub tools)
- ✅ Binary artifacts work across platforms
- ✅ Release pipeline integrity (GitHub Release → PyPI → Homebrew)
- ✅ Release pipeline integrity (GitHub Release → PyPI)

## Benefits

Expand All @@ -219,7 +219,6 @@ Promotion integration tests run on:
- Guarantees shipped releases work end-to-end
- Users can trust the README golden scenario
- Cross-platform binary verification
- Automatic Homebrew formula updates

## Debugging Test Failures

Expand Down Expand Up @@ -258,7 +257,3 @@ Promotion integration tests run on:
so contributors without those credentials still get a clean
`SKIPPED` rather than a hard failure.
3. Keep tests focused and fast.

---

This testing strategy ensures we ship with confidence while maintaining fast development cycles.
76 changes: 76 additions & 0 deletions tests/integration/release_homebrew_update_harness.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
"""Structural release guard plus a hermetic model of the tap's local write."""

from __future__ import annotations

import argparse
import json
import re
import subprocess
from pathlib import Path

import yaml


def _assert_release_workflow_has_no_package_manager_push(workflow_path: Path) -> None:
workflow = yaml.safe_load(workflow_path.read_text(encoding="utf-8"))
for job_name, job in workflow.get("jobs", {}).items():
rendered = json.dumps(job)
forbidden = ("GH_PKG_PAT", "repository-dispatch", "repository_dispatch", "gh pr create")
found = [token for token in forbidden if token in rendered]
package_manager_job = any(
name in f"{job_name} {rendered}".lower() for name in ("homebrew", "scoop")
)
if package_manager_job and re.search(
r"(?:secrets\.[A-Z0-9_]*(?:PAT|TOKEN)|github\.token)",
rendered,
re.IGNORECASE,
):
found.append("workflow token")
if found:
raise RuntimeError(
f"release workflow job {job_name} restores a package-manager "
f"push/auth path: {', '.join(found)}"
)


def _replace_assignment(formula: str, key: str, value: str) -> str:
assignment = re.compile(rf'(?m)^ {re.escape(key)} "[^"]*"$')
updated, count = assignment.subn(f' {key} "{value}"', formula)
if count != 1:
raise RuntimeError(f"formula must contain exactly one {key} assignment")
return updated


def update_tap(workflow_path: Path, release_path: Path, tap_repo: Path) -> None:
"""Model the tap's repository-local formula commit from release metadata."""
_assert_release_workflow_has_no_package_manager_push(workflow_path)

release = json.loads(release_path.read_text(encoding="utf-8"))
formula_path = tap_repo / "Formula" / "apm.rb"
formula = formula_path.read_text(encoding="utf-8")
formula = _replace_assignment(formula, "version", release["tag_name"].removeprefix("v"))
formula = _replace_assignment(formula, "url", release["asset_url"])
formula = _replace_assignment(formula, "sha256", release["sha256"])
formula_path.write_text(formula, encoding="utf-8")

subprocess.run(["git", "add", str(formula_path)], cwd=tap_repo, check=True)
subprocess.run(
["git", "commit", "-m", f"Update APM to {release['tag_name']}"],
cwd=tap_repo,
check=True,
capture_output=True,
text=True,
)


def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--workflow", type=Path, required=True)
parser.add_argument("--release", type=Path, required=True)
parser.add_argument("--tap-repo", type=Path, required=True)
args = parser.parse_args()
update_tap(args.workflow, args.release, args.tap_repo)


if __name__ == "__main__":
main()
103 changes: 103 additions & 0 deletions tests/integration/test_release_homebrew_update.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
"""Regression coverage for the Homebrew release ownership boundary."""

from __future__ import annotations

import json
import os
import subprocess
import sys
from pathlib import Path
from urllib.parse import urlparse


def _git(repo: Path, *args: str) -> str:
result = subprocess.run(
["git", *args],
cwd=repo,
check=True,
capture_output=True,
text=True,
)
return result.stdout.strip()


def test_homebrew_update_commits_without_cross_repo_auth(tmp_path: Path) -> None:
"""Run a local tap commit after guarding the release workflow auth boundary."""
repo_root = Path(__file__).parents[2]
tap_repo = tmp_path / "homebrew-apm"
formula_dir = tap_repo / "Formula"
formula_dir.mkdir(parents=True)
formula = formula_dir / "apm.rb"
formula.write_text(
"class Apm < Formula\n"
' version "0.0.1"\n'
' url "https://example.invalid/apm-v0.0.1.tar.gz"\n'
f' sha256 "{"0" * 64}"\n'
"end\n",
encoding="utf-8",
)
_git(tap_repo, "init", "-b", "main")
_git(tap_repo, "config", "user.name", "Homebrew update harness")
_git(tap_repo, "config", "user.email", "harness@example.invalid")
_git(tap_repo, "add", "Formula/apm.rb")
_git(tap_repo, "commit", "-m", "Initial formula")

release = tmp_path / "release.json"
release.write_text(
json.dumps(
{
"tag_name": "v9.8.7",
"asset_url": "https://github.com/microsoft/apm/releases/download/v9.8.7/apm.tar.gz",
"sha256": "a" * 64,
}
),
encoding="utf-8",
)

fake_bin = tmp_path / "bin"
fake_bin.mkdir()
gh_marker = tmp_path / "gh-was-called"
fake_gh = fake_bin / "gh"
fake_gh.write_text(
f"#!/bin/sh\nprintf called > {gh_marker}\nexit 97\n",
encoding="utf-8",
)
fake_gh.chmod(0o755)

env = os.environ.copy()
env.pop("GH_PKG_PAT", None)
env.pop("GH_TOKEN", None)
env["PATH"] = f"{fake_bin}{os.pathsep}{env['PATH']}"
subprocess.run(
[
sys.executable,
str(Path(__file__).with_name("release_homebrew_update_harness.py")),
"--workflow",
str(repo_root / ".github" / "workflows" / "build-release.yml"),
"--release",
str(release),
"--tap-repo",
str(tap_repo),
],
cwd=repo_root,
env=env,
check=True,
)

updated = formula.read_text(encoding="utf-8")
assignments = {
line.strip().split(" ", 1)[0]: line.strip().split('"')[1]
for line in updated.splitlines()
if line.strip().startswith(("version ", "url ", "sha256 "))
}
assert assignments["version"] == "9.8.7"
parsed = urlparse(assignments["url"])
assert (parsed.scheme, parsed.hostname, parsed.path) == (
"https",
"github.com",
"/microsoft/apm/releases/download/v9.8.7/apm.tar.gz",
)
assert assignments["sha256"] == "a" * 64
assert _git(tap_repo, "rev-list", "--count", "HEAD") == "2"
assert _git(tap_repo, "log", "-1", "--pretty=%s") == "Update APM to v9.8.7"
assert not gh_marker.exists()
Loading