Skip to content

test: guard secondmate harness marker isolation - #2235

Open
4mb1t10n wants to merge 4 commits into
kunchenguid:mainfrom
4mb1t10n:fm/fm-1243
Open

test: guard secondmate harness marker isolation#2235
4mb1t10n wants to merge 4 commits into
kunchenguid:mainfrom
4mb1t10n:fm/fm-1243

Conversation

@4mb1t10n

Copy link
Copy Markdown

Intent

Implement #1243 exactly as specified: make tests/fm-secondmate-harness.test.sh hermetic against ambient harness markers so the suite behaves the same inside Claude Code, Pi, pi-signed, or Grok sessions. Keep the change test-only and exclude all product harness-resolution changes. Reproduce the ambient-marker failure and prove the corrected focused harness-resolution suite. Target main, fully close #1243 with a standalone , and preserve the existing harness semantics. Do not use a database or browser stack and never run local Cypress. Complete the No Mistakes pipeline plus exact-head CI and mergeability verification, but never merge.

What Changed

  • Add a regression guard that verifies Claude Code, Pi, pi-signed, and Grok harness markers are absent before the focused secondmate harness-resolution tests run.

Risk Assessment

✅ Low: The test-only change adds an executable guard for all supported ambient harness markers, complements the existing top-level marker scrub, and does not alter product harness semantics.

Testing

Confirmed the nine-line test-only diff and exercised the executable focused harness-resolution behavior under ambient Claude, Pi, pi-signed, and Grok markers; every environment scrubbed its inherited marker and preserved existing fallback plus model/effort semantics, with reviewer-visible TAP transcripts captured and no worktree changes.

Evidence: Claude ambient-marker transcript
ok - secondmate harness suite scrubs ambient harness markers
ok - A1 fm-harness.sh secondmate resolves the fallback chain; crew mode unchanged
ok - C1 fm-harness.sh secondmate-model/secondmate-effort resolve the optional tokens; bare harness stays empty (backward-compat)
# focused harness-resolution tests passed
Evidence: Pi ambient-marker transcript
ok - secondmate harness suite scrubs ambient harness markers
ok - A1 fm-harness.sh secondmate resolves the fallback chain; crew mode unchanged
ok - C1 fm-harness.sh secondmate-model/secondmate-effort resolve the optional tokens; bare harness stays empty (backward-compat)
# focused harness-resolution tests passed
Evidence: pi-signed ambient-marker transcript
ok - secondmate harness suite scrubs ambient harness markers
ok - A1 fm-harness.sh secondmate resolves the fallback chain; crew mode unchanged
ok - C1 fm-harness.sh secondmate-model/secondmate-effort resolve the optional tokens; bare harness stays empty (backward-compat)
# focused harness-resolution tests passed
Evidence: Grok ambient-marker transcript
ok - secondmate harness suite scrubs ambient harness markers
ok - A1 fm-harness.sh secondmate resolves the fallback chain; crew mode unchanged
ok - C1 fm-harness.sh secondmate-model/secondmate-effort resolve the optional tokens; bare harness stays empty (backward-compat)
# focused harness-resolution tests passed

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • Inspected git diff b5d430d6fdcd961ce9b681bf196f365c1825c284 4255fd852d1c384c2607c52f4959275b9d7e0366 -- tests/fm-secondmate-harness.test.sh to confirm the change is test-only.
  • Executed the focused test_ambient_harness_markers_are_scrubbed, test_harness_resolution, and test_secondmate_model_effort_tokens functions under CLAUDECODE=1.
  • Executed the same focused cases under PI_CODING_AGENT=true FM_PI_HARNESS=pi.
  • Executed the same focused cases under PI_CODING_AGENT=true FM_PI_HARNESS=pi-signed.
  • Executed the same focused cases under GROK_AGENT=1.
  • Attempted a base-commit regression run and a longer target-suite run; concurrent lifecycle fixtures interfered with one another, so those transcripts were excluded from acceptance evidence and all related processes were stopped.
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests/fm-secondmate-harness.test.sh does not neutralise ambient CLAUDECODE, so the whole detect_own block fails inside a Claude Code session

1 participant