Skip to content

fix: make the failure sentinel diagnosable (version stamp, bounded history, scoped hint) - #72

Draft
m3talux wants to merge 1 commit into
mainfrom
fix/c1-sentinel-hygiene
Draft

fix: make the failure sentinel diagnosable (version stamp, bounded history, scoped hint)#72
m3talux wants to merge 1 commit into
mainfrom
fix/c1-sentinel-hygiene

Conversation

@m3talux

@m3talux m3talux commented Jun 9, 2026

Copy link
Copy Markdown
Owner

Context — diagnosed production incident (2026-06-09)

During the cross-version index incident (a stale pre-0.8 plugin cache writing v0.7 float vectors into the v0.8 int8[256] store — see the companion fence PR), the failure sentinel actively hindered diagnosis:

  1. No attribution — multiple immutable plugin caches (different versions) share one ~/.superbrain; an unattributed last-failure.txt line could have been written by any of them.
  2. No historylast-failure.txt is overwrite-on-write and read-and-clear, so each failure destroyed the previous one.
  3. Misleading hintsb-session-start appended "set ANTHROPIC_API_KEY if it persists" to EVERY failure, including index errors, steering diagnosis toward auth/quota.

No data was lost in the incident; this PR is purely about diagnosability next time.

Change

  • src/sentinel.tswriteFailure() stamps every failure line with the writing plugin's package version ([vX.Y.Z], resolved via pluginRoot(), cached per process; "unknown" fallback).
  • Failure history: every failure is also appended to ~/.superbrain/failures.log, windowed to the last MAX_FAILURE_LOG_LINES = 100 lines (same windowed-log pattern as the reject-queue cap). The log is never read-and-cleared, so history survives the SessionStart sentinel clear. Best-effort: history can never break the sentinel write.
  • bin/sb-session-start.ts — the ANTHROPIC_API_KEY hint is now scoped to failures matching /distill/i (the only failures where the API-key escape hatch helps); all other failures keep the "fixed automatically next checkpoint" copy without it.

Tests (red → green, tests/sentinelHygiene.test.ts)

  • Version stamp present in last-failure.txt (was red: no stamp existed).
  • History appends across multiple failures and stays bounded at exactly 100 lines, oldest rotated out (was red: no failures.log).
  • Hint scoping: an index-failure message produces SessionStart output without ANTHROPIC_API_KEY; a distill-failure message keeps it (index case was red).
  • Capture-failure surfacing E2E (this seam had no coverage): vec_chunks recreated as int8[8] to reproduce the incident's insert failure, then the real distill entrypoint (bin/sb-distill.ts) run with SUPERBRAIN_DISTILL_STUB — asserts the note .md IS written, exit code 0, and the sentinel contains index failed with the version stamp.

Status

  • Full suite: 120 files / 796 tests passed (baseline at main: 119/790).
  • dist/ rebuilt and committed; git diff --exit-code dist clean after build.
  • Banked for the June 12 reassessment — DRAFT, do not merge, no release cut.

🤖 Generated with Claude Code

…story, scoped hint)\n\nDuring the 2026-06 cross-version index incident the sentinel was nearly\nuseless for diagnosis: multiple immutable plugin caches shared one\n~/.superbrain so an unattributed last-failure.txt line could have been\nwritten by any version; read-and-clear semantics meant each failure\ndestroyed the previous one; and SessionStart appended 'set\nANTHROPIC_API_KEY' to EVERY failure (index errors included), steering\ndiagnosis toward auth/quota.\n\n- writeFailure() now stamps every line with the writing plugin's package\n version ([vX.Y.Z], resolved via pluginRoot()).\n- Every failure is also appended to ~/.superbrain/failures.log, windowed\n to the last 100 lines (same pattern as the reject-queue cap); the log\n is never read-and-cleared, so history survives the sentinel clear.\n- sb-session-start only appends the ANTHROPIC_API_KEY hint for distill\n failures.\n\nTests (red first, tests/sentinelHygiene.test.ts): version stamp present;\nhistory appends and stays bounded; hint scoped (index failure output has\nno ANTHROPIC_API_KEY, distill failure keeps it); and a capture-failure\nsurfacing E2E â�� vec_chunks corrupted to int8[8], distill entrypoint run\nwith a stub envelope: the note .md is still written, exit 0, and the\nsentinel contains 'index failed' with the version stamp.\n\nCo-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant