Skip to content

fix(composer): classify a Unicode-space-separated prompt glyph as an empty composer - #1995

Open
mackcee wants to merge 4 commits into
kunchenguid:mainfrom
mackcee:fm/fm-composer-read-unreliable
Open

fix(composer): classify a Unicode-space-separated prompt glyph as an empty composer#1995
mackcee wants to merge 4 commits into
kunchenguid:mainfrom
mackcee:fm/fm-composer-read-unreliable

Conversation

@mackcee

@mackcee mackcee commented Aug 9, 2026

Copy link
Copy Markdown

Intent

Fix a deterministic composer misclassification: fm_backend_composer_state returns pending for a claude composer that is affirmatively empty. pending means "real unsubmitted text is sitting there", so every caller that must not type over the captain's half-written message refuses to act. Observed cost: bin/fm-supervise-daemon.sh deferred every away-mode escalation for 8.3 hours overnight (2105 deferrals reading "composer not confirmed-empty" against only 3 genuine "pane busy"), leaving two finished pieces of work unshipped; bin/fm-send.sh reported "text not submitted (delivery unconfirmed; verdict=pending)" for a message that had visibly landed, inviting a duplicate steer.

ROOT CAUSE (accepted, and it SUPERSEDES the two hypotheses the original task brief proposed - the brief blamed either the prompt glyph's truecolor luminance sitting above the 128 ghost threshold, or the input area being drawn with horizontal rules instead of a boxed border; firstmate later instructed me to discard both). Claude renders the idle composer as U+276F followed by U+00A0 NO-BREAK SPACE. fm_composer_classify_content accepted a bare glyph and glyph+ASCII-space as empty, but not glyph+U+00A0, which fell through to pending.

The brief required establishing the cause by counterfactual rather than by reading the code and forming an opinion, changing one condition at a time and recording whether the verdict moved. I did that against the live panes and both original hypotheses were falsified: recolouring the glyph (including to the exact truecolor luminance suspected) did NOT move the verdict, and deleting the horizontal rules did NOT move it, while changing ONLY the separator did. It reproduces identically on tmux and herdr; on tmux the row is nothing but the glyph and the separator, and there the glyph is a 256-colour foreground that is never luminance-tested at all. Upstream filed the same finding as #1988 with the same proof; the fix is shaped to close it, and its "Suggested directions" 1 and 2 (normalise Unicode spaces generally rather than enumerating glyph+separator pairs; add a regression pinning glyph+U+00A0 to empty) are implemented.

REQUIRED APPROACH. Prefer normalising Unicode whitespace generally over enumerating more glyph+separator pairs, and fix the single-character strip too: ${content#??}/${content#?} assumed a one-character separator, which breaks on any multi-byte separator independently of the match (measured: it drops one BYTE under LC_ALL=C and one CHARACTER under a UTF-8 locale, leaving locale-dependent trailing bytes on a multibyte glyph). Distinguish the composer's own decoration from user content using the most structural signal available rather than another brightness number, and do not let a single vendor string be load-bearing - a rendered glyph and its colour are exactly what a harness release can change without warning.

HARD CONSTRAINT, deliberately chosen and central to review: do NOT weaken the predicate. This guard's entire purpose is to never type over text a human has half-written. empty must remain an affirmative verdict, real typed text MUST still read pending, and unknown must stay available for genuinely unreadable panes. Making it too permissive would make firstmate silently destroy the captain's unsent input, which is a worse and silent failure than the one being fixed. So the fix follows the Unicode White_Space=Yes property (a standards fact) rather than a brightness threshold or another observed string; whitespace-only content already classified empty when the whitespace was ASCII, and any non-whitespace character anywhere still classifies pending. The bare dead-shell-prompt refusal is unchanged with or without a separator.

DELIBERATE DECISIONS a reviewer reading only the diff would not know:

  • U+200B ZERO WIDTH SPACE is deliberately EXCLUDED from normalization because Unicode gives it White_Space=No; including it would substitute my own guess for the property the owner claims to follow. This boundary is pinned by a test asserting U+200B still reads pending.
  • The prompt glyphs were spelled out in three separate places, which bin/fm-composer-lib.sh's own header already flagged as a drift hazard ("must be listed in ALL THREE places"); they are now declared exactly once each and reached through helpers. This is behaviour-preserving.
  • The shared normalizing trim is applied in all four adapters (tmux, herdr, orca, cmux) so a non-ASCII space cannot defeat one adapter's structural row scan while another's still matches.
  • Normalization substitutes a space rather than deleting, so "foobar" is not silently joined into one token.
  • The live guard types its OWN marker, confirms the harness rendered it, clears it with C-u, and confirms it is gone before reading the empty composer - so empty is affirmative rather than assumed. It accepts a startup trust dialog with Enter ONLY after the pane has demonstrably not reflected typed input, so it can never submit a real composer or consume model tokens. It reports an absent harness explicitly and refuses a pass that checked nothing.

TESTS, both required by .agents/skills/firstmate-coding-guidelines "Harness-dependent checks":

  • A portable regression under tests/ extending existing scripts rather than adding a runner, pinning the real captured row shapes: the empty composer that must read empty, and a genuinely-typed-into one that must still read pending (the second matters more than the first). Because U+00A0 is invisible in any capture, the fixtures assert they still contain the separator so no case can pass vacuously. Also pins the whole White_Space property set and locale independence.
  • The env-gated, self-skipping live guard in the live-harness-optin family exercising every INSTALLED harness for real and failing by name and version, since a stub can only confirm the assumption written into it.
    Verified against real claude 2.1.226 on tmux 3.5a: the empty row is e2 9d af c2 a0, and both the portable regression and the live guard fail on that exact row when the fix is reverted. The dated per-harness result is recorded in docs/verification/runtime-backends.md as required.

SCOPE, explicitly bounded by the brief and reaffirmed by firstmate: the composer-state read and its callers only. Do NOT restructure away mode, do NOT change the wake queue, and do NOT touch the Stop-hook watcher path (that path works and is currently the only reliable way firstmate gets woken). Upstream issue 1988's directions 3 and 4 are away-mode changes and are deliberately NOT implemented here.

A third reported symptom - fm-send.sh --resolve-key <key> refusing with "no open decision or blocker with that key" while the wake drain listed that key open - was to be treated as possibly a SEPARATE defect and not folded in silently. I established it IS separate: a status line written as needs-decision: [key=slug] ... puts the key token AFTER the colon, while _fm_decision_key only parses it BEFORE the colon, so the decision silently degrades to key default. Different subsystem and different mechanism from the composer bug, and it already has its own task and branch (fm/fm-decision-key-position). It is deliberately excluded from this change.

Known pre-existing failures NOT caused by this change, each verified to reproduce on the unmodified tree: tests/fm-afk-inject-herdr-e2e.test.sh fails under LC_ALL=C with the fix reverted too; tests/fm-lint.test.sh needs the pinned shellcheck on PATH; tests/fm-test-run.test.sh needs LC_ALL=C collation. With the pinned shellcheck and LC_ALL=C, the full changed-selection suite is 71 scripts with only that pre-existing away-mode E2E failing.

What Changed

  • Added fm_composer_normalize_spaces_var / fm_composer_normalize_trim_var to bin/fm-composer-lib.sh, mapping every non-ASCII code point with the Unicode White_Space=Yes property (U+0085, U+00A0, U+1680, U+2000–U+200A, U+2028, U+2029, U+202F, U+205F, U+3000 — deliberately not U+200B, which is White_Space=No) onto an ASCII space before every trim, border strip, and glyph comparison, so claude's empty row + U+00A0 now classifies empty instead of pending. The predicate is not otherwise loosened: any non-whitespace character still reads pending, and a bare shell glyph still reads unknown. The same shared trim replaces the open-coded ASCII-only trims in the tmux, herdr, orca, and cmux structural row scans, where the same separator made a bordered composer read unknown.
  • Consolidated the prompt glyphs into single declarations (FM_COMPOSER_AGENT_PROMPT_GLYPHS, FM_COMPOSER_SHELL_PROMPT_GLYPHS) reached via fm_composer_leading_prompt_glyph_var / fm_composer_blank_leading_prompt_var, replacing the three inline copies in the classifier plus the drifted copy in fm_tmux_composer_geometry_spaces (which omitted muse's ). The glyph strip now removes the matched literal instead of ${content#?}/${content#??}, which dropped one byte under LC_ALL=C and one character under a UTF-8 locale.
  • Added regression coverage pinning the captured e2 9d af c2 a0 row to empty and typed input to pending (tests/fm-composer-lib.test.sh, tests/fm-composer-ghost.test.sh, tests/fm-backend-herdr.test.sh, with fixtures asserting the separator is present so no case passes vacuously), plus a new env-gated live guard tests/fm-composer-harness-drift-live-e2e.test.sh that types and clears its own marker before reading each installed harness; bin/fm-test-run.sh routes bin/fm-composer* and bin/fm-tmux-lib.sh changes to the live-harness-optin family. Backend docs, the harness-adapters skill, and docs/verification/runtime-backends.md record the whitespace boundary and the dated per-harness table (claude 2.1.226 on tmux 3.5a; every uninstalled harness listed as unverified).

Risk Assessment

✅ Low: The change is well-bounded to the composer-state read and its four adapters, the predicate is verifiably not weakened in the dangerous direction (bare dead-shell prompts, ZWSP, and real typed text all still refuse), the Unicode table matches the standard property exactly, both fix rounds' claims check out against the current code, and the only outstanding item is a test-enumeration gap over data I verified correct.

Testing

I proved the intent end-to-end against the real harness rather than only in fixtures: driving bin/fm-supervise-daemon.sh's actual away-mode escalation guard against a live claude 2.1.226 composer on tmux 3.5a reproduced the reported "composer not confirmed-empty (state=pending)" deferral on the reverted classifier and showed the escalation accepted on the fix, against a composer the demo had itself emptied and whose bytes decode to e2 9d af c2 a0; the hard constraint holds on the same live pane, since real unsubmitted text still reads pending and still defers. The opt-in live drift guard passes on the fix and fails by name and version when reverted, and reports the seven uninstalled harnesses and the uncovered bordered path rather than reading as full coverage. All portable suites for the changed classifier and the four adapters pass, each new regression fails on base sources in both C and UTF-8 locales, and the test-runner change correctly selects 71 scripts including the new live guard. The one failure seen, the away-mode herdr E2E under LC_ALL=C, reproduces identically on the unmodified base tree and passes here in a UTF-8 locale, so it is pre-existing and not caused by this change. No visual artifact applies: the affected surface is a terminal composer read and a daemon log line, both captured as CLI transcripts, and the defect is invisible in any rendered capture because U+00A0 only appears once the bytes are decoded.

Evidence: Away-mode escalation guard against a live claude composer: before/after the fix, plus the safety half

== BEFORE THE FIX (composer classifier reverted to base commit 2d2be63) == -- claude version: 2.1.226 (Claude Code) -- marker cleared; the composer is now affirmatively EMPTY. Its raw bytes: e2 9d af c2 a0 (e2 9d af = U+276F '\xe2\x9d\xaf' ; c2 a0 = U+00A0 NO-BREAK SPACE) -- what fm_backend_composer_state reads off that live pane: pending -- running the real inject_msg() away-mode escalation guard: inject_msg -> DEFERRED (escalation withheld) -- daemon log: [2026-08-09T06:40:02+0000] inject deferred: supervisor composer not confirmed-empty (state=pending: pending input, dead-shell prompt, or unreadable pane) == AFTER THE FIX (worktree at deb3692) == -- what fm_backend_composer_state reads off that live pane: empty -- running the real inject_msg() away-mode escalation guard: inject_msg -> ACCEPTED (escalation would be delivered to the captain) -- daemon log: == AFTER THE FIX - SAFETY HALF: real unsubmitted text still blocks injection == -- marker deliberately LEFT in the composer (unsubmitted human text). Its raw bytes: e2 9d af c2 a0 7a 71 78 66 6d 67 75 61 72 64 64 65 6d 6f -- what fm_backend_composer_state reads off that live pane: pending -- running the real inject_msg() away-mode escalation guard: inject_msg -> DEFERRED (escalation withheld) -- daemon log: [2026-08-09T06:41:09+0000] inject deferred: supervisor composer not confirmed-empty (state=pending: pending input, dead-shell prompt, or unreadable pane)

### Away-mode escalation guard against a REAL claude 2.1.226 composer on tmux 3.5a
### (bin/fm-supervise-daemon.sh inject_msg; only the final send is stubbed, so no tokens are spent)

== BEFORE THE FIX (composer classifier reverted to base commit 2d2be63) ==
-- claude version: 2.1.226 (Claude Code)
-- pane with our typed marker still in the composer (the 'human is half-way through' case):
     ❯ zqxfmguarddemo
-- marker cleared; the composer is now affirmatively EMPTY. Its raw bytes:
      e2 9d af c2 a0 
     (e2 9d af = U+276F '\xe2\x9d\xaf' ; c2 a0 = U+00A0 NO-BREAK SPACE)
-- what fm_backend_composer_state reads off that live pane:
     pending
-- running the real inject_msg() away-mode escalation guard:
     inject_msg -> DEFERRED (escalation withheld)
-- daemon log:
     [2026-08-09T06:40:02+0000] inject deferred: supervisor composer not confirmed-empty (state=pending: pending input, dead-shell prompt, or unreadable pane)

== AFTER THE FIX (worktree at deb3692) ==
-- claude version: 2.1.226 (Claude Code)
-- pane with our typed marker still in the composer (the 'human is half-way through' case):
     ❯ zqxfmguarddemo
-- marker cleared; the composer is now affirmatively EMPTY. Its raw bytes:
      e2 9d af c2 a0 
     (e2 9d af = U+276F '\xe2\x9d\xaf' ; c2 a0 = U+00A0 NO-BREAK SPACE)
-- what fm_backend_composer_state reads off that live pane:
     empty
-- running the real inject_msg() away-mode escalation guard:
     inject_msg -> ACCEPTED (escalation would be delivered to the captain)
-- daemon log:

== AFTER THE FIX - SAFETY HALF: real unsubmitted text still blocks injection ==
-- claude version: 2.1.226 (Claude Code)
-- pane with our typed marker still in the composer (the 'human is half-way through' case):
     ❯ zqxfmguarddemo
-- marker deliberately LEFT in the composer (unsubmitted human text). Its raw bytes:
      e2 9d af c2 a0 7a 71 78 66 6d 67 75 61 72 64 64 65 6d 6f 
     (e2 9d af = U+276F '\xe2\x9d\xaf' ; c2 a0 = U+00A0 NO-BREAK SPACE)
-- what fm_backend_composer_state reads off that live pane:
     pending
-- running the real inject_msg() away-mode escalation guard:
     inject_msg -> DEFERRED (escalation withheld)
-- daemon log:
     [2026-08-09T06:41:09+0000] inject deferred: supervisor composer not confirmed-empty (state=pending: pending input, dead-shell prompt, or unreadable pane)
Evidence: Live harness drift guard on the fix (real claude 2.1.226, tmux 3.5a)

# claude 2.1.226 (Claude Code): startup row bytes 20 e2 9d af 20 31 2e 20 59 65 73 2c 20 49 20 74 72 75 73 74 20 74 68 69 73 20 66 6f 6c 64 65 72 # claude 2.1.226 (Claude Code): typed input was not rendered back; accepting a startup confirmation with Enter and retrying # claude 2.1.226 (Claude Code): composer reached through the bare-row path # claude 2.1.226 (Claude Code): EMPTY composer row bytes e2 9d af c2 a0 ok - composer drift: claude 2.1.226 (Claude Code) reads pending with real typed text and empty once cleared (via the bare-row path) # not installed on this machine: codex opencode pi pi-signed grok kimi muse # checked 1 installed harness(es) end to end # structural paths exercised: bare-row # no installed harness drew a BORDERED composer box, so the bordered structural path is unverified by this run (tests/fm-composer-ghost.test.sh pins it portably)

# claude 2.1.226 (Claude Code): startup row bytes  20 e2 9d af 20 31 2e 20 59 65 73 2c 20 49 20 74 72 75 73 74 20 74 68 69 73 20 66 6f 6c 64 65 72 
# claude 2.1.226 (Claude Code): typed input was not rendered back; accepting a startup confirmation with Enter and retrying
# claude 2.1.226 (Claude Code): composer reached through the bare-row path
# claude 2.1.226 (Claude Code): EMPTY composer row bytes  e2 9d af c2 a0 
ok - composer drift: claude 2.1.226 (Claude Code) reads pending with real typed text and empty once cleared (via the bare-row path)
# skip: codex is not installed on this machine, so its composer rendering is unverified here
# skip: opencode is not installed on this machine, so its composer rendering is unverified here
# skip: pi is not installed on this machine, so its composer rendering is unverified here
# skip: pi-signed is not installed on this machine, so its composer rendering is unverified here
# skip: grok is not installed on this machine, so its composer rendering is unverified here
# skip: kimi is not installed on this machine, so its composer rendering is unverified here
# skip: muse is not installed on this machine, so its composer rendering is unverified here
# not installed on this machine: codex opencode pi pi-signed grok kimi muse
# checked 1 installed harness(es) end to end
# structural paths exercised: bare-row
# no installed harness drew a BORDERED composer box, so the bordered structural path is unverified by this run (tests/fm-composer-ghost.test.sh pins it portably)
Evidence: Same live guard against base-commit sources — fails by name and version

not ok - COMPOSER DRIFT: claude 2.1.226 (Claude Code) has a composer this guard just emptied - the typed marker is gone from the rendered pane - yet it classifies 'pending' instead of 'empty', reached through the bare-row path. ... Decoded composer row: e2 9d af c2 a0 (U+00A0 NO-BREAK SPACE is the byte pair c2 a0 ...)

# claude 2.1.226 (Claude Code): startup row bytes  20 e2 9d af 20 31 2e 20 59 65 73 2c 20 49 20 74 72 75 73 74 20 74 68 69 73 20 66 6f 6c 64 65 72 
# claude 2.1.226 (Claude Code): typed input was not rendered back; accepting a startup confirmation with Enter and retrying
# claude 2.1.226 (Claude Code): composer reached through the bare-row path
not ok - COMPOSER DRIFT: claude 2.1.226 (Claude Code) has a composer this guard just emptied - the typed marker is gone from the rendered pane - yet it classifies 'pending' instead of 'empty', reached through the bare-row path. Every caller that must not overwrite unsubmitted input now refuses to act against this harness: away-mode escalations stop being delivered and steer confirmations report landed messages as unconfirmed. On the bare-row path a 'pending' verdict means the separator or prompt glyph this release renders is not one bin/fm-composer-lib.sh recognises (bare-row drift usually reads 'pending'; a bordered box whose blanked geometry no longer matches its border reads 'unknown'). Decoded composer row:  e2 9d af c2 a0  (U+00A0 NO-BREAK SPACE is the byte pair c2 a0; fm_composer_normalize_trim_var maps every Unicode White_Space=Yes code point, so a byte run outside that property is the thing to add).
Evidence: Harness script used for the live away-mode guard demo (evidence-only, outside the worktree)
#!/usr/bin/env bash
# Product-level demo: drive bin/fm-supervise-daemon.sh's real away-mode
# escalation guard (inject_msg) against a REAL, live, affirmatively-empty
# claude composer in a private tmux server, and show the daemon log line the
# captain would actually read.
#
# The only thing stubbed is the final send (fm_backend_send_text_submit), so no
# text is ever typed into or submitted to the real harness and no model tokens
# are consumed. Everything before it - pane-exists, busy-guard, composer-guard -
# is the real product code reading the real pane.
#
# usage: daemon-guard-demo.sh <tree-root> <label>
set -u
ROOT=$1
LABEL=$2

REAL_TMUX=$(command -v tmux)
SOCKET="fm-guard-demo-$$"
LAB=$(mktemp -d "${TMPDIR:-/tmp}/fm-guard-demo.XXXXXX")
SESSION=demo
cleanup() { "$REAL_TMUX" -L "$SOCKET" kill-server >/dev/null 2>&1 || true; rm -rf "$LAB"; }
trap cleanup EXIT

mkdir -p "$LAB/shim" "$LAB/wt" "$LAB/state"
cat > "$LAB/shim/tmux" <<SH
#!/usr/bin/env bash
exec "$REAL_TMUX" -L "$SOCKET" "\$@"
SH
chmod +x "$LAB/shim/tmux"
PATH="$LAB/shim:$PATH"; export PATH

MARKER=zqxfmguarddemo
TARGET="$SESSION:claude"

"$REAL_TMUX" -L "$SOCKET" new-session -d -s "$SESSION" -n control -c "$LAB/wt"
"$REAL_TMUX" -L "$SOCKET" new-window -d -t "$SESSION:" -n claude -c "$LAB/wt" -- "$(command -v claude)"
sleep 6

pane_has() { "$REAL_TMUX" -L "$SOCKET" capture-pane -p -t "$TARGET" 2>/dev/null | LC_ALL=C grep -qa "$MARKER"; }

# Prove the pane is really at a composer by typing our OWN marker and seeing the
# harness render it back; a startup trust dialog never takes it.
at_composer=0
for _ in 1 2 3; do
  "$REAL_TMUX" -L "$SOCKET" send-keys -l -t "$TARGET" "$MARKER"
  for _ in $(seq 1 20); do pane_has && { at_composer=1; break; }; sleep 0.5; done
  [ "$at_composer" = 1 ] && break
  "$REAL_TMUX" -L "$SOCKET" send-keys -t "$TARGET" C-u >/dev/null 2>&1 || true
  "$REAL_TMUX" -L "$SOCKET" send-keys -t "$TARGET" Enter >/dev/null 2>&1 || true
  sleep 4
done
[ "$at_composer" = 1 ] || { echo "ABORT: never reached a real claude composer"; exit 2; }

echo "== $LABEL =="
echo "-- claude version: $(claude --version 2>/dev/null | head -1)"
echo "-- pane with our typed marker still in the composer (the 'human is half-way through' case):"
"$REAL_TMUX" -L "$SOCKET" capture-pane -p -t "$TARGET" | grep -a "$MARKER" | tail -1 | sed 's/^/     /'

# Clear it, and confirm it is GONE from the rendered pane: the composer is now
# affirmatively empty, because we just emptied it. With CLEAR=0 the marker is
# deliberately LEFT in place, standing in for a captain's half-written message:
# the guard must still refuse, or firstmate would type over it.
if [ "${CLEAR:-1}" = 1 ]; then
  "$REAL_TMUX" -L "$SOCKET" send-keys -t "$TARGET" C-u >/dev/null 2>&1 || true
  cleared=0
  for _ in $(seq 1 40); do pane_has || { cleared=1; break; }; sleep 0.5; done
  [ "$cleared" = 1 ] || { echo "ABORT: composer did not clear"; exit 2; }
  sleep 2
  echo "-- marker cleared; the composer is now affirmatively EMPTY. Its raw bytes:"
else
  pane_has || { echo "ABORT: the marker vanished on its own"; exit 2; }
  echo "-- marker deliberately LEFT in the composer (unsubmitted human text). Its raw bytes:"
fi

row=$("$REAL_TMUX" -L "$SOCKET" capture-pane -p -t "$TARGET" | grep -a "$(printf '\xe2\x9d\xaf')" | tail -1)
printf '%s' "$row" | LC_ALL=C od -An -tx1 | tr -s ' \n' ' ' | sed 's/^/     /'
echo ""
echo "     (e2 9d af = U+276F '\xe2\x9d\xaf' ; c2 a0 = U+00A0 NO-BREAK SPACE)"

# Now the real daemon guard.
# shellcheck source=/dev/null
. "$ROOT/bin/fm-supervise-daemon.sh"
LOG="$LAB/daemon.log"; : > "$LOG"
FM_STATE_OVERRIDE="$LAB/state"; export FM_STATE_OVERRIDE
afk_enter "$LAB/state"
FM_SUPERVISOR_BACKEND=tmux
FM_SUPERVISOR_TARGET="$TARGET"
# Stub ONLY the final send so the demo never types into or submits to the real
# harness. Reaching this stub at all means every guard ahead of it passed.
fm_backend_send_text_submit() { echo "STUBBED-SEND" >&2; printf 'empty'; }

echo "-- what fm_backend_composer_state reads off that live pane:"
echo "     $(fm_backend_composer_state tmux "$TARGET")"
echo "-- running the real inject_msg() away-mode escalation guard:"
if inject_msg "two finished pieces of work are unshipped, come look" 2>/dev/null; then
  echo "     inject_msg -> ACCEPTED (escalation would be delivered to the captain)"
else
  echo "     inject_msg -> DEFERRED (escalation withheld)"
fi
echo "-- daemon log:"
sed 's/^/     /' "$LOG"

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • ⚠️ bin/fm-tmux-lib.sh:190 - Intent conformance: the criteria state "The shared normalizing trim is applied in all four adapters (tmux, herdr, orca, cmux) so a non-ASCII space cannot defeat one adapter's structural row scan while another's still matches." For tmux that is not the case. The trim was applied to fm_tmux_composer_row_state (lines 132/134/141), but tmux's structural row scan - which bin/fm-composer-lib.sh's own header names as "tmux's visible-pane box scan" - still uses the raw ASCII-only trim at fm-tmux-lib.sh:190-192, and fm_tmux_composer_geometry_spaces:166/172-175 still tests the leftover bytes with [![:space:]]. Verified read-only against the changed code: fm_tmux_find_composer_box 1 with pane rows '╭────────╮' / '│ ❯<U+00A0> │' / '╰────────╯' returns "0 2 1" (geometry_ambiguous=1), while the identical pane with an ASCII space returns "0 2 0". fm_tmux_composer_state:342 converts that flag into 'unknown' even though fm_tmux_composer_row_state classifies the row 'empty' - so a bordered tmux composer separated with U+00A0 still never reads confirmed-empty, and every caller that must not overwrite unsubmitted input still refuses to act. That is the exact authorized failure this change fixes, surviving on the sibling path in the same file. herdr's ANSI tail scan and orca/cmux's read-screen scans were normalized, so tmux is precisely the "one adapter defeated while another matches" case the criterion says cannot happen. Earliest supported shared boundary: run each captured line through fm_composer_normalize_trim inside fm_tmux_find_composer_box's loop (and in fm_tmux_composer_geometry_spaces' probe) instead of the open-coded ${line#"${line%%[![:space:]]*}"} pair. Not proven live-reachable for claude today (its tmux composer is a bare glyph row, not a box), which is why this is a warning rather than an error - but it contradicts a stated deliberate coverage claim, so it is the author's call whether to close it here or as a follow-up.
  • ℹ️ bin/fm-composer-lib.sh:140 - _fm_composer_normalize_space_var is documented as replacing in place "so the wrapper below adds no subshell", but fm_composer_normalize_trim prints to stdout and all 12 call sites consume it via command substitution, so every call forks anyway - plus one heredoc temp file per call for the 19-entry mapping loop. The row loops are the cost: bin/backends/orca.sh:276 runs this once per captured line with FM_BACKEND_ORCA_COMPOSER_LINES defaulting to 200, where the previous code was pure parameter expansion with zero forks. Measured on this worktree: 200 command substitutions of fm_composer_normalize_trim take ~72ms versus ~1ms for the parameter-expansion form, per composer read, and composer_state is polled by bin/fm-supervise-daemon.sh and re-read inside the send-submit retry loops. Absolute cost is small, so this is informational: exposing the existing in-place form as a public fm_composer_normalize_trim_var <varname> and using it in the orca/cmux/herdr row loops would deliver the design the comment already describes.
  • ℹ️ bin/fm-tmux-lib.sh:169 - The change's stated deliberate decision is that the prompt glyphs "are now declared exactly once each and reached through helpers", and FM_COMPOSER_AGENT_PROMPT_GLYPHS does that for the classifier. fm_tmux_composer_geometry_spaces is a fourth place that still spells them out inline ('>' , '❯', '›') and is already drifted: it omits '⟩' (muse, U+27E9), which the classifier does recognise. Consequence on the bordered tmux path: a muse composer's content row keeps its multibyte glyph, sed 's/[!-~]/ /g' cannot flatten it, the [![:space:]] test at line 174 fires, geometry_spaces returns 1, and fm_tmux_find_composer_box sets geometry_ambiguous=1 - so an otherwise-empty muse box reads 'unknown' rather than 'empty'. Pre-existing rather than introduced here, and it is the same drift hazard the header comment was rewritten to eliminate; reaching FM_COMPOSER_AGENT_PROMPT_GLYPHS from this helper would close it.
  • ℹ️ bin/backends/herdr.sh:2708 - fm_backend_herdr_pi_separator_row still uses the open-coded ASCII-only trim and then requires the remainder be nothing but '─' ([ -z "${row//─/}" ]). A Pi rule padded or terminated with any non-ASCII space fails that test, so no complete separator pair is found, fm_backend_herdr_composer_state's 'separated' shape is never established, and the verdict degrades to 'unknown' - the same refuse-to-act outcome, reached through the structural half of the herdr adapter that the row loop at line 2782 normalized but this helper did not. No evidence any Pi release pads its rules that way today, so this is informational; routing this trim through fm_composer_normalize_trim like its sibling would make the herdr adapter internally consistent.

🔧 Fix: normalize Unicode spaces and glyphs in tmux/herdr structural scans
3 infos still open:

  • ℹ️ bin/fm-composer-lib.sh:31 - The fix round added a header claim that is not accurate: "An ADAPTER that needs to recognise a prompt glyph in its own structural scan - tmux's box-geometry probe is the one such caller - reaches the same declarations ... rather than becoming a fourth copy" (bin/fm-composer-lib.sh:30-35). herdr's adapter has a second such scan: FM_BACKEND_HERDR_BARE_PROMPT_RE at bin/backends/herdr.sh:2700 is '^(❯|›)', spelling out two of the three agent glyphs inline and omitting muse's '⟩'. It is the sole gate for the bare shape in fm_backend_herdr_composer_state (herdr.sh:2793-2799): a row that is neither '│''│' nor matched by that regex leaves found=0 and the verdict is unknown. So this is a second declaration of ❯ and ›, which also contradicts the accepted intent's deliberate decision that the glyphs "are now declared exactly once each and reached through helpers". I could NOT establish that the failure is live-reachable, and I am reporting that honestly rather than asserting it: the repo's two records disagree on muse's composer shape. .agents/skills/harness-adapters/SKILL.md:419 says "Bordered box whose prompt glyph is ", in which case herdr's '│''│' case matches first and this regex never matters; docs/verification/muse.md:150-151 shows the actual tmux capture-pane -p -e rows as a border-free line, in which case an empty muse composer on herdr never establishes a shape and reads unknown while the same pane on tmux reads empty. The code is pre-existing and unchanged by this diff (its alternation form has a documented locale rationale at herdr.sh:2691-2699, so it cannot simply become a bracket class), and it was not in the round-1 findings or the fix instructions. Author's call: either reach FM_COMPOSER_AGENT_PROMPT_GLYPHS when building that regex and reconcile the two muse records, or narrow the new header claim so it does not assert a uniqueness the tree does not have.
  • ℹ️ bin/fm-composer-lib.sh:169 - fm_composer_normalize_trim (the value-returning form) has zero callers after the fix round converted all twelve call sites to fm_composer_normalize_trim_var - I grepped every *.sh and *.md in the tree and the only hits are its own definition and the doc line in docs/verification/runtime-backends.md:184, which names the _var form. It is a public, untested function in a shared library, so nothing would catch it drifting from the _var form it wraps. Either drop it, or keep it and note it is currently unused; the comment at :165-168 already explains it is deliberately not the default, so a reader has no way to tell "reserved for future callers" from "leftover".
  • ℹ️ bin/fm-composer-lib.sh:403 - fm_composer_classify_content still does content=$(_fm_composer_strip_leading_prompt &#34;$content&#34;), a command substitution, immediately after the same file's comment at :165-168 argues the value-returning form is not the default precisely because "consuming it costs a subshell per call". _fm_composer_strip_leading_prompt is now a thin wrapper over fm_composer_leading_prompt_glyph_var, so the fork is removable in place with no new mapping and no behavior change: if fm_composer_leading_prompt_glyph_var glyph &#34;$content&#34;; then content=${content#*&#34;$glyph&#34;}; fi. Cost is small and bounded - the early returns above mean this line is only reached for rows that carry real content, and a tmux content row already forks three times (strip_ansi, strip_ghost, and the per-row sed -n Np in fm_tmux_composer_state:338) - so this is efficiency tidying, not a regression. Verified equivalence of the substitution while reviewing: fm_composer_leading_prompt_glyph_var only matches a glyph at the first non-whitespace position and classify_content trims before this line, so #*&#34;$glyph&#34; can never strip past the intended glyph.

🔧 Fix: drop dead trim wrapper, inline glyph strip, correct header claim
1 info still open:

  • ℹ️ tests/fm-composer-lib.test.sh:163 - test_unicode_whitespace_property_set_is_covered independently spells out 11 separators ('\302\205' 00A0 1680 2000 2007 200A 2028 2029 202F 205F 3000) but the implementation's FM_COMPOSER_UNICODE_SPACES holds 19. The eight unpinned entries are U+2001-U+2006, U+2008 and U+2009 - the interior of the one contiguous run in the table, and the run most exposed to a single-digit slip because the table is hand-transcribed octal ('\0342\0200\0201' .. '\0342\0200\0212' at bin/fm-composer-lib.sh:141-143). I decoded all 19 entries at runtime and they are byte-correct today, so this is a coverage gap and not a live defect; the point is that a future edit dropping or mistyping U+2003 would not fail anything, because the only other test that touches the set (test_every_declared_prompt_glyph_is_blankable_and_classified) iterates the glyph declarations, not the space table, and iterating the table against itself would pass with a wrong byte sequence in it. The intent states the portable regression 'pins the whole White_Space property set'; extending the existing literal loop to all 19 escapes (the assertions already in the test body need no change) is what makes that true. Independent literal spelling is the right shape here - do not replace it with a loop over FM_COMPOSER_UNICODE_SPACES, which would be self-referential.
✅ **Test** - passed

✅ No issues found.

  • FM_COMPOSER_HARNESS_DRIFT=1 bash tests/fm-composer-harness-drift-live-e2e.test.sh — live guard, real claude 2.1.226 on tmux 3.5a: passes (empty row e2 9d af c2 a0, bare-row path); same guard against base-commit sources fails with a named COMPOSER DRIFT error
  • Manual end-to-end: drove the real inject_msg() away-mode escalation guard in bin/fm-supervise-daemon.sh against a live claude composer emptied by the harness itself (marker typed, rendered back, cleared with C-u, confirmed gone), with only fm_backend_send_text_submit stubbed so no text was submitted and no model tokens spent — reproduced the reported inject deferred: supervisor composer not confirmed-empty (state=pending...) on reverted sources and ACCEPTED on the fix
  • Manual end-to-end safety half: same live pane with real unsubmitted text left in the composer — fm_backend_composer_state reads pending and inject_msg still DEFERS
  • LC_ALL=C bash tests/fm-composer-lib.test.sh (13 ok; White_Space property set, U+200B stays pending, locale independence across C/C.utf8/en_US.utf8, dead-shell refusal, all 7 declared glyphs blankable)
  • LC_ALL=C bash tests/fm-composer-ghost.test.sh (bordered + unbordered tmux paths, muse , ambiguous geometry still unknown)
  • LC_ALL=C bash tests/fm-backend-herdr.test.sh (new +U+00A0 empty/pending cases and U+00A0-padded Pi separator cases)
  • Revert proof: ran tests/fm-composer-lib.test.sh and tests/fm-composer-ghost.test.sh against a tree with bin/fm-composer-lib.sh, bin/fm-tmux-lib.sh and the three adapters restored to 2d2be63 — both fail (got &#39;pending&#39;) in C and C.UTF-8
  • LC_ALL=C bash tests/fm-backend-orca.test.sh, tests/fm-backend-cmux.test.sh, tests/fm-backend.test.sh, tests/fm-test-run.test.sh — all pass
  • LC_ALL=C bash bin/fm-test-run.sh --changed --base 2d2be63 --list — selects 71 scripts and includes tests/fm-composer-harness-drift-live-e2e.test.sh, confirming the changed-file map wiring
  • LC_ALL=C bash tests/fm-afk-inject-herdr-e2e.test.sh on this tree and on a clean 2d2be63 checkout — fails identically on both (pre-existing); passes on this tree in the default UTF-8 locale
🔧 **Document** - 1 issue found → auto-fixed ✅
  • ⚠️ .agents/skills/harness-adapters/SKILL.md:419 - Two doc surfaces still contradict each other on muse's composer shape, and I could not resolve it here because muse is not installed on this machine. .agents/skills/harness-adapters/SKILL.md:419 records muse's composer as a "Bordered box whose prompt glyph is (U+27E9)", while docs/verification/muse.md:150-151 shows a captured row with no box-drawing border - just plus an ASCII space with a background fill. Which one is true decides whether herdr's FM_BACKEND_HERDR_BARE_PROMPT_RE (which spells out and and omits ) is reachable at all: if muse draws a bare row, an empty muse composer reads unknown on herdr while the same pane reads empty on tmux. This change's new comment in bin/fm-composer-lib.sh names the contradiction and defers the permissive regex fix to its own task; the doc side needs one live muse capture to settle which surface is stale, then a correction in the wrong one. Follow-up, not fixable in this change.

🔧 Fix: cross-reference muse composer shape contradiction on both surfaces
✅ Re-checked - no issues remain.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@kunchenguid

kunchenguid commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Automated reminder: thanks for the PR! This branch currently has a merge conflict with the base branch.

When you get a chance, please rebase onto (or merge) the latest base branch, resolve the conflict, and push. After that, checks will re-run and the PR will get looked at again.

Noted for firstmate#1995 at ed2c271c.

@mackcee
mackcee force-pushed the fm/fm-composer-read-unreliable branch from ed2c271 to 04b709c Compare August 10, 2026 02:38
@kunchenguid kunchenguid removed the wheelhouse:pending-contributor-action Managed by Wheelhouse label Aug 10, 2026
kunchenguid added a commit that referenced this pull request Aug 10, 2026
…le matrix fixed

Consolidate every composer shape - bordered boxes (all families, geometry,
titled bottom borders), bare agent-glyph rows and their wrap regions,
opencode's left bar, and pi's identity-gated separator pair - into
fm_composer_classify_screen in bin/fm-composer-lib.sh. Adapters now
contribute only a capture and a declarative capability descriptor
(styled/cursor/identity/rows); capability differences change how confidently
a shape is judged, never what the shapes are, so a new harness shape is
teachable in exactly one place.

Correctness fixes landed as part of the consolidation (audit
data/fm-composer-consolidation-audit-s1):
- locale-safe Unicode-space normalization in the shared owner (closes the
  fleet-wide half of #1988; cmux's local byte-exact NBSP case deleted;
  naming converges with PR #1995's normalization primitive)
- muse's bare glyph joins the shared set, unbreaking muse on herdr/cmux/orca
- orca learns the borderless bare shape, drops its backward-paged composer
  window, and can no longer classify a stale startup banner as the composer
- tmux tolerates a titled bottom border, unbreaking grok steering
- the left-bar shape makes opencode readable on every backend
- zellij gets a real classifier through dump-screen --ansi, replacing the
  content-diff submit heuristic that could confirm an undelivered message
  and close a --resolve-key decision (the fleet's only false positive)
- fm-spawn's kimi launch-readiness regex (the fourth shape copy) now routes
  through the shared classifier

The strict blank-row posture applies fleet-wide (captain decision
blank-row-injection-posture): no positive container proof = unknown = defer,
replacing tmux's permissive blank-cursor-row rule. Away-mode injection was
re-validated end to end on real tmux (defer on partial input and unproven
rows, clean delivery with swallowed-Enter retry into proven-empty
composers). The tmux submit core gains a baseline-idle turn-started
conversion so pi steering stays confirmed while its working screen hides
the composer; busy conversion without that baseline remains forbidden.

Plain-capture backends now degrade a glyph row carrying trailing text to
unknown instead of a false pending, per the approved capability rule.

Portable regressions pin the full byte-capture matrix from the audit under
a UTF-8 locale and LC_ALL=C, the strict-vs-permissive divergence, and
deliberate signal separation; the opt-in live guard
(tests/fm-composer-matrix-live-e2e.test.sh) verified every installed
harness against the real classifier, recorded in
docs/verification/runtime-backends.md.
kunchenguid added a commit that referenced this pull request Aug 11, 2026
* refactor(composer): one shape owner behind thin capture adapters, whole matrix fixed

Consolidate every composer shape - bordered boxes (all families, geometry,
titled bottom borders), bare agent-glyph rows and their wrap regions,
opencode's left bar, and pi's identity-gated separator pair - into
fm_composer_classify_screen in bin/fm-composer-lib.sh. Adapters now
contribute only a capture and a declarative capability descriptor
(styled/cursor/identity/rows); capability differences change how confidently
a shape is judged, never what the shapes are, so a new harness shape is
teachable in exactly one place.

Correctness fixes landed as part of the consolidation (audit
data/fm-composer-consolidation-audit-s1):
- locale-safe Unicode-space normalization in the shared owner (closes the
  fleet-wide half of #1988; cmux's local byte-exact NBSP case deleted;
  naming converges with PR #1995's normalization primitive)
- muse's bare glyph joins the shared set, unbreaking muse on herdr/cmux/orca
- orca learns the borderless bare shape, drops its backward-paged composer
  window, and can no longer classify a stale startup banner as the composer
- tmux tolerates a titled bottom border, unbreaking grok steering
- the left-bar shape makes opencode readable on every backend
- zellij gets a real classifier through dump-screen --ansi, replacing the
  content-diff submit heuristic that could confirm an undelivered message
  and close a --resolve-key decision (the fleet's only false positive)
- fm-spawn's kimi launch-readiness regex (the fourth shape copy) now routes
  through the shared classifier

The strict blank-row posture applies fleet-wide (captain decision
blank-row-injection-posture): no positive container proof = unknown = defer,
replacing tmux's permissive blank-cursor-row rule. Away-mode injection was
re-validated end to end on real tmux (defer on partial input and unproven
rows, clean delivery with swallowed-Enter retry into proven-empty
composers). The tmux submit core gains a baseline-idle turn-started
conversion so pi steering stays confirmed while its working screen hides
the composer; busy conversion without that baseline remains forbidden.

Plain-capture backends now degrade a glyph row carrying trailing text to
unknown instead of a false pending, per the approved capability rule.

Portable regressions pin the full byte-capture matrix from the audit under
a UTF-8 locale and LC_ALL=C, the strict-vs-permissive divergence, and
deliberate signal separation; the opt-in live guard
(tests/fm-composer-matrix-live-e2e.test.sh) verified every installed
harness against the real classifier, recorded in
docs/verification/runtime-backends.md.

* no-mistakes(review): Fix Pi glyph ambiguity and complete profile matrix

* no-mistakes(review): Preserve bare verdict when Pi identity probe is absent

* no-mistakes(review): Harden composer structure and titled-border geometry

* no-mistakes(review): Require proven idle baseline and strict Zellij guard

* no-mistakes(review): Reject box bottom borders as composer input rows

* no-mistakes(review): Prove Zellij probe typing before classifier retries

* no-mistakes(review): Preserve Pi identity uncertainty and scan full left-bar drafts

* no-mistakes(review): Verify Zellij text lands before submitting

* no-mistakes(review): Scope Zellij typing verification to selected composer content

* no-mistakes(review): Verify Zellij pastes through composer-scoped content deltas

* no-mistakes(review): Prove wrapped bare Zellij pastes through composer extraction

* no-mistakes(review): Invalidate stale cursorless composers below dead shell prompts

* no-mistakes(review): Handle shell prompt placeholders in composer extraction

* no-mistakes(review): Classify cursorless bare continuation regions safely

* no-mistakes(review): Reject stale cursorless containers below live activity

* no-mistakes(review): Preserve prompt glyphs in wrapped Zellij pastes

* no-mistakes(review): Reject live shell rows during composer extraction

* no-mistakes(review): Preserve wrapped glyph continuations through submit retries

* no-mistakes(review): Scope idle placeholders to proven positions

* no-mistakes(review): Restore boxed placeholders and live prompt reanchoring

* no-mistakes(review): Fix Zellij placeholder and wrapped glyph paste proof

* no-mistakes(document): Align composer architecture documentation

* no-mistakes(lint): Fix ShellCheck warnings in composer refactor

* no-mistakes: apply CI fixes

* docs(verification): record the trusted-checkout live matrix rerun

The pipeline's isolated gate worktree is untrusted, so claude, grok, and
muse stopped at first-launch trust dialogs there (the guard refuses to
confirm them by design). This rerun from the trusted checkout at the final
validated head verified all six installed harnesses, the strict blank-row
deferral, and the hardened zellij false-positive probe live.

* no-mistakes(document): Align composer verification evidence

* no-mistakes: apply CI fixes

* no-mistakes(review): Restore proven box bottom-cursor classification

* no-mistakes(review): Preserve styled placeholder-like drafts as pending

* no-mistakes(document): Align composer safety and Zellij delivery documentation

* no-mistakes: apply CI fixes

* docs(verification): refresh the live matrix with the final-head trusted rerun

The post-validation rerun from the trusted checkout verified all six
installed harnesses at the branch's final head, including Claude 2.1.227
(auto-updated since the audit's captures) and Grok, which the untrusted
gate worktree could not verify past their first-launch trust dialogs.
…y composer

Claude 2.x separates its composer prompt glyph from the composer's content
with U+00A0 NO-BREAK SPACE rather than an ASCII space, so an affirmatively
empty composer renders as exactly `❯` U+00A0. Every trim and glyph
comparison in the shared classifier tested only POSIX `[[:space:]]`, which
bash excludes U+00A0 from under the C and UTF-8 locales alike, so that lone
separator survived as apparent typed content and the row classified
`pending` - the verdict meaning "a human's unsubmitted text is sitting
there". Every caller that must not overwrite unsubmitted input then refused
to act: away-mode escalation delivery deferred 2105 times across 8.3 hours
against an idle pane, and steer confirmation reported landed messages as
unconfirmed.

Established by counterfactual against the live panes rather than by reading
the code: varying the separator alone moved the verdict, while recolouring
the glyph (including to the truecolor luminance originally suspected) and
removing the input area's horizontal rules did not. Reproduced identically
on tmux and herdr, where the tmux row is nothing but the glyph and the
separator.

fm_composer_normalize_trim now maps every code point carrying the Unicode
property White_Space=Yes onto an ASCII space before any trim, border strip,
or glyph comparison. That is a standards property rather than one more
vendor-observed string, so it covers whichever space-like character a
harness switches to next, and it cannot make the predicate more permissive:
whitespace-only content already classified `empty` when the whitespace was
ASCII, and any non-whitespace character anywhere still classifies `pending`.
The bare dead-shell refusal is unchanged, with or without a separator.

The leading-glyph strip used `${content#?}`, which drops one byte under
LC_ALL=C and one character under a UTF-8 locale, leaving locale-dependent
trailing bytes on a multibyte glyph; it now removes the glyph as a literal
string. The prompt glyphs themselves were spelled out in three separate
places, which the file's own header flagged as a drift hazard, and are now
declared once each.

The four adapters route their structural row trims through the same shared
helper, so a non-ASCII space cannot defeat one adapter's row scan while
another's still matches.

Tests, per the harness-dependent-check rule:
- Portable regressions pinning the real captured rows in CI - the empty
  composer that must read `empty` and the typed-into one that must still
  read `pending` - asserting the fixtures still carry the invisible
  separator so no case can pass vacuously, and pinning the whole
  White_Space property set plus locale independence.
- tests/fm-composer-harness-drift-live-e2e.test.sh, env-gated and
  self-skipping, drives every installed harness for real: it types a
  marker, confirms the harness renders it, clears it, confirms it is gone,
  and only then reads the affirmatively empty composer. It fails naming the
  harness and version with the decoded row bytes, reports an absent harness
  explicitly, and refuses a pass that checked nothing.

Verified against real claude 2.1.226 on tmux 3.5a: the empty row is
`e2 9d af c2 a0`, and the guard fails on that exact row without this fix.
Dated per-harness evidence is recorded in docs/verification/runtime-backends.md.

Refs kunchenguid#1988
@mackcee
mackcee force-pushed the fm/fm-composer-read-unreliable branch from 04b709c to f689500 Compare August 11, 2026 02:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants