Skip to content

fix(desktop): transcribe only the local microphone in Huddles - #7043

Open
hr-o wants to merge 1 commit into
block:mainfrom
hr-o:fix/huddle-transcript-attribution
Open

fix(desktop): transcribe only the local microphone in Huddles#7043
hr-o wants to merge 1 commit into
block:mainfrom
hr-o:fix/huddle-transcript-attribution

Conversation

@hr-o

@hr-o hr-o commented Aug 29, 2026

Copy link
Copy Markdown

Related work

No open PR or issue fixes this. Closest existing work, none of which overlaps the fix itself:

Problem

A Huddle transcript is published as a kind:9 signed with the local user's key. But playout.rs fed every remote human peer's audio into the same STT pipeline: each active peer slot that wasn't a locally-synthesized TTS publisher or a known agent was summed into one mono buffer by mix_remote_stt_samples and pushed in via push_remote_audio.

The result is forged provenance. When Alice speaks, Bob's desktop transcribes her and publishes her words as a message signed by Bob — independently, on every listening desktop in the room. There is no diarization and no host election, so a three-desktop Huddle turns one sentence from Alice into a kind:9 from Bob and another from Carol, each claiming to be that user's own speech.

The remote path was also deliberately ungated by push-to-talk and mute, so a listener who never unmuted still published other people's speech under their own key.

Fix

Remove the producer rather than filter at the publisher.

The alternative was to carry an origin through the text layer — TranscriptSegment { origin, text } — and admit only LocalMicrophone before publishing. But once playout.rs stops feeding remote audio in, that origin has exactly one possible value: RemoteHuman becomes an unconstructed variant, push_remote_audio a method with no caller, and any test would assert an unreachable branch. So the invariant is expressed by deleting the producer and documenting it on the single surviving entry point.

Secondary benefit: every listening desktop stops running Parakeet over audio it must discard.

Gone: SttAudioOrigin and its second VAD stream, push_remote_audio, track_local_floor, mix_remote_stt_samples, f32_samples_to_le_bytes, the mix block in the playout receive loop, and the remote_stt_pipeline weak handle in HuddleState. The STT channel now carries plain PCM. Net −152 lines across 5 files, all under desktop/src-tauri/src/huddle/.

No relay change. No protocol change.

Known regression

Legacy text-only agents lose the accidental proxy by which mobile speech reached them, and stay silent until an agent audio runtime lands.

Mobile has no local STT, so a phone's speech reached text-driven agents only because some desktop in the room transcribed it and published it under that desktop user's key. That path is now gone, and nothing replaces it in this PR.

Speech-to-speech agents are unaffected — they receive the Opus room directly.

This is a deliberate trade: correct attribution over an undesigned capability. Publishing one participant's speech under another participant's signature is forged provenance in a signed-event system, and the lost behaviour was a side effect of every listener transcribing the room, not a designed feature.

Verification

Automated, on this branch's base:

  • cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings — clean
  • Tauri fmt + clippy under -D warnings — clean
  • cargo test --workspace (Tauri) — 3004 passed / 0 failed / 18 ignored
  • Full just ci green: desktop JS 5799/0, desktop and web builds, flutter analyze clean, mobile 2007 tests

Manual, on real hardware — one Windows desktop as a muted listener, one phone as the speaker, against a stock ghcr.io/block/buzz:main relay:

vanilla 0.5.19 this branch
Phone speaks, listener muted 2 kind:9 events published by the listener, carrying the phone's words 0 events
Listener speaks 1 event, correctly its own 1 event, correctly its own

The listener's own transcript is the control that makes the zero meaningful: STT was running, VAD was flushing, NIP-98 authenticated and the relay accepted, 42 seconds after the phone joined the room. It simply had nothing to say about the phone's audio.

Why there is no new regression test

The usual shape — a test asserting that remote audio never reaches the publisher — has no code path left to exercise. Removing the producer means there is no way to construct the failing input: push_remote_audio has no caller, RemoteHuman has no constructor, and the mixing function is gone. A test could only assert that a symbol no longer exists, which the compiler already enforces at every call site.

So the guarantee is structural rather than behavioural. What the PR does instead:

  • documents the invariant on SttPipeline::push_audio, the single surviving entry point, and in the module header
  • deletes the test for the removed mixer, and renames remote_agent_identity_is_excluded_from_the_human_stt to …_from_the_human_floor, which is what it actually asserts now
  • proves the behaviour end-to-end on hardware, in the before/after table above

Not included

No UI change, so there are no screenshots.

Restoring speech input for text-only agents is deliberately out of scope. It belongs with an agent audio runtime that joins the Opus room directly, which is follow-up work.


The pre-fix run used the released 0.5.19 client, so the defect is reproducible in shipped code — push_remote_audio arrived in #6056 on 2026-08-22, three days before that tag.

Each Huddle transcript is a kind:9 event. The local user signs this
event. Thus, do not send audio from other participants to the STT
pipeline. If the desktop transcribes remote audio, it incorrectly
identifies the remote speaker as the local user. This error occurs on
each desktop that receives the audio.

Remove the remote-human transcription path. Do not filter remote
transcripts before publication. The STT pipeline now receives audio only
from the local microphone. This design also prevents Parakeet from
processing remote audio that the desktop cannot publish correctly.

Known regression: legacy text-only agents no longer receive transcripts
of speech from mobile participants. These agents cannot respond to that
speech until an agent audio runtime is available. An agent that joins
the Opus room directly does not depend on transcripts and is therefore
unaffected, but no such agent exists yet.

Signed-off-by: hr-o <126079238+hr-o@users.noreply.github.com>
@hr-o
hr-o requested a review from a team as a code owner August 29, 2026 20:35
@github-actions

Copy link
Copy Markdown

🔐 Codex Security Review

Status: review required for the current range.

The current range is 00e61eafa917d296104006576b7a2ddbfd58bb5a...afad63572fa53825d038ecc874cac130fb29dd6e.
A new review must complete for this exact range. When manual authorization
is required, a Block organization member must comment exactly
@buzz-security-review afad63572fa53825d038ecc874cac130fb29dd6e to authorize a new review.
Any previous review applies only to its recorded range.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant