Skip to content

fix(mobile): start voice input in the device language when its offline speech model is missing - #5925

Merged
iscekic merged 1 commit into
mainfrom
kwf/voice-input-fails-on-a-germa-f790
Sep 7, 2026
Merged

fix(mobile): start voice input in the device language when its offline speech model is missing#5925
iscekic merged 1 commit into
mainfrom
kwf/voice-input-fails-on-a-germa-f790

Conversation

@iscekic

@iscekic iscekic commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Request

Voice input fails on a German-locale Android phone. A colleague reported that
German is not available: he taps the voice input control and gets an error.
His device locale is German.

Reproduce first on an Android emulator with the device locale set to German
(de-DE): set the system language to Deutsch, open the app, tap voice input,
and record the error. If it does not reproduce, answer no_change with the
evidence.

Then fix it so voice input works in the device locale, not English alone.
Cover the case where the device locale has no on-device speech model: the app
must say what is wrong and what the user can do, never a bare error.

Prove the fix live on the same German-locale emulator with a recording, and
show English still works after the change.

Changelog for users

  • Voice input works in the device language even when its offline speech model isn't installed: when online transcription is allowed, recognition now falls back to the network in the device language instead of failing on every tap; with consent unset, the usual online-processing disclosure comes first and Allow starts it.
  • With online transcription declined, tapping voice input for a language without an offline model now explains that the language's offline speech files aren't installed on the phone and offers Download; accepting triggers the system's offline model download, and once it completes the next tap recognizes on-device in that language.
  • When the system schedules that download in the background, the app confirms it and asks the user to try voice input again later; when triggering it fails, the app shows an actionable message instead of a bare error.
  • The error for an unavailable device language now says what to do: install the language's offline speech files or use another app language.
  • All new prompts ship translated in every app language.
  • English voice input is unchanged and still reaches the listening state.

Changelog for maintainers

  • Root cause: the start path trusted the service-level on-device check, so requiresOnDeviceRecognition was set for a language whose offline model was missing and every start failed with language-not-supported. The refined gate lives in apps/mobile/src/lib/voice-input/use-voice-input-actions.ts (supportsOnDevice = service supports on-device AND the resolved language is installed) — start the review there.
  • apps/mobile/src/lib/voice-input/voice-input-language.ts now caches locales together with installedLocales from getSupportedLocales, and adds the per-language gate, an endonym display name (de-DE → 'Deutsch') for the alert copy, and invalidateVoiceRecognitionLocalesCache(); the invalidation after a download trigger is load-bearing — without it the gate would serve the stale pre-download list for the rest of the session.
  • The gate is conservative by design: it never blocks a start when the service exposes no per-language data, when the locale fetch fails, or when the language is unknown to the service; only 'supported but not installed' counts as missing.
  • Not exercised live: the declined-consent download alert (the emulator run had consent granted and there is no consent hook in the harness); that path is covered by unit tests only.
  • The Download button calls the Android-only androidTriggerOfflineModelDownload; if that path is reached on iOS the rejection lands in the catch and shows the actionable toast, but the alert itself is platform-blind — worth a look.
  • New tests: per-language gate fallbacks and endonym naming, plus integration scenarios for the network fallback in de-DE, disclosure-then-network start, the download offer and trigger, the download-scheduled toast, on-device recognition on the tap after a completed download, the failure toast, and the no-per-language-data case; the unavailableLanguage copy change is asserted in the two error-classification suites.

E2E proof

[e1] Prove the fix live on the same German-locale emulator with a recording, and show English still works after the change. — android emulator-5554: app language Deutsch; mic showed disclosure 'Sprachaufnahmen werden online verarbeitet' with NICHT JETZT/ERLAUBEN (e1-tap-mic-de.png, e1-disclosure.log), then listening 'Aufnahme läuft …' / 'Spracheingabe beenden' (e1-erlauben-listen.png, e1-de-listen.log). After switching to English, Start voice input reached Listening (e1-english-mic.png, e1-en-listen.log). Recording e1-voice-de-en.mp4. Declined-consent download alert not exercised (consent granted; no e2e consent hook). UX: disclosure and listening labeled with next actions; no UX-DEFECT.

e1-voice-de-en.mp4.trim.mp4

[e1] Prove the fix live on the same German-locale emulator with a recording, and show English still works after the change. — e2e-mobile-app/e1-erlauben-listen.png

[e1] Prove the fix live on the same German-locale emulator with a recording, and show English still works after the change. — e2e-mobile-app/e1-english-mic.png

Follow-ups (not changed here)

  • e1-open-profile.png,e1-prefs-open.png,e1-prefs.png — Completely black frame with no status bar, chrome, or content, so the screen has no visible empty/error/loading state.

Comment thread apps/mobile/src/lib/voice-input/use-voice-input-actions.ts
Comment thread apps/mobile/src/lib/voice-input/voice-input-language.ts Outdated
@kilo-code-bot

kilo-code-bot Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (6 files)
  • apps/mobile/src/i18n/resolve-language.ts
  • apps/mobile/src/i18n/resolve-language.test.ts
  • apps/mobile/src/lib/voice-input/use-voice-input-actions.ts
  • apps/mobile/src/lib/voice-input/use-voice-input.test.ts
  • apps/mobile/src/lib/voice-input/voice-input-language.ts
  • apps/mobile/src/lib/voice-input/voice-input-language.test.ts
Previous Review Summary (commit c5b1ba2)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit c5b1ba2)

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
apps/mobile/src/lib/voice-input/use-voice-input-actions.ts 166 Download offer is platform-blind but the trigger is Android-only

SUGGESTION

File Line Issue
apps/mobile/src/lib/voice-input/voice-input-language.ts 203 Primary-subtag lookup misnames pt-BR and Chinese
Files Reviewed (94 files)
  • apps/mobile/src/lib/voice-input/use-voice-input-actions.ts - 1 issue
  • apps/mobile/src/lib/voice-input/voice-input-language.ts - 1 issue
  • apps/mobile/src/lib/voice-input/voice-input-recognition-mode.ts
  • apps/mobile/src/lib/voice-input/use-voice-input.test.ts
  • apps/mobile/src/lib/voice-input/voice-input-language.test.ts
  • apps/mobile/src/lib/voice-input/voice-input-controller-results.test.ts
  • apps/mobile/src/lib/voice-input/voice-input-state.test.ts
  • apps/mobile/src/i18n/locales/*.json (87 catalogs)

Fix these issues in Kilo Cloud


Reviewed by grok-4.6 · Input: 65.4K · Output: 14.2K · Cached: 473K

Review guidance: REVIEW.md from base branch main

@iscekic
iscekic marked this pull request as draft September 7, 2026 05:59
@iscekic
iscekic force-pushed the kwf/voice-input-fails-on-a-germa-f790 branch from 1d2c141 to b1b03d5 Compare September 7, 2026 08:42
@iscekic
iscekic marked this pull request as ready for review September 7, 2026 10:42
@iscekic iscekic added the human-ready The PR is ready for human review. label Sep 7, 2026
@iscekic
iscekic merged commit ba90783 into main Sep 7, 2026
32 of 33 checks passed
@iscekic
iscekic deleted the kwf/voice-input-fails-on-a-germa-f790 branch September 7, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

human-ready The PR is ready for human review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants