feat(language): follow the active keyboard layout while dictating (#837)

* feat(language): follow the active keyboard layout while dictating

Adds a "Follow keyboard layout" switch to Settings -> Speech Model ->
Advanced. With it on, the active keyboard layout is re-read at the start
of every dictation and the matching language is used, so someone who
switches layouts to work in two languages gets the right one without
opening Settings.

#796 already seeded the language once on first run, but from the
*configured* primary layout, which never changes as the user switches
between them: a "fr,us" setup reports fr forever. detect_active_keyboard_
layout() reads GNOME's input sources instead. It takes mru-sources rather
than the current index, because that index goes stale -- observed still
reading 0 after a switch to the second layout -- which is why the IBus
injection path already trusts the MRU list (#497, #738).

An active source that is an IBus engine rather than an xkb layout is a
real answer that cannot be mapped, so the language is left to the engine's
own auto-detect. Borrowing the configured xkb layout there would pin
English for someone typing Japanese through mozc-jp.

The mode is a switch over the language picker rather than an entry inside
it, since it is a mode and not a language. While on, the picker is
disabled and shows what the layout resolves to; turning it off pins that.

A language change no longer re-initialises the engine when the loaded
model takes the language per utterance, which is the whole whisper
family, so following a layout costs a settings read rather than a model
reload. VOSK loads one model per language, Parakeet ignores the picker,
and an English-only model has no non-English weights, so Settings refuses
the mode for all three rather than offering a switch that does nothing.

Fixes #821

* fix(language): keep follow mode armed and map IBus engines

Swap English-only whisper.cpp weights to the multilingual sibling while
passing the layout sentinel into reconfigure, so follow mode stays armed.
Map common GNOME IBus engines onto catalogue languages instead of
leaving them unmapped.

* fix(language): stop IBus hyphen false-maps and gate the sibling swap

The last-token hyphen heuristic mapped mozc-us and foo-us to English.
Restrict it to *-jp/*-kr and name Mozc's extra engines as Japanese.
Only whisper.cpp may consult the whisper.cpp download catalog.

* fix(language): keep multilingual weights while following layout

The picker shows the active layout while follow is on, often en-us.
Deriving whisper.cpp .en from that display made Settings refuse the
mode as English-only, then the next auto-apply persisted a concrete
language and disarmed follow. Use LANGUAGE_FOLLOWS_LAYOUT for .en vs
multilingual derivation instead, including on the load path before
the switch is synced.

* fix(language): type follow helpers and log sibling-swap failures

* fix(test): stub follow-layout model rec so mocked psutil does not blow up

follow-mode English-only guard calls get_recommended_whispercpp_model, which
reads RAM through psutil. Full CI leaves a MagicMock in sys.modules["psutil"],
so ram_gb >= 16 raises TypeError. Patch the settings_dialog alias to return a
multilingual tiny rec so the assertion still holds without real RAM.

* fix(language): catch specific errors on layout lookup and sibling swap

* fix(language): swallow more sibling-swap reconfigure failures

ImportError, TypeError, AttributeError, and ChecksumError from the
multilingual swap must not abort dictation or disarm follow mode.

---------

Co-authored-by: Jatin K Malik <jatinkrmalik@gmail.com>
This commit is contained in:
Vivek A Nadig
2026-09-29 01:06:32 -07:00
committed by GitHub
co-authored by Jatin K Malik
parent 9646011e77
commit 7bdbedce0b
7 changed files with 1103 additions and 12 deletions
+17
View File
@@ -85,6 +85,23 @@ Open **Settings → Speech Model**. The page starts with a simple setup (languag
Parakeet runs NVIDIA NeMo ASR models through sherpa-onnx. The default bundle is **v3-european** (25 European languages). **v2-english** is English-only. Parakeet ignores the catalog language picker (language is treated as auto).
### Language
Pick the language you dictate in from **Language**, or leave it on auto-detect.
If you work in more than one language and switch keyboard layouts to do it, turn
on **Follow keyboard layout** instead. Vocalinux then reads your active layout at
the start of every dictation and uses the matching language, so switching layout
switches dictation language with it. The Language picker greys out while this is
on and shows what your current layout resolves to; turning it off pins that
language.
Available for whisper.cpp, Whisper, Faster Whisper, and Remote API. VOSK loads a
separate model per language, so following a layout would mean a model reload on
the hotkey; Parakeet does not use the language picker at all. Active-layout
detection uses GNOME's input-source settings; on other desktops it falls back to
the configured primary layout.
### Model size (whisper.cpp / Whisper)
| Size | Approx. size | Tradeoff |