7bdbedce0b
* feat(language): follow the active keyboard layout while dictating Adds a "Follow keyboard layout" switch to Settings -> Speech Model -> Advanced. With it on, the active keyboard layout is re-read at the start of every dictation and the matching language is used, so someone who switches layouts to work in two languages gets the right one without opening Settings. #796 already seeded the language once on first run, but from the *configured* primary layout, which never changes as the user switches between them: a "fr,us" setup reports fr forever. detect_active_keyboard_ layout() reads GNOME's input sources instead. It takes mru-sources rather than the current index, because that index goes stale -- observed still reading 0 after a switch to the second layout -- which is why the IBus injection path already trusts the MRU list (#497, #738). An active source that is an IBus engine rather than an xkb layout is a real answer that cannot be mapped, so the language is left to the engine's own auto-detect. Borrowing the configured xkb layout there would pin English for someone typing Japanese through mozc-jp. The mode is a switch over the language picker rather than an entry inside it, since it is a mode and not a language. While on, the picker is disabled and shows what the layout resolves to; turning it off pins that. A language change no longer re-initialises the engine when the loaded model takes the language per utterance, which is the whole whisper family, so following a layout costs a settings read rather than a model reload. VOSK loads one model per language, Parakeet ignores the picker, and an English-only model has no non-English weights, so Settings refuses the mode for all three rather than offering a switch that does nothing. Fixes #821 * fix(language): keep follow mode armed and map IBus engines Swap English-only whisper.cpp weights to the multilingual sibling while passing the layout sentinel into reconfigure, so follow mode stays armed. Map common GNOME IBus engines onto catalogue languages instead of leaving them unmapped. * fix(language): stop IBus hyphen false-maps and gate the sibling swap The last-token hyphen heuristic mapped mozc-us and foo-us to English. Restrict it to *-jp/*-kr and name Mozc's extra engines as Japanese. Only whisper.cpp may consult the whisper.cpp download catalog. * fix(language): keep multilingual weights while following layout The picker shows the active layout while follow is on, often en-us. Deriving whisper.cpp .en from that display made Settings refuse the mode as English-only, then the next auto-apply persisted a concrete language and disarmed follow. Use LANGUAGE_FOLLOWS_LAYOUT for .en vs multilingual derivation instead, including on the load path before the switch is synced. * fix(language): type follow helpers and log sibling-swap failures * fix(test): stub follow-layout model rec so mocked psutil does not blow up follow-mode English-only guard calls get_recommended_whispercpp_model, which reads RAM through psutil. Full CI leaves a MagicMock in sys.modules["psutil"], so ram_gb >= 16 raises TypeError. Patch the settings_dialog alias to return a multilingual tiny rec so the assertion still holds without real RAM. * fix(language): catch specific errors on layout lookup and sibling swap * fix(language): swallow more sibling-swap reconfigure failures ImportError, TypeError, AttributeError, and ChecksumError from the multilingual swap must not abort dictation or disarm follow mode. --------- Co-authored-by: Jatin K Malik <jatinkrmalik@gmail.com>
163 lines
7.6 KiB
Markdown
163 lines
7.6 KiB
Markdown
# User guide
|
|
|
|
How to use Vocalinux day to day. Install first: [INSTALL.md](INSTALL.md).
|
|
|
|
## Getting started
|
|
|
|
1. Launch Vocalinux (`vocalinux` or the application menu)
|
|
2. Find the microphone icon in the system tray
|
|
3. Start dictation with the tray menu or your keyboard shortcut
|
|
4. Speak into the focused application; text is injected when an utterance completes
|
|
5. Stop by releasing the key (push-to-talk, default on new installs) or with the same shortcut (toggle)
|
|
|
|
### Start on login
|
|
|
|
Enable **Start on Login** from the first-run dialog, tray menu, or Settings. Vocalinux writes an XDG autostart entry (`~/.config/autostart/vocalinux.desktop`) and starts as a normal user app (`--start-minimized`). It does not create a systemd service.
|
|
|
|
Works on common desktop environments (GNOME, KDE, Xfce, Cinnamon, MATE, LXQt). Minimal window-manager sessions may need their own autostart helper.
|
|
|
|
### Status icons
|
|
|
|
| Icon state | Meaning |
|
|
|------------|---------|
|
|
| Gray (off) | Inactive |
|
|
| Blue (on) | Listening |
|
|
| Orange | Processing speech |
|
|
|
|
### Dictation formatting
|
|
|
|
Vocalinux capitalizes the start of dictation and letters after `.`, `!`, or `?`. Each completed utterance leaves a trailing space so the next session does not glue onto the previous sentence.
|
|
|
|
### Dictating into terminals
|
|
|
|
When Vocalinux injects through the clipboard (the usual Wayland / ydotool path), it sends **Ctrl+V** in ordinary text fields and **Ctrl+Shift+V** in terminal emulator windows. Auto-detect works on X11 and on Hyprland, Sway, and niri. On GNOME or KDE Wayland, set **Settings → Dictation → Clipboard Paste Shortcut** to **Ctrl+Shift+V**.
|
|
|
|
On non-US layouts such as German Neo, that chord uses the key that types **v** on the active layout (not physical KEY_V). Nested terminal panels inside an IDE are often invisible to window-class detection. If paste lands as a literal `^V` or does nothing, open **Settings → Dictation → Clipboard Paste Shortcut** and choose **Ctrl+Shift+V**.
|
|
|
|
## Shortcuts
|
|
|
|
Configure under **Settings → Shortcuts**:
|
|
|
|
| Mode | Behavior |
|
|
|------|----------|
|
|
| **Push-to-talk** (default on new installs) | Hold Right Alt (Option on Mac-layout keyboards) while speaking; release to stop |
|
|
| **Toggle** | Double-tap the configured shortcut key to start/stop |
|
|
|
|
Existing configs keep their saved shortcut. Left/right modifier keys and custom modifier+key combos (for example `Alt+R`) are supported.
|
|
|
|
## Voice commands
|
|
|
|
Optional spoken commands for punctuation and editing (can be disabled in Settings).
|
|
English phrases always work. With a non-English recognition language, matching
|
|
punctuation and line-break phrases in that language are also recognized
|
|
(Italian *virgola* / *punto*, French *virgule* / *point*, and similar).
|
|
|
|
| Command | Action |
|
|
|---------|--------|
|
|
| "new line" / "new paragraph" | Line break |
|
|
| "period" / "full stop" / "dot" | `.` |
|
|
| "comma" | `,` |
|
|
| "question mark" | `?` |
|
|
| "exclamation point" / "exclamation mark" | `!` |
|
|
| "semicolon" | `;` |
|
|
| "colon" | `:` |
|
|
| "delete that" / "scratch that" | Delete last sentence |
|
|
| "capitalize" / "uppercase" | Capitalize next word |
|
|
| "all caps" | Next word in ALL CAPS |
|
|
|
|
Editing and formatting action phrases (`delete that`, `undo`, `capitalize`, and similar)
|
|
are currently English-only.
|
|
|
|
## Engines and models
|
|
|
|
Open **Settings → Speech Model**. The page starts with a simple setup (language and speed/accuracy). Expand **Advanced** for engine, model size, and specialization. Sidebar search (Ctrl+F) works across pages.
|
|
|
|
### Engines
|
|
|
|
| Engine | Best for | GPU | Footprint |
|
|
|--------|----------|-----|-----------|
|
|
| **whisper.cpp** (default) | Most users | Vulkan (AMD, Intel, NVIDIA) | ~74MB default model |
|
|
| **Whisper** (OpenAI) | PyTorch/CUDA workflows | NVIDIA/CUDA | Large (PyTorch stack) |
|
|
| **Faster Whisper** | CPU Whisper (CTranslate2 / INT8) | Optional CUDA | Similar model sizes to Whisper |
|
|
| **VOSK** | Low RAM / older machines | CPU | ~40MB |
|
|
| **Parakeet** | CPU dictation; 25 European languages | CPU | ~639MB v3-european |
|
|
| **Remote API** | Offload to a server | N/A (server-side) | Opt-in; see [HTTP_REMOTE.md](HTTP_REMOTE.md) |
|
|
|
|
Parakeet runs NVIDIA NeMo ASR models through sherpa-onnx. The default bundle is **v3-european** (25 European languages). **v2-english** is English-only. Parakeet ignores the catalog language picker (language is treated as auto).
|
|
|
|
### Language
|
|
|
|
Pick the language you dictate in from **Language**, or leave it on auto-detect.
|
|
|
|
If you work in more than one language and switch keyboard layouts to do it, turn
|
|
on **Follow keyboard layout** instead. Vocalinux then reads your active layout at
|
|
the start of every dictation and uses the matching language, so switching layout
|
|
switches dictation language with it. The Language picker greys out while this is
|
|
on and shows what your current layout resolves to; turning it off pins that
|
|
language.
|
|
|
|
Available for whisper.cpp, Whisper, Faster Whisper, and Remote API. VOSK loads a
|
|
separate model per language, so following a layout would mean a model reload on
|
|
the hotkey; Parakeet does not use the language picker at all. Active-layout
|
|
detection uses GNOME's input-source settings; on other desktops it falls back to
|
|
the configured primary layout.
|
|
|
|
### Model size (whisper.cpp / Whisper)
|
|
|
|
| Size | Approx. size | Tradeoff |
|
|
|------|--------------|----------|
|
|
| tiny | ~74MB | Fastest; real-time friendly |
|
|
| base | ~141MB | Balance of speed and accuracy |
|
|
| small | ~465MB | Better accuracy |
|
|
| medium | ~1.5GB | High accuracy |
|
|
| large | ~3.0GB | Best accuracy; heavier |
|
|
|
|
For whisper.cpp, also pick a **Specialization**: standard multilingual, English-only, quantized (lower memory), Turbo, or legacy large. English-only specializations limit the language selector to English. Exact IDs (for example `medium.en-q5_0`, `large-v3-turbo`) work with `--model`.
|
|
|
|
### Removing unused models
|
|
|
|
If leftover files are on disk that are not the model currently selected, **Unused downloads** appears under the model info card. Expand it to delete leftovers one at a time. Confirming removes those files from `~/.local/share/vocalinux`. Packaged system-wide VOSK models are left alone.
|
|
|
|
### GPU
|
|
|
|
whisper.cpp prefers Vulkan when the bundled pywhispercpp libraries include it, then CUDA, then CPU. Host tools such as `vulkaninfo` only describe the machine. The engine follows the libraries actually loaded. On multi-GPU machines a discrete Vulkan device is preferred; override under **Settings → Performance → Vulkan GPU**. Check logs with `vocalinux --debug`.
|
|
|
|
Pip wheels of pywhispercpp are often CUDA builds. In that case Vocalinux uses CUDA device 0. `install.sh` rebuilds pywhispercpp with Vulkan or CUDA when it can.
|
|
|
|
### Auto-pause and keep-alive
|
|
|
|
Under Settings:
|
|
|
|
- **Auto-pause apps**: unload the model while listed apps run
|
|
- **Model keep-alive**: unload after idle timeout to free GPU/CPU
|
|
|
|
## Tips for better recognition
|
|
|
|
1. Use a decent microphone and reduce background noise when you can
|
|
2. Speak clearly at a natural pace
|
|
3. Prefer `tiny`/`base` for snappy dictation; larger models when accuracy matters more than latency
|
|
4. English-only or quantized specializations help when they match your use case
|
|
5. Confirm Vulkan/CUDA in debug logs if transcription is slower than expected
|
|
|
|
## CLI
|
|
|
|
```bash
|
|
vocalinux --help
|
|
vocalinux --version
|
|
vocalinux --debug
|
|
vocalinux --engine whisper_cpp
|
|
vocalinux --engine faster_whisper
|
|
vocalinux --engine parakeet
|
|
vocalinux --model medium.en-q5_0
|
|
vocalinux --wayland
|
|
vocalinux --start-minimized
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
```bash
|
|
vocalinux --debug
|
|
```
|
|
|
|
See [TROUBLESHOOTING.md](TROUBLESHOOTING.md) for tray, audio, injection, and model issues. Distro notes: [DISTRO_COMPATIBILITY.md](DISTRO_COMPATIBILITY.md). Updates: [UPDATE.md](UPDATE.md). Help channels: [SUPPORT.md](../SUPPORT.md).
|