voculinux/docs/CUSTOM_DICTIONARY_DESIGN.md

Custom Dictionary Support Design

Scope

Custom dictionary support has two separate jobs:

1. Custom terms bias local Whisper, whisper.cpp, and Faster Whisper recognition toward names and jargon.
2. Transcript corrections deterministically replace a known misheard phrase after transcription.

They appear together on the Custom Dictionary Settings page, but are not interchangeable. A correction replacement does not implicitly become a prompt term. That coupling would make recognition bias depend on correction order and would make the external terms contract surprising.

File contracts

Both files live in Voculinux's XDG-aware configuration directory: $XDG_CONFIG_HOME/voculinux (or ~/.config/voculinux when unset).

File	Contract	Consumer
dictionary.txt	UTF-8 (BOM accepted), one term per line. Blank lines and lines beginning with # are ignored. Terms are de-duplicated case-insensitively while retaining the first spelling.	Voculinux and an external accessibility scanner.
custom-dictionary-corrections.json	UTF-8 JSON object: {"version": 1, "corrections": [{"heard": "super base", "replacement": "Supabase"}]}.	Voculinux only.

The terms file remains deliberately simple and scanner-friendly. Corrections need two unambiguous fields, a version, and exact replacement casing, so JSON is used rather than overloading the line file with an ambiguous delimiter grammar. The UI writes either file by atomic replacement; a failed UI save preserves the old file and reports an error.

The application reads both files for each completed transcript. An externally edited file therefore takes effect on the next dictation segment without an app restart. Missing, unreadable, malformed, or unsupported correction files are ignored safely; dictation continues without their entries.

The terms file may contain any number of lines, but Voculinux limits a single prompt to the first 200 valid terms and 2,000 characters. This bounds decoding work while leaving the scanner's complete line-file contract intact. The Custom Dictionary page keeps both an add/remove editor and a validated file chooser for the terms file.

Processing and engine behavior

For every completed audio segment, Voculinux applies this order:

1. The selected engine transcribes audio.
2. Live phrase corrections run against the raw transcript.
3. Voice-command interpretation runs on that corrected text.
4. Remaining text and actions are dispatched.

Corrections run before commands so a correction can prevent a command-like misrecognition from being acted on. This also means users must not create a replacement that deliberately creates a voice command unless that is intended.

Corrections match literal phrases case-insensitively, only where the adjacent characters are not Unicode word characters (or common combining marks). Longer phrases win over shorter overlaps; equal-length entries retain JSON order. Replacement text is inserted exactly as written. Inputs are normalized to NFC for matching, so composed/decomposed accents are handled consistently.

Engine	Custom terms	Corrections
OpenAI Whisper	initial_prompt	Yes
whisper.cpp	Advanced initial prompt followed by custom terms; explicit empty prompt clears reused native state	Yes
Faster Whisper	initial_prompt	Yes
VOSK	No prompt-bias API	Yes
Parakeet	No prompt-bias API	Yes
Remote API	No prompt is added to current request formats	Yes

--dictionary-file PATH preserves the #767 session-only behavior: PATH is used as the custom terms file, it enables terms for that session, and Settings does not modify the saved terms file or enablement. Corrections remain loaded from the standard JSON file. The previous experimental #768 text_injection.custom_dictionary config list is read only as a fallback until the JSON corrections file exists; it is never silently deleted or migrated.

The #767 persisted configuration keys are retained unchanged: dictionary.enabled, dictionary.file_path (default ~/.config/voculinux/dictionary.txt), and dictionary.max_words. Existing Peony settings therefore continue to select the scanner's dictionary.txt file. New configured paths are accepted only when they can be expanded and are either absent (so the UI can create them) or readable regular files; a failed save leaves the prior setting in place. Invalid configured and --dictionary-file paths are safely ignored and shown as unavailable rather than raising.

Alternatives rejected

• One mixed line file: cannot express arbitrary phrases and replacements without reserving/escaping delimiters, which weakens the scanner contract.
• Corrections in config.json: makes a scanner compete with application settings writes and offers no durable external-file schema.
• Post-command correction: allows command-like transcription errors to be acted on before they can be fixed.
• Automatically adding correction replacements to prompts: rejected because replacements are not automatically included in the recognition prompt; corrections work post-transcription on every engine.

Human test checklist

1. Add Voculinux, PyGObject, and a non-ASCII term in Custom terms; confirm dictionary.txt is UTF-8, one line per term, and an external edit changes the next Whisper, whisper.cpp, and Faster Whisper dictation segment.
2. Add super base → Supabase; test lowercase, uppercase, punctuation, an overlapping short correction, C++, and non-ASCII text. Confirm replacement casing is exact and embedded text (for example supersize) is not replaced.
3. With voice commands enabled, add a correction that changes a command-like misrecognition into ordinary text. Confirm it is injected rather than acted on. Also verify an intentionally command-like replacement is avoided.
4. Edit each file externally while Voculinux is running. Test an invalid JSON correction file and invalid UTF-8 terms file; confirm dictation continues and the Settings status/save feedback is clear.
5. Test the UI with keyboard only: Add buttons via Enter, empty validation, update/remove feedback, narrow window, long entries, and high-contrast theme.
6. Test all engines: Whisper, whisper.cpp, and Faster Whisper should use terms and corrections; VOSK, Parakeet, and remote API should use corrections only, with the VOSK terms warning.
7. Start with --dictionary-file /path/to/terms.txt; confirm the session uses that file, Settings disables terms editing, saved settings remain unchanged, and corrections still apply.
8. Start with an unresolved path such as --dictionary-file ~missing-user/terms.txt and configure an unreadable path through the file chooser. Confirm Settings reports the path safely, dictation continues without terms, and the prior saved dictionary.file_path remains intact.