xavierk/voculinux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

feat(asr): add Parakeet TDT 0.6B engine via sherpa-onnx (#802)

Browse Source 
* feat(asr): add Parakeet engine via sherpa-onnx

Adds a "parakeet" engine backed by sherpa-onnx, running NVIDIA Parakeet
TDT 0.6B int8 models on CPU: v3 (25 European languages, auto-detected)
and v2 (English). Output is punctuated and capitalized.

On an Intel Core Ultra 7 155H it decodes at about 12x realtime on CPU
(RTF 0.08), roughly 8x faster than whisper.cpp large-v3-turbo over the
same clips with its Vulkan GPU backend (RTF 0.59). Being a transducer it
also has no 30s window to pad, so a short push-to-talk utterance returns
in ~0.3s instead of ~2.4s.

Follows the existing engine layout: utils/parakeet_model_info.py holds
the catalog and the get_model_path / is_model_downloaded /
list_downloaded_models / delete_model helpers, while _init_parakeet /
_transcribe_with_parakeet / _download_parakeet_model follow the same shape as their
whisper.cpp counterparts and reuse _stream_model_download for progress
and cancellation.

Registered in the engine dispatch, the settings engine/model/language
pickers, the model info card, the download prompts, the unused-downloads
list and install.sh.

Models download to <models_dir>/parakeet/<model>/ via temp files renamed
on success; is_model_downloaded requires all four files, so an
interrupted download is retried rather than loaded.

Requires the new "parakeet" extra (sherpa-onnx).

* fix(install): write only the engine key for parakeet

The parakeet branch duplicated a full default config.json. ConfigManager
deep-merges DEFAULT_CONFIG on load, so every other key was redundant:
a config holding only "engine": "parakeet" still resolves
parakeet_model_size to v3-european along with the VAD, shortcut and
text-injection defaults.

* fix(install): keep push-to-talk defaults and switch existing configs

Writing only the engine key left config.json without a shortcuts
section, which _migrate_shortcuts_config reads as a pre-push-to-talk
upgrade and pins back to ctrl+ctrl/toggle - contradicting the
installer's own "Hold Right Alt while you speak" message. Seed that
section too.

Point the config at parakeet with set_configured_engine rather than only
on first create, so --engine=parakeet also switches a machine that
already has a config.json, and give print_welcome_message a parakeet arm
so the summary is not blank.

The interactive picker overwrites SELECTED_ENGINE, so document the flag
combination that reaches this branch.

* fix(asr): pin Parakeet bundles and report one progress bar

Pin each bundle to a Hugging Face commit instead of resolve/main, so a
file replaced upstream cannot change what an install downloads. This is
the same approach whispercpp_model_info takes with its pinned revision.

Remap each file's progress into its slice of the bundle. The four files
were reported through the single-file downloader independently, so the
fraction ran to 1.0 and restarted four times; it now advances once and
names the file being fetched.

* fix(asr): reject a truncated Parakeet download

_stream_model_download only rejects an empty body, so a truncated but
nonempty response was renamed into the model directory and then reported
as installed, because is_model_downloaded checks existence.

Record the byte size of each file at the pinned revision and check it
before the rename, the same order whisper.cpp verifies its digest in. A
short file is a truncated transfer rather than an upstream change, since
the revision is pinned, so it raises and the temp file is removed.

* style(asr): annotate the Parakeet model catalog

mypy inferred the catalog values as object, so reading file_sizes out of
an entry raised attr-defined and no-any-return under `just typecheck`.

* fix(asr): pin Parakeet digests in model_checksums.txt

AGENTS.md requires every speech model to be verified against a pinned
digest before install, failing closed on an unpinned one. Parakeet was
checking only the byte size.

Teach scripts/generate-model-checksums.py to emit Parakeet entries: the
ONNX files are LFS so Hugging Face publishes their sha256, and tokens.txt
is hashed from the downloaded bytes the way VOSK archives are. Several
bundles ship the same file names, so a key carries the model name.

_download_parakeet_model now calls verify_model_file before the rename
and catches ChecksumError alongside the other download failures, matching
the whisper.cpp downloader.

* fix(asr): verify on-disk Parakeet models before load

Existing bundle files were trusted by path alone, so a leftover or
copied-in tree could reach sherpa-onnx unchecked. Hash each file against
its pin on download skip and on init, and cover the Parakeet pins in CI.

* fix(ui): hide Parakeet language picker that ignores the setting

Parakeet language coverage is the model choice, not a catalog language.
Hide the picker and persist auto so Settings does not write a lying value.

* fix(asr): normalize Parakeet language to auto at startup

CLI --language and stale saved languages still reached SpeechRecognitionManager
even though Parakeet ignores catalog language. Force auto on the shared helper
used by main and the manager so stored settings stay honest.

* fix(ui): restore language after leaving Parakeet

Remember the prior Whisper/cpp language across a Parakeet visit so switching
back does not leave recognition stuck on auto. Keep Parakeet forced to auto
and the picker hidden. Also satisfy black on the startup normalizer path.

* fix(ui): keep language memory across Vosk fallback

Vosk's unsupported-language en-us coercion was writing into
_last_non_parakeet_language, so Whisper→Parakeet→Vosk→Whisper restored
en-us instead of the user's Greek (or other) preference. Remember the
pre-coercion catalog language and prefer it when leaving Vosk.

* fix(ui): keep language memory when entering Parakeet from Vosk

Vosk's coerced en-us fallback was still written into
_last_non_parakeet_language on Parakeet entry, so Vosk→Parakeet→Whisper
restored en-us instead of the pre-coercion preference (Greek or auto).
Share one coerced-fallback predicate across entry, leave-Vosk, and language
sync, and cover those paths with regression tests.

* fix(tests): set initializing flags false on language-change stubs

Plain Mock attributes are truthy and were short-circuiting the handler.

---------

Co-authored-by: Alex Thunder <AlexThunder@gmail.com>
This commit is contained in:
Jatin K Malik
2026-09-07 14:32:13 -07:00
committed by GitHub
co-authored by Alex Thunder
parent 946e19ee41
commit 8f4aeca2f1
16 changed files with 1444 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/vocalinux/main.py
+11 -1
View File
@@ -82,7 +82,7 @@ def parse_arguments():
parser.add_argument(
"--engine",
type=str,
choices=["vosk", "whisper", "whisper_cpp", "remote_api"],
choices=["vosk", "whisper", "whisper_cpp", "parakeet", "remote_api"],
help="Speech recognition engine to use (whisper_cpp recommended for best performance)",
)
parser.add_argument("--wayland", action="store_true", help="Force Wayland compatibility mode")
@@ -394,6 +394,16 @@ def main():
language = saved_settings.get("language", args.language)
logger.info(f"Using language={language} (from saved config)")
# Parakeet coverage is the model, not a catalog language. Normalize after
# CLI vs saved resolution so --language / stale config cannot leave an
# unused value on SpeechRecognitionManager.
resolved_language = language
language = recognition_manager.normalize_language_for_engine(engine, language)
if language != resolved_language:
logger.info(
"Parakeet ignores catalog language; using language=auto " f"(was {resolved_language})"
)
if cli_model_set:
model_size = args.model
logger.info(f"Using model={model_size} (from command line)")
src/vocalinux/speech_recognition/recognition_manager.py
+241 -2
View File
@@ -21,6 +21,7 @@ from typing import Callable, Optional
from ..common_types import RecognitionState
from ..ui.audio_feedback import play_error_sound, play_start_sound, play_stop_sound
from ..utils import parakeet_model_info as parakeet
from ..utils.host_process import host_env
from ..utils.model_checksums import (
ChecksumError,
@@ -84,6 +85,18 @@ def resolve_whisper_language(language: str) -> Optional[str]:
return language
def normalize_language_for_engine(engine: str, language: str) -> str:
"""Return the language the given engine actually consumes.
Parakeet coverage is the model (v2-english vs v3-european), not a catalog
language. Any leftover CLI or saved code is dropped so callers cannot store
an unconsumed value.
"""
if engine == "parakeet":
return "auto"
return language
# ALSA error handler to suppress warnings during PyAudio initialization
def _setup_alsa_error_handler():
"""Set up an error handler to suppress ALSA warnings."""
@@ -982,7 +995,7 @@ class SpeechRecognitionManager:
"""
self.engine = engine
self.model_size = model_size
self.language = language
self.language = normalize_language_for_engine(engine, language)
self.stop_sound_guard_ms = kwargs.get("stop_sound_guard_ms", 200)
self.state = RecognitionState.IDLE
self.audio_thread = None
@@ -1076,7 +1089,7 @@ class SpeechRecognitionManager:
os.makedirs(MODELS_DIR, exist_ok=True)
logger.info(
f"Initializing speech recognition with {engine} engine, {language} language and {model_size} model"
f"Initializing speech recognition with {engine} engine, {self.language} language and {model_size} model"
)
# Initialize the selected speech recognition engine
@@ -1086,6 +1099,8 @@ class SpeechRecognitionManager:
self._init_whisper()
elif engine == "whisper_cpp":
self._init_whispercpp()
elif engine == "parakeet":
self._init_parakeet()
elif engine == "remote_api":
self._init_remote_api()
else:
@@ -1763,6 +1778,216 @@ class SpeechRecognitionManager:
logger.error(f"Error in whisper.cpp transcription: {e} ({audio_info})", exc_info=True)
return ""
def _download_parakeet_model(self):
"""Download the Parakeet model files with progress tracking."""
import requests
self._download_cancelled = False
model_dir = parakeet.get_model_path(self.model_size)
os.makedirs(model_dir, exist_ok=True)
logger.info(f"Downloading Parakeet '{self.model_size}' model to {model_dir}")
# _stream_model_download reports 0..1 per file, so remap each file into
# its own slice of the bundle: the bar advances once across the whole
# download instead of restarting for each of the four files.
outer_callback = self._download_progress_callback
total_files = len(parakeet.MODEL_FILES)
def file_progress(index, name):
def report(fraction, speed, status):
if outer_callback:
outer_callback(
(index + fraction) / total_files,
speed,
f"{name} ({index + 1}/{total_files}) - {status}",
)
return report
temp_file = None
try:
for index, filename in enumerate(parakeet.MODEL_FILES):
dest_path = os.path.join(model_dir, filename)
key = parakeet.manifest_key(self.model_size, filename)
# Existence is not enough: a leftover or copied-in file must
# still match its pin. Verify, and only skip the download when
# the digest is good.
if os.path.exists(dest_path):
try:
verify_model_file(dest_path, key)
continue
except ChecksumError as error:
logger.error(
"Existing Parakeet model file at %s is not trustworthy: %s",
dest_path,
error,
)
if expected_for(key) is None:
raise
try:
os.remove(dest_path)
except OSError as remove_error:
logger.error("Could not remove %s: %s", dest_path, remove_error)
raise
logger.info(
"Removed the unverified model file; it will be downloaded again"
)
temp_file = dest_path + ".tmp"
url = parakeet.get_model_file_url(self.model_size, filename)
self._download_progress_callback = file_progress(index, filename)
self._stream_model_download(url, temp_file)
# Verify before the rename, as the whisper.cpp downloader does:
# sherpa-onnx loads these through native code, so an unverified
# file must never reach a path is_model_downloaded() trusts.
verify_model_file(temp_file, key)
os.rename(temp_file, dest_path)
temp_file = None
# RequestException=Exception under the test mocks; do not catch
# Timeout separately (it is not a real exception type there).
except requests.exceptions.RequestException as e:
logger.error(f"Failed to download the Parakeet model: {e}")
if temp_file and os.path.exists(temp_file):
os.remove(temp_file)
msg = str(e).lower()
if "timeout" in msg or type(e).__name__ == "Timeout":
raise RuntimeError(
"Model download timed out (Hugging Face may be slow or unavailable). "
"Check your network and try again."
) from e
raise RuntimeError(f"Failed to download Parakeet model: {e}") from e
except (ChecksumError, OSError, RuntimeError, ValueError) as e:
logger.error(f"An error occurred during Parakeet model download: {e}")
if temp_file and os.path.exists(temp_file):
os.remove(temp_file)
raise
finally:
self._download_progress_callback = outer_callback
logger.info("Parakeet model downloaded successfully")
if self._download_progress_callback:
self._download_progress_callback(1.0, 0, "Complete!")
@staticmethod
def _parakeet_model_is_verified(model_size: str, model_dir: str) -> bool:
"""Hash each bundle file against its pin. Delete only a digest/size mismatch.
An unpinned name is refused, not deleted. If remove fails, return False
so the caller does not hand the files to sherpa-onnx.
"""
verified = True
for filename in parakeet.MODEL_FILES:
path = os.path.join(model_dir, filename)
key = parakeet.manifest_key(model_size, filename)
try:
verify_model_file(path, key)
except ChecksumError as error:
logger.error("Parakeet model file at %s is not trustworthy: %s", path, error)
verified = False
if expected_for(key) is None:
continue
try:
os.remove(path)
except OSError as remove_error:
logger.error("Could not remove %s: %s", path, remove_error)
return False
logger.info("Removed the unverified model file; it will be downloaded again")
return verified
def _init_parakeet(self):
"""Initialize the Parakeet speech recognition engine."""
try:
import sherpa_onnx
# Validate model size for Parakeet
valid_models = list(parakeet.PARAKEET_MODEL_INFO.keys())
if self.model_size not in valid_models:
logger.warning(
f"Model size '{self.model_size}' not valid for Parakeet. "
f"Valid options: {valid_models}. Using '{parakeet.RECOMMENDED_MODEL}' instead."
)
self.model_size = parakeet.RECOMMENDED_MODEL
model_dir = parakeet.get_model_path(self.model_size)
# A bundle that is merely present did not necessarily come through
# the download path: files could be copied in, or left over from a
# cancelled download. sherpa-onnx loads these through native code,
# so hash them here; a failure demotes the bundle to "not downloaded".
if parakeet.is_model_downloaded(
self.model_size
) and not self._parakeet_model_is_verified(self.model_size, model_dir):
if parakeet.is_model_downloaded(self.model_size):
logger.error(
"Refusing to load unverified Parakeet model at %s",
model_dir,
)
self._model_initialized = False
if self._defer_download:
return
raise RuntimeError(f"Parakeet model at {model_dir} failed verification")
if not parakeet.is_model_downloaded(self.model_size):
if self._defer_download:
logger.info(
f"Parakeet model '{self.model_size}' not found at {model_dir}. "
"Will download when needed."
)
self._model_initialized = False
return # Don't block startup
else:
logger.info(f"Downloading Parakeet '{self.model_size}' model...")
self._download_parakeet_model()
# NOTE: do not take _model_lock here. It is a non-reentrant Lock and
# reconfigure()/reinitialize_after_resume() already hold it when they
# call this, so acquiring it would deadlock the caller.
self.model = None
self.model = sherpa_onnx.OfflineRecognizer.from_transducer(
encoder=os.path.join(model_dir, "encoder.int8.onnx"),
decoder=os.path.join(model_dir, "decoder.int8.onnx"),
joiner=os.path.join(model_dir, "joiner.int8.onnx"),
tokens=os.path.join(model_dir, "tokens.txt"),
model_type="nemo_transducer",
feature_dim=128,
num_threads=4,
)
self._model_initialized = True
logger.info(f"Loaded Parakeet model from {model_dir}")
except ImportError as e:
logger.error(f"Failed to import sherpa-onnx: {e}")
logger.error("Please install it with 'pip install sherpa-onnx'")
self.state = RecognitionState.ERROR
raise
except (FileNotFoundError, RuntimeError, OSError) as e:
logger.error(f"Failed to initialize Parakeet engine: {e}", exc_info=True)
self.state = RecognitionState.ERROR
raise
def _transcribe_with_parakeet(self, audio_buffer: list[bytes]) -> str:
"""Transcribe 16-bit PCM chunks (16kHz) with Parakeet."""
import numpy as np
if not audio_buffer:
return ""
audio_data = np.frombuffer(b"".join(audio_buffer), dtype=np.int16)
audio_float = audio_data.astype(np.float32) / 32768.0
# Lock model access to prevent a race with reconfigure() setting self.model to None
with self._model_lock:
if self.model is None:
logger.warning("Model is None during transcription, returning empty result")
return ""
stream = self.model.create_stream()
stream.accept_waveform(16000, audio_float)
self.model.decode_stream(stream)
return stream.result.text.strip()
def _init_remote_api(self):
"""Initialize remote API speech recognition engine.
@@ -3017,6 +3242,9 @@ class SpeechRecognitionManager:
elif self.engine == "whisper_cpp":
text = self._transcribe_with_whispercpp(audio_buffer)
elif self.engine == "parakeet":
text = self._transcribe_with_parakeet(audio_buffer)
elif self.engine == "remote_api":
# Snapshot the HTTP session under lock to prevent race with
# reconfigure() / reinitialize_after_resume() which close/recreate
@@ -3203,6 +3431,13 @@ class SpeechRecognitionManager:
self.language = language
restart_needed = True
# Parakeet never consumes catalog language. Apply after engine/language
# updates so switching TO parakeet also clears a leftover code.
normalized_language = normalize_language_for_engine(self.engine, self.language)
if normalized_language != self.language:
self.language = normalized_language
restart_needed = True
# Update VOSK specific params if provided
if vad_sensitivity is not None:
self.vad_sensitivity = max(1, min(5, int(vad_sensitivity)))
@@ -3289,6 +3524,8 @@ class SpeechRecognitionManager:
self._init_whisper()
elif self.engine == "whisper_cpp":
self._init_whispercpp()
elif self.engine == "parakeet":
self._init_parakeet()
elif self.engine == "remote_api":
self._init_remote_api()
else:
@@ -3516,6 +3753,8 @@ class SpeechRecognitionManager:
self._init_whisper()
elif self.engine == "whisper_cpp":
self._init_whispercpp()
elif self.engine == "parakeet":
self._init_parakeet()
elif self.engine == "remote_api":
self._init_remote_api()
else:
src/vocalinux/ui/config_manager.py
+1
View File
@@ -77,6 +77,7 @@ DEFAULT_CONFIG = {
# the variant is then derived from the selected language, because a bare size
# name ("medium") is indistinguishable from the multilingual variant id.
"whisper_cpp_model_variant": "",
"parakeet_model_size": "v3-european", # Parakeet TDT 0.6B v3 (25 European languages)
"vad_sensitivity": 3, # Voice Activity Detection sensitivity (1-5)
"silence_timeout": 2.0, # Seconds of silence before stopping
"stop_sound_guard_ms": 200, # Small tail trim to avoid the stop sound without clipping speech
src/vocalinux/ui/settings_dialog.py
+191 -15
View File
@@ -34,6 +34,7 @@ from gi.repository import Gdk, GLib, GObject, Gtk, Pango # noqa: E402
from ..common_types import RecognitionState # noqa: E402
from ..speech_recognition.silero_vad import is_silero_available # noqa: E402
from ..utils import parakeet_model_info as parakeet # noqa: E402
from ..utils.paths import models_dir # noqa: E402
from ..utils.update_checker import ( # noqa: E402
DEFAULT_UPDATE_CHANNEL,
@@ -149,6 +150,9 @@ ENGINE_MODELS = {
"whisper_cpp": [
*WHISPERCPP_MODEL_SIZES,
], # whisper.cpp top-level size buckets; variants are selected separately
"parakeet": [
*parakeet.MODEL_SIZES,
], # Parakeet TDT 0.6B int8 bundles
"remote_api": [], # Remote API does not need local models
}
@@ -166,6 +170,7 @@ ENGINE_DISPLAY_NAMES = {
"vosk": "Vosk",
"whisper": "Whisper",
"whisper_cpp": "whisper.cpp",
"parakeet": "Parakeet",
"remote_api": "Remote API",
}
@@ -244,6 +249,25 @@ def _language_is_english(language_id: str) -> bool:
return SUPPORTED_LANGUAGES.get(language_id, {}).get("whisper") == "en"
def _vosk_display_is_coerced_fallback(
previous_engine: Optional[str],
displayed: Optional[str],
remembered: Optional[str],
) -> bool:
"""Return whether ``displayed`` is Vosk's en-us stand-in for ``remembered``.
Vosk has no auto-detect and no model for some catalog languages (e.g. Greek).
Those selections are shown as en-us; that fallback is not a user preference
and must not replace ``remembered`` (including auto).
"""
return (
previous_engine == "vosk"
and displayed == "en-us"
and bool(remembered)
and remembered != "en-us"
)
def _recommended_whispercpp_variant_for_language(
recommended_model: str,
reason: str,
@@ -621,7 +645,13 @@ def get_available_engines():
Detect which speech recognition engines are available/installed.
Returns a dictionary of engine_name -> availability (bool).
"""
engines = {"vosk": False, "whisper": False, "whisper_cpp": False, "remote_api": False}
engines = {
"vosk": False,
"whisper": False,
"whisper_cpp": False,
"parakeet": False,
"remote_api": False,
}
# Check VOSK
try:
@@ -647,6 +677,14 @@ def get_available_engines():
except ImportError:
pass
# Check Parakeet (sherpa-onnx)
try:
import sherpa_onnx # noqa: F401
engines["parakeet"] = True
except ImportError:
pass
# Remote API is always available (only requires requests package)
try:
import requests # noqa: F401
@@ -1453,6 +1491,10 @@ def recommended_model_for_engine(
elif engine == "vosk":
model_id, reason = _get_recommended_vosk_model()
size_mb = VOSK_MODEL_INFO.get(model_id, {}).get("size_mb", 0)
elif engine == "parakeet":
model_id = parakeet.RECOMMENDED_MODEL
reason = parakeet.RECOMMENDED_REASON
size_mb = parakeet.PARAKEET_MODEL_INFO.get(model_id, {}).get("size_mb", 0)
else:
# Remote API transcribes server-side; there is nothing to download.
return None
@@ -1892,6 +1934,12 @@ class SettingsDialog(Gtk.Dialog):
self._processing_language_change = (
False # Flag to prevent recursive language change handling
)
# Last user catalog-language preference across Parakeet visits and across
# engine-coerced fallbacks (e.g. Vosk forcing en-us for Greek). Parakeet
# forces language=auto for honesty; this restores the preference when the
# user returns to Whisper/cpp. Do not store engine-coerced fallbacks here.
self._last_non_parakeet_language = None
self._engine_for_language_memory = None
self._applying_settings = False # Flag to prevent recursive settings application
self._advanced_prompt_dirty = False
self._about_release_url = ""
@@ -4708,6 +4756,14 @@ class SettingsDialog(Gtk.Dialog):
settings = self._get_current_settings()
self.current_engine = settings["engine"]
self.language = settings["language"]
if self.current_engine == "parakeet":
# Config may already be auto; keep any non-auto leftover as restore seed.
self._last_non_parakeet_language = (
self.language if self.language and self.language != "auto" else None
)
else:
self._last_non_parakeet_language = self.language
self._engine_for_language_memory = self.current_engine
self.current_model_size = settings["model_size"]
self.current_vad = settings.get("vad_sensitivity", 3)
self.current_silence = settings.get("silence_timeout", 2.0)
@@ -5003,9 +5059,18 @@ class SettingsDialog(Gtk.Dialog):
def _sync_language_options_for_selected_model(self, preferred_language: Optional[str] = None):
"""Refresh language options and keep the current model/language pair valid."""
engine = self._get_selected_engine()
language_to_keep = (
preferred_language or self.language_combo.get_active_id() or self.language
)
if engine == "parakeet":
# Coverage is the model (v2-english vs v3-european), not this picker.
# Remember a non-auto preferred leftover, but never clobber memory with
# the forced auto that follows an engine switch into Parakeet.
if preferred_language and preferred_language != "auto":
self._last_non_parakeet_language = preferred_language
self.language = "auto"
language_to_keep = "auto"
else:
language_to_keep = (
preferred_language or self.language_combo.get_active_id() or self.language
)
self._processing_language_change = True
try:
@@ -5018,9 +5083,16 @@ class SettingsDialog(Gtk.Dialog):
fallback_language = self._default_language_for_engine(engine)
self._set_combo_active_id_or_first(self.language_combo, fallback_language)
self.language = (
self.language_combo.get_active_id() or self._default_language_for_engine(engine)
)
if engine == "parakeet":
self.language = "auto"
else:
self.language = (
self.language_combo.get_active_id() or self._default_language_for_engine(engine)
)
if self.language and not _vosk_display_is_coerced_fallback(
engine, self.language, self._last_non_parakeet_language
):
self._last_non_parakeet_language = self.language
finally:
self._processing_language_change = False
@@ -5058,6 +5130,8 @@ class SettingsDialog(Gtk.Dialog):
smallest_model = None
if engine == "whisper":
recommended_model, _ = _get_recommended_whisper_model()
elif engine == "parakeet":
recommended_model = parakeet.RECOMMENDED_MODEL
else:
recommended_model, _ = _get_recommended_vosk_model()
@@ -5069,6 +5143,9 @@ class SettingsDialog(Gtk.Dialog):
elif engine == "vosk" and size in VOSK_MODEL_INFO:
info = VOSK_MODEL_INFO[size]
is_downloaded = _is_vosk_model_downloaded(size, self.language)
elif engine == "parakeet":
info = parakeet.PARAKEET_MODEL_INFO[size]
is_downloaded = parakeet.is_model_downloaded(size)
else:
is_downloaded = False
info = {"size_mb": 0}
@@ -5216,6 +5293,9 @@ class SettingsDialog(Gtk.Dialog):
size = (self.model_combo.get_active_id() or "").lower()
language = self.language_combo.get_active_id() or self.language
return vosk_model_dirname(size, language)
if engine == "parakeet":
model_id = self.model_combo.get_active_id()
return model_id.lower() if model_id else None
return None
def _list_unused_downloads(self) -> list[tuple[str, str, str]]:
@@ -5272,6 +5352,16 @@ class SettingsDialog(Gtk.Dialog):
model.dirname == active_id,
)
)
elif engine == "parakeet":
for name in parakeet.list_downloaded_models():
items.append(
(
name,
_model_display_name(name),
_format_size(parakeet.PARAKEET_MODEL_INFO[name]["size_mb"]),
name == active_id,
)
)
return items
@@ -5356,6 +5446,8 @@ class SettingsDialog(Gtk.Dialog):
_delete_whisper_model(model_id)
elif engine == "vosk":
delete_vosk_model(model_id)
elif engine == "parakeet":
parakeet.delete_model(model_id)
else:
raise ValueError(f"No local models to delete for engine {engine}")
@@ -5404,16 +5496,71 @@ class SettingsDialog(Gtk.Dialog):
programmatic = self._initializing or self._applying_settings
current_lang = None if self._applying_settings else self.language_combo.get_active_id()
if current_lang:
if engine == "vosk" and (
current_lang == "auto" or not SUPPORTED_LANGUAGES.get(current_lang, {}).get("vosk")
previous_engine = self._engine_for_language_memory
# When Vosk coerces an unsupported language to en-us, keep the user
# preference in memory and re-apply it after sync (which would otherwise
# write the coerced active value back into memory).
preserve_language_memory = None
vosk_coerced = False
if engine == "parakeet":
# Force auto for honesty, but remember the prior catalog preference so
# leaving Parakeet can restore Whisper/cpp language instead of auto.
# Only capture on entry: a re-fired changed signal while already on
# Parakeet would otherwise overwrite memory with the forced auto.
if previous_engine != "parakeet":
remembered = current_lang or self.language
# Vosk may still display coerced en-us for an unsupported
# catalog language. Entering Parakeet must not record that
# fallback over the pre-coercion preference (e.g. Greek or auto).
if remembered and not _vosk_display_is_coerced_fallback(
previous_engine, remembered, self._last_non_parakeet_language
):
self._last_non_parakeet_language = remembered
self.language = "auto"
elif current_lang is None and self._applying_settings:
# Programmatic resync must not rewrite the language the user picked.
pass
else:
# Prefer remembered preference over a prior engine's coerced active
# value (Vosk en-us for Greek). Trust the combo for other engines.
if (
current_lang
and current_lang != "auto"
and not _vosk_display_is_coerced_fallback(
previous_engine, current_lang, self._last_non_parakeet_language
)
):
self.language = "en-us"
elif engine in ["whisper", "whisper_cpp", "remote_api"] and not current_lang:
self.language = "auto"
chosen = current_lang
elif self._last_non_parakeet_language:
chosen = self._last_non_parakeet_language
else:
chosen = current_lang or self.language or self._default_language_for_engine(engine)
if engine == "vosk" and (
chosen == "auto" or not SUPPORTED_LANGUAGES.get(chosen, {}).get("vosk")
):
# Keep the pre-coercion preference (including auto) so sync cannot
# persist Vosk's en-us fallback as the remembered Whisper language.
preserve_language_memory = (
chosen
if chosen and chosen != "auto"
else (self._last_non_parakeet_language or "auto")
)
chosen = "en-us"
vosk_coerced = True
self.language = chosen
# Never store the coerced en-us fallback as the user preference.
if vosk_coerced:
self._last_non_parakeet_language = preserve_language_memory
elif chosen:
self._last_non_parakeet_language = chosen
self._engine_for_language_memory = engine
self._populate_model_options()
self._sync_language_options_for_selected_model(self.language)
if vosk_coerced and preserve_language_memory is not None:
self._last_non_parakeet_language = preserve_language_memory
self._update_engine_specific_ui()
self._update_model_info()
self._update_voice_commands_for_engine()
@@ -5519,7 +5666,7 @@ class SettingsDialog(Gtk.Dialog):
continue
is_downloaded = _is_vosk_model_downloaded("small", lang_code)
display_text += " ✓" if is_downloaded else " ↓"
elif engine in ["whisper", "whisper_cpp", "remote_api"]:
elif engine in ["whisper", "whisper_cpp", "parakeet", "remote_api"]:
if english_only_whispercpp and lang_info.get("whisper") != "en":
continue
# Both Whisper and whisper.cpp support auto-detect
@@ -5532,6 +5679,11 @@ class SettingsDialog(Gtk.Dialog):
def _update_language_warning(self):
"""Update language help text for the selected engine/model/language."""
if self._get_selected_engine() == "parakeet":
self.language_warning.set_markup("")
self.language_warning.hide()
return
lang_code = self.language_combo.get_active_id()
lang_info = SUPPORTED_LANGUAGES.get(lang_code, {})
@@ -5574,6 +5726,8 @@ class SettingsDialog(Gtk.Dialog):
self._processing_language_change = True
try:
self.language = lang_code
if _engine_from_display(engine) != "parakeet":
self._last_non_parakeet_language = lang_code
self._populate_model_options()
self._update_language_warning()
self._update_model_info()
@@ -5636,6 +5790,11 @@ class SettingsDialog(Gtk.Dialog):
self._update_model_info()
self._refresh_unused_downloads()
self._update_language_warning()
if engine == "parakeet":
self.language_row.hide()
self.language_warning.hide()
else:
self.language_row.show_all()
self._update_model_picker_tooltips()
self._update_advanced_tab_sensitivity()
@@ -5709,6 +5868,14 @@ class SettingsDialog(Gtk.Dialog):
is_downloaded = _is_vosk_model_downloaded(model_name, self.language)
recommended, reason = _get_recommended_vosk_model()
extra_info = f"Size: {_format_size(info['size_mb'])}"
elif engine == "parakeet":
if model_name not in parakeet.PARAKEET_MODEL_INFO:
self.model_info_card.hide()
return
info = parakeet.PARAKEET_MODEL_INFO[model_name]
is_downloaded = parakeet.is_model_downloaded(model_name)
recommended, reason = parakeet.RECOMMENDED_MODEL, parakeet.RECOMMENDED_REASON
extra_info = f"Size: {_format_size(info['size_mb'])}"
else:
self.model_info_card.hide()
return
@@ -5789,6 +5956,9 @@ class SettingsDialog(Gtk.Dialog):
elif engine == "vosk" and not _is_vosk_model_downloaded(model_name, self.language):
needs_download = True
model_info = VOSK_MODEL_INFO.get(model_name, {"size_mb": 50})
elif engine == "parakeet" and not parakeet.is_model_downloaded(model_name):
needs_download = True
model_info = parakeet.PARAKEET_MODEL_INFO.get(model_name, {"size_mb": 639})
if needs_download:
if not self.speech_engine.try_begin_download():
@@ -6003,7 +6173,10 @@ class SettingsDialog(Gtk.Dialog):
model_variant = ""
else:
model_size = model_id.lower() if model_id else "small"
language = language_id if language_id else self._default_language_for_engine(engine)
if engine == "parakeet":
language = "auto"
else:
language = language_id if language_id else self._default_language_for_engine(engine)
vad = int(self.vad_spin.get_value())
silence = self.silence_spin.get_value()
@@ -6242,6 +6415,9 @@ For now, the engine has been reverted to VOSK."""
elif engine == "vosk" and not _is_vosk_model_downloaded(model_name, self.language):
needs_download = True
model_info = VOSK_MODEL_INFO.get(model_name, {"size_mb": 50})
elif engine == "parakeet" and not parakeet.is_model_downloaded(model_name):
needs_download = True
model_info = parakeet.PARAKEET_MODEL_INFO.get(model_name, {"size_mb": 639})
if needs_download:
if not self.speech_engine.try_begin_download():
src/vocalinux/utils/model_checksums.txt
+12 -1
View File
@@ -2,7 +2,7 @@
#
# Generated by scripts/generate-model-checksums.py -- do not edit by hand.
# Regenerate with `just model-checksums` after adding a model to
# vosk_model_info.py or whispercpp_model_info.py.
# vosk_model_info.py, whispercpp_model_info.py or parakeet_model_info.py.
#
# Read by src/vocalinux/utils/model_checksums.py and by install.sh; keep the
# format (filename, algorithm, digest, size in bytes) parseable by awk.
@@ -16,6 +16,9 @@
#
# VOSK digests are the sha256 of bytes this script downloaded, not the md5
# Alphacephei publishes; that md5 is only checked against them at that point.
#
# Parakeet bundles are pinned per model in parakeet_model_info.py, so their
# keys carry the model name: several bundles ship the same file names.
ggml-base-q5_1.bin sha256 422f1ae452ade6f30a004d7e5c6a43195e4433bc370bf23fac9cc591f01a8898 59707625
ggml-base-q8_0.bin sha256 c577b9a86e7e048a0b7eada054f4dd79a56bbfa911fbdacf900ac5b567cbb7d9 81768585
@@ -84,3 +87,11 @@ vosk-model-small-uk-v3-small.zip sha256 ee05b6a8c796c53c61cd1c821d38e897
vosk-model-small-vn-0.4.zip sha256 efe5c8494212110471a79befc48c79da679e5b1fc52a4ffb500222ff86d622e5 33656337
vosk-model-uk-v3.zip sha256 b1a0dbb9a19bfb6cdbad5bb4d43b71281ab57ad4a8c2a372656c83b994b72b1f 371048965
vosk-model-vn-0.4.zip sha256 a8ce6ee52e369b8f23f56e87e037db10322e6d94779b08d5e1ab21cb4b737380 74423562
parakeet-v3-european-encoder.int8.onnx sha256 acfc2b4456377e15d04f0243af540b7fe7c992f8d898d751cf134c3a55fd2247 652184281
parakeet-v3-european-decoder.int8.onnx sha256 179e50c43d1a9de79c8a24149a2f9bac6eb5981823f2a2ed88d655b24248db4e 11845275
parakeet-v3-european-joiner.int8.onnx sha256 3164c13fc2821009440d20fcb5fdc78bff28b4db2f8d0f0b329101719c0948b3 6355277
parakeet-v3-european-tokens.txt sha256 d58544679ea4bc6ac563d1f545eb7d474bd6cfa467f0a6e2c1dc1c7d37e3c35d 93939
parakeet-v2-english-encoder.int8.onnx sha256 a32b12d17bbbc309d0686fbbcc2987b5e9b8333a7da83fa6b089f0a2acd651ab 652184296
parakeet-v2-english-decoder.int8.onnx sha256 b6bb64963457237b900e496ee9994b59294526439fbcc1fecf705b31a15c6b4e 7257753
parakeet-v2-english-joiner.int8.onnx sha256 7946164367946e7f9f29a122407c3252b680dbae9a51343eb2488d057c3c43d2 1739080
parakeet-v2-english-tokens.txt sha256 ec182b70dd42113aff6c5372c75cac58c952443eb22322f57bbd7f53977d497d 9384
src/vocalinux/utils/parakeet_model_info.py
+140
View File
@@ -0,0 +1,140 @@
"""
Parakeet model information for Vocalinux.
This module provides model metadata and download URLs for the Parakeet
engine, which runs NVIDIA NeMo ASR models through sherpa-onnx.
"""
import logging
import os
import shutil
from typing import Any, Dict
from .paths import is_within_directory, models_dir
logger = logging.getLogger(__name__)
# Parakeet model information
# Models are downloaded from Hugging Face as sherpa-onnx ONNX bundles.
#
# Each bundle is pinned to a Hugging Face commit rather than tracking main, so
# a file replaced upstream cannot change what an install downloads.
_HF_BASE_URL = "https://huggingface.co"
PARAKEET_MODEL_INFO: Dict[str, Dict[str, Any]] = {
"v3-european": {
"repo": "csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8",
"revision": "2bda32ec70b097a55adaa07d9a7173915b43cc78",
"size_mb": 639,
"desc": "Parakeet TDT 0.6B v3 (int8), 25 European languages",
},
"v2-english": {
"repo": "csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8",
"revision": "1ab9323565ddb038682214b292f588070a538ce2",
"size_mb": 630,
"desc": "Parakeet TDT 0.6B v2 (int8), English",
},
}
MODEL_SIZES = ["v3-european", "v2-english"]
# Model offered when the engine has nothing downloaded yet.
RECOMMENDED_MODEL = "v3-european"
RECOMMENDED_REASON = "25 European languages"
# Files that make up a sherpa-onnx transducer bundle.
MODEL_FILES = ["encoder.int8.onnx", "decoder.int8.onnx", "joiner.int8.onnx", "tokens.txt"]
def get_model_path(model_name: str) -> str:
"""
Get the path where a model should be stored.
Args:
model_name: Name of the model (for example v3-european)
Returns:
Path to the model directory
"""
parakeet_dir = os.path.join(models_dir(), "parakeet")
os.makedirs(parakeet_dir, exist_ok=True)
return os.path.join(parakeet_dir, model_name)
def is_model_downloaded(model_name: str) -> bool:
"""
Check if a Parakeet model is downloaded.
Args:
model_name: Name of the model
Returns:
True if model exists, False otherwise
"""
model_path = get_model_path(model_name)
return all(os.path.exists(os.path.join(model_path, f)) for f in MODEL_FILES)
def list_downloaded_models() -> list[str]:
"""Return Parakeet models occupying disk, complete or partially downloaded.
Mirrors the VOSK helper in listing anything present so Settings can offer
deletion; a cancelled download leaves files behind that must be reclaimable.
"""
return [name for name in MODEL_SIZES if os.path.isdir(get_model_path(name))]
def delete_model(model_name: str) -> str:
"""Delete a downloaded Parakeet model directory.
Returns:
The removed filesystem path.
Raises:
ValueError: Unknown model name, or path outside the models directory.
FileNotFoundError: The model directory is not present.
OSError: The directory could not be removed.
"""
if model_name not in PARAKEET_MODEL_INFO or os.path.basename(model_name) != model_name:
raise ValueError(f"Unknown Parakeet model: {model_name}")
path = get_model_path(model_name)
if not is_within_directory(path, models_dir()):
raise ValueError("Refusing to delete a path outside the models directory")
if not os.path.isdir(path):
raise FileNotFoundError(path)
shutil.rmtree(path)
logger.info("Deleted Parakeet model %s (%s)", model_name, path)
return path
def manifest_key(model_name: str, filename: str) -> str:
"""Key this bundle file is pinned under in model_checksums.txt.
Several bundles ship the same file names, so the model name is part of the
key rather than the bare filename other engines use.
"""
return f"parakeet-{model_name}-{filename}"
def get_model_file_url(model_name: str, filename: str) -> str:
"""
Get the download URL for one file of a Parakeet model.
Args:
model_name: Name of the model
filename: File within the model bundle
Returns:
Hugging Face download URL
"""
model_info = PARAKEET_MODEL_INFO.get(model_name)
if not model_info:
raise ValueError(f"Unknown Parakeet model: {model_name}")
return (
f"{_HF_BASE_URL}/{model_info['repo']}/resolve/{model_info['revision']}"
f"/{filename}?download=true"
)