feat(asr): add Parakeet TDT 0.6B engine via sherpa-onnx (#802)
* feat(asr): add Parakeet engine via sherpa-onnx Adds a "parakeet" engine backed by sherpa-onnx, running NVIDIA Parakeet TDT 0.6B int8 models on CPU: v3 (25 European languages, auto-detected) and v2 (English). Output is punctuated and capitalized. On an Intel Core Ultra 7 155H it decodes at about 12x realtime on CPU (RTF 0.08), roughly 8x faster than whisper.cpp large-v3-turbo over the same clips with its Vulkan GPU backend (RTF 0.59). Being a transducer it also has no 30s window to pad, so a short push-to-talk utterance returns in ~0.3s instead of ~2.4s. Follows the existing engine layout: utils/parakeet_model_info.py holds the catalog and the get_model_path / is_model_downloaded / list_downloaded_models / delete_model helpers, while _init_parakeet / _transcribe_with_parakeet / _download_parakeet_model follow the same shape as their whisper.cpp counterparts and reuse _stream_model_download for progress and cancellation. Registered in the engine dispatch, the settings engine/model/language pickers, the model info card, the download prompts, the unused-downloads list and install.sh. Models download to <models_dir>/parakeet/<model>/ via temp files renamed on success; is_model_downloaded requires all four files, so an interrupted download is retried rather than loaded. Requires the new "parakeet" extra (sherpa-onnx). * fix(install): write only the engine key for parakeet The parakeet branch duplicated a full default config.json. ConfigManager deep-merges DEFAULT_CONFIG on load, so every other key was redundant: a config holding only "engine": "parakeet" still resolves parakeet_model_size to v3-european along with the VAD, shortcut and text-injection defaults. * fix(install): keep push-to-talk defaults and switch existing configs Writing only the engine key left config.json without a shortcuts section, which _migrate_shortcuts_config reads as a pre-push-to-talk upgrade and pins back to ctrl+ctrl/toggle - contradicting the installer's own "Hold Right Alt while you speak" message. Seed that section too. Point the config at parakeet with set_configured_engine rather than only on first create, so --engine=parakeet also switches a machine that already has a config.json, and give print_welcome_message a parakeet arm so the summary is not blank. The interactive picker overwrites SELECTED_ENGINE, so document the flag combination that reaches this branch. * fix(asr): pin Parakeet bundles and report one progress bar Pin each bundle to a Hugging Face commit instead of resolve/main, so a file replaced upstream cannot change what an install downloads. This is the same approach whispercpp_model_info takes with its pinned revision. Remap each file's progress into its slice of the bundle. The four files were reported through the single-file downloader independently, so the fraction ran to 1.0 and restarted four times; it now advances once and names the file being fetched. * fix(asr): reject a truncated Parakeet download _stream_model_download only rejects an empty body, so a truncated but nonempty response was renamed into the model directory and then reported as installed, because is_model_downloaded checks existence. Record the byte size of each file at the pinned revision and check it before the rename, the same order whisper.cpp verifies its digest in. A short file is a truncated transfer rather than an upstream change, since the revision is pinned, so it raises and the temp file is removed. * style(asr): annotate the Parakeet model catalog mypy inferred the catalog values as object, so reading file_sizes out of an entry raised attr-defined and no-any-return under `just typecheck`. * fix(asr): pin Parakeet digests in model_checksums.txt AGENTS.md requires every speech model to be verified against a pinned digest before install, failing closed on an unpinned one. Parakeet was checking only the byte size. Teach scripts/generate-model-checksums.py to emit Parakeet entries: the ONNX files are LFS so Hugging Face publishes their sha256, and tokens.txt is hashed from the downloaded bytes the way VOSK archives are. Several bundles ship the same file names, so a key carries the model name. _download_parakeet_model now calls verify_model_file before the rename and catches ChecksumError alongside the other download failures, matching the whisper.cpp downloader. * fix(asr): verify on-disk Parakeet models before load Existing bundle files were trusted by path alone, so a leftover or copied-in tree could reach sherpa-onnx unchecked. Hash each file against its pin on download skip and on init, and cover the Parakeet pins in CI. * fix(ui): hide Parakeet language picker that ignores the setting Parakeet language coverage is the model choice, not a catalog language. Hide the picker and persist auto so Settings does not write a lying value. * fix(asr): normalize Parakeet language to auto at startup CLI --language and stale saved languages still reached SpeechRecognitionManager even though Parakeet ignores catalog language. Force auto on the shared helper used by main and the manager so stored settings stay honest. * fix(ui): restore language after leaving Parakeet Remember the prior Whisper/cpp language across a Parakeet visit so switching back does not leave recognition stuck on auto. Keep Parakeet forced to auto and the picker hidden. Also satisfy black on the startup normalizer path. * fix(ui): keep language memory across Vosk fallback Vosk's unsupported-language en-us coercion was writing into _last_non_parakeet_language, so Whisper→Parakeet→Vosk→Whisper restored en-us instead of the user's Greek (or other) preference. Remember the pre-coercion catalog language and prefer it when leaving Vosk. * fix(ui): keep language memory when entering Parakeet from Vosk Vosk's coerced en-us fallback was still written into _last_non_parakeet_language on Parakeet entry, so Vosk→Parakeet→Whisper restored en-us instead of the pre-coercion preference (Greek or auto). Share one coerced-fallback predicate across entry, leave-Vosk, and language sync, and cover those paths with regression tests. * fix(tests): set initializing flags false on language-change stubs Plain Mock attributes are truthy and were short-circuiting the handler. --------- Co-authored-by: Alex Thunder <AlexThunder@gmail.com>
This commit is contained in:
Jatin K Malik
co-authored by
Alex Thunder
Alex Thunder
parent
946e19ee41
commit
8f4aeca2f1
@@ -122,6 +122,46 @@ def _model_names() -> tuple[list[str], list[str]]:
|
||||
return sorted(whispercpp), sorted(vosk)
|
||||
|
||||
|
||||
def _parakeet_entries() -> list[Entry]:
|
||||
"""Digests for every file of every Parakeet bundle, at its pinned revision.
|
||||
|
||||
The ONNX files are LFS, so Hugging Face publishes their sha256; tokens.txt
|
||||
is a plain blob and is hashed from the bytes this script downloads, the same
|
||||
way VOSK archives are.
|
||||
"""
|
||||
sys.path.insert(0, str(SRC))
|
||||
from vocalinux.utils.parakeet_model_info import (
|
||||
MODEL_FILES,
|
||||
PARAKEET_MODEL_INFO,
|
||||
manifest_key,
|
||||
)
|
||||
|
||||
entries = []
|
||||
for model_name, info in PARAKEET_MODEL_INFO.items():
|
||||
repo, revision = info["repo"], info["revision"]
|
||||
payload = _fetch_json(
|
||||
f"https://huggingface.co/api/models/{repo}/revision/{revision}?blobs=true"
|
||||
)
|
||||
published = {}
|
||||
for sibling in payload.get("siblings", []):
|
||||
lfs = sibling.get("lfs") or {}
|
||||
digest = lfs.get("oid") or lfs.get("sha256")
|
||||
if digest:
|
||||
published[sibling["rfilename"]] = (digest, int(lfs.get("size", 0)))
|
||||
|
||||
for filename in MODEL_FILES:
|
||||
key = manifest_key(model_name, filename)
|
||||
if filename in published:
|
||||
digest, size = published[filename]
|
||||
else:
|
||||
url = f"https://huggingface.co/{repo}/resolve/{revision}/{filename}?download=true"
|
||||
print(f" {key} (not LFS, hashing bytes)...", flush=True)
|
||||
digest, size = _download_and_hash(url, "")
|
||||
entries.append(Entry(key, "sha256", digest, size))
|
||||
|
||||
return entries
|
||||
|
||||
|
||||
def _whispercpp_entries(wanted: Iterable[str]) -> tuple[list[Entry], str]:
|
||||
payload = _fetch_json(HF_API)
|
||||
revision = payload.get("sha")
|
||||
@@ -192,7 +232,7 @@ def _render(entries: list[Entry], revision: str) -> str:
|
||||
"#",
|
||||
"# Generated by scripts/generate-model-checksums.py -- do not edit by hand.",
|
||||
"# Regenerate with `just model-checksums` after adding a model to",
|
||||
"# vosk_model_info.py or whispercpp_model_info.py.",
|
||||
"# vosk_model_info.py, whispercpp_model_info.py or parakeet_model_info.py.",
|
||||
"#",
|
||||
"# Read by src/vocalinux/utils/model_checksums.py and by install.sh; keep the",
|
||||
"# format (filename, algorithm, digest, size in bytes) parseable by awk.",
|
||||
@@ -206,6 +246,9 @@ def _render(entries: list[Entry], revision: str) -> str:
|
||||
"#",
|
||||
"# VOSK digests are the sha256 of bytes this script downloaded, not the md5",
|
||||
"# Alphacephei publishes; that md5 is only checked against them at that point.",
|
||||
"#",
|
||||
"# Parakeet bundles are pinned per model in parakeet_model_info.py, so their",
|
||||
"# keys carry the model name: several bundles ship the same file names.",
|
||||
"",
|
||||
]
|
||||
for entry in entries:
|
||||
@@ -233,7 +276,12 @@ def main() -> int:
|
||||
vosk_entries = _vosk_entries(vosk_names, args.refresh)
|
||||
print(f" VOSK: {len(vosk_entries)} sha256 digests")
|
||||
|
||||
MANIFEST.write_text(_render(whispercpp_entries + vosk_entries, revision), encoding="utf-8")
|
||||
parakeet_entries = _parakeet_entries()
|
||||
print(f" Parakeet: {len(parakeet_entries)} sha256 digests")
|
||||
|
||||
MANIFEST.write_text(
|
||||
_render(whispercpp_entries + vosk_entries + parakeet_entries, revision), encoding="utf-8"
|
||||
)
|
||||
print(f"Wrote {MANIFEST.relative_to(REPO_ROOT)}")
|
||||
return 0
|
||||
|
||||
|
||||
Reference in New Issue
Block a user