feat(diarization): per-speaker transcripts via TinyDiarize + --transcribe-file (#884)
* feat(transcribe): add TinyDiarize file transcription surface Catalog small.en-tdrz (hosted at akashmjn/tinydiarize-whisper.cpp, pinned commit) while keeping it out of every dictation-model selection path: its [SPEAKER_TURN] markup would type noise into the focused window. The checksum generator resolves side-repo models at their own pinned revision, parakeet-style. Add speech_recognition.diarization (strict-WAV fast path with ffmpeg fallback, tdrz transcription, speaker-turn parsing, timestamped formatting) and surface it through a tray Transcribe Audio File flow with on-demand model download plus a headless --transcribe-file CLI. TranscriptDialog offers copy and save of the diarized transcript. Closes #756 * docs(user-guide): document --transcribe-file * fix(transcribe): pass tdrz_enable as a pywhispercpp kwarg pywhispercpp's Model takes decode params as keyword arguments; a params= dict crashes on the C-level whisper_full_params object. Also let --transcribe-file report unexpected failures cleanly instead of dumping a traceback. * fix(tray): dismiss the tdrz download dialog on response * test(diarization): pin real wave/numpy/tempfile around sibling mocks Several test modules replace sys.modules entries with MagicMocks at import time and never restore them, so diarization's wave/numpy/ tempfile bindings resolved to mocks when this file ran after them in CI. An autouse fixture reimports the real stdlib modules per test, the same remedy test_audio_feedback already applies for wave. * fix(transcribe): verify tdrz model, preload libs, gate formats on ffmpeg - Hash the on-disk TinyDiarize model against the pinned digest before native code maps it, matching the dictation load path - Extract the bundled-lib preloader to utils/pywhispercpp_loader.py and preload before the lazy pywhispercpp import so the headless --transcribe-file path works on source-built installs without RPATH - Offer only WAV in the file picker when ffmpeg is absent; name the dependency in the filter label - Type the new tray/diarization callbacks; document --transcribe-file and the new modules in AGENTS.md * style(audio): finish type annotations on capture helpers and download path _setup_alsa_error_handler now declares Optional[Any] and its ctypes callback gets concrete parameter types; the remaining capture helpers take audio: Any (PyAudio has no importable type) and test_audio_input's device_index becomes Optional[int], which also clears the preexisting implicit-Optional error. _download_whispercpp_model gains -> None and its HF url is narrowed to str so the concatenation and download call stop adding new mypy findings. * style(tests): wrap long monkeypatch line in preload test * refactor(main): narrow transcribe-file errors and add file context * style(ui): annotate TranscriptDialog constructor * fix(diarize): log worker tracebacks, isort test helpers, document --transcribe-file --------- Co-authored-by: Jatin K Malik <jatinkrmalik@gmail.com>
This commit is contained in:
co-authored by
Jatin K Malik
parent
b4babcaeca
commit
8ebc4ce2ba
@@ -114,13 +114,12 @@ def _model_names() -> tuple[list[str], list[str]]:
|
||||
"""Model names the application can actually download, straight from the package."""
|
||||
sys.path.insert(0, str(SRC))
|
||||
from vocalinux.utils.vosk_model_info import VOSK_MODEL_INFO
|
||||
from vocalinux.utils.whispercpp_model_info import WHISPERCPP_MODEL_INFO, whispercpp_model_file
|
||||
from vocalinux.utils.whispercpp_model_info import WHISPERCPP_MODEL_INFO
|
||||
|
||||
vosk = {
|
||||
name for info in VOSK_MODEL_INFO.values() for name in info["languages"].values() if name
|
||||
}
|
||||
whispercpp = {whispercpp_model_file(name) for name in WHISPERCPP_MODEL_INFO}
|
||||
return sorted(whispercpp), sorted(vosk)
|
||||
return sorted(WHISPERCPP_MODEL_INFO), sorted(vosk)
|
||||
|
||||
|
||||
def _parakeet_entries() -> list[Entry]:
|
||||
@@ -205,7 +204,27 @@ def _faster_whisper_entries() -> list[Entry]:
|
||||
return entries
|
||||
|
||||
|
||||
def _hf_lfs_digests(repo: str, revision: str) -> dict[str, tuple[str, int]]:
|
||||
"""sha256/size per LFS file in ``repo`` at ``revision``, from the HF API."""
|
||||
payload = _fetch_json(
|
||||
f"https://huggingface.co/api/models/{repo}/revision/{revision}?blobs=true"
|
||||
)
|
||||
published = {}
|
||||
for sibling in payload.get("siblings", []):
|
||||
lfs = sibling.get("lfs") or {}
|
||||
digest = lfs.get("sha256") or lfs.get("oid")
|
||||
if digest:
|
||||
published[sibling["rfilename"]] = (digest, int(lfs.get("size", 0)))
|
||||
return published
|
||||
|
||||
|
||||
def _whispercpp_entries(wanted: Iterable[str]) -> tuple[list[Entry], str]:
|
||||
sys.path.insert(0, str(SRC))
|
||||
from vocalinux.utils.whispercpp_model_info import (
|
||||
whispercpp_model_file,
|
||||
whispercpp_model_source,
|
||||
)
|
||||
|
||||
payload = _fetch_json(HF_API)
|
||||
revision = payload.get("sha")
|
||||
if not revision:
|
||||
@@ -218,20 +237,36 @@ def _whispercpp_entries(wanted: Iterable[str]) -> tuple[list[Entry], str]:
|
||||
if digest:
|
||||
by_name[sibling["rfilename"]] = (digest, int(lfs.get("size", 0)))
|
||||
|
||||
# Catalog entries hosted outside the main repository pin their own commit
|
||||
# in whispercpp_model_info.py (same contract parakeet bundles follow).
|
||||
side_repo_digests: dict[str, dict[str, tuple[str, int]]] = {}
|
||||
|
||||
entries, missing = [], []
|
||||
for filename in wanted:
|
||||
if filename not in by_name:
|
||||
for model_name in wanted:
|
||||
filename = whispercpp_model_file(model_name)
|
||||
repo, pinned_revision = whispercpp_model_source(model_name)
|
||||
if repo != HF_REPO:
|
||||
if repo not in side_repo_digests:
|
||||
side_repo_digests[repo] = _hf_lfs_digests(repo, pinned_revision)
|
||||
found = side_repo_digests[repo].get(filename)
|
||||
if found is None:
|
||||
missing.append(f"{filename} ({repo})")
|
||||
continue
|
||||
digest, size = found
|
||||
elif filename not in by_name:
|
||||
missing.append(filename)
|
||||
continue
|
||||
digest, size = by_name[filename]
|
||||
else:
|
||||
digest, size = by_name[filename]
|
||||
entries.append(Entry(filename, "sha256", digest, size))
|
||||
|
||||
if missing:
|
||||
raise SystemExit(
|
||||
f"Hugging Face {HF_REPO}@{revision} publishes no sha256 for: {', '.join(missing)}.\n"
|
||||
f"Hugging Face publishes no sha256 for: {', '.join(missing)}.\n"
|
||||
"Either the model was renamed upstream or whispercpp_model_info.py lists "
|
||||
"a file that does not exist."
|
||||
)
|
||||
entries.sort(key=lambda entry: entry.filename)
|
||||
return entries, revision
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user