feat(diarization): per-speaker transcripts via TinyDiarize + --transcribe-file (#884)

* feat(transcribe): add TinyDiarize file transcription surface

Catalog small.en-tdrz (hosted at akashmjn/tinydiarize-whisper.cpp,
pinned commit) while keeping it out of every dictation-model selection
path: its [SPEAKER_TURN] markup would type noise into the focused
window. The checksum generator resolves side-repo models at their own
pinned revision, parakeet-style.

Add speech_recognition.diarization (strict-WAV fast path with ffmpeg
fallback, tdrz transcription, speaker-turn parsing, timestamped
formatting) and surface it through a tray Transcribe Audio File flow
with on-demand model download plus a headless --transcribe-file CLI.
TranscriptDialog offers copy and save of the diarized transcript.

Closes #756

* docs(user-guide): document --transcribe-file

* fix(transcribe): pass tdrz_enable as a pywhispercpp kwarg

pywhispercpp's Model takes decode params as keyword arguments; a
params= dict crashes on the C-level whisper_full_params object. Also
let --transcribe-file report unexpected failures cleanly instead of
dumping a traceback.

* fix(tray): dismiss the tdrz download dialog on response

* test(diarization): pin real wave/numpy/tempfile around sibling mocks

Several test modules replace sys.modules entries with MagicMocks at
import time and never restore them, so diarization's wave/numpy/
tempfile bindings resolved to mocks when this file ran after them in
CI. An autouse fixture reimports the real stdlib modules per test, the
same remedy test_audio_feedback already applies for wave.

* fix(transcribe): verify tdrz model, preload libs, gate formats on ffmpeg

- Hash the on-disk TinyDiarize model against the pinned digest before
  native code maps it, matching the dictation load path
- Extract the bundled-lib preloader to utils/pywhispercpp_loader.py and
  preload before the lazy pywhispercpp import so the headless
  --transcribe-file path works on source-built installs without RPATH
- Offer only WAV in the file picker when ffmpeg is absent; name the
  dependency in the filter label
- Type the new tray/diarization callbacks; document --transcribe-file
  and the new modules in AGENTS.md

* style(audio): finish type annotations on capture helpers and download path

_setup_alsa_error_handler now declares Optional[Any] and its ctypes
callback gets concrete parameter types; the remaining capture helpers
take audio: Any (PyAudio has no importable type) and test_audio_input's
device_index becomes Optional[int], which also clears the preexisting
implicit-Optional error. _download_whispercpp_model gains -> None and
its HF url is narrowed to str so the concatenation and download call
stop adding new mypy findings.

* style(tests): wrap long monkeypatch line in preload test

* refactor(main): narrow transcribe-file errors and add file context

* style(ui): annotate TranscriptDialog constructor

* fix(diarize): log worker tracebacks, isort test helpers, document --transcribe-file

---------

Co-authored-by: Jatin K Malik <jatinkrmalik@gmail.com>
This commit is contained in:
devin-ai-integration[bot]
2026-09-30 00:44:55 -07:00
committed by GitHub
co-authored by Jatin K Malik
parent b4babcaeca
commit 8ebc4ce2ba
21 changed files with 1421 additions and 193 deletions
+42 -7
View File
@@ -114,13 +114,12 @@ def _model_names() -> tuple[list[str], list[str]]:
"""Model names the application can actually download, straight from the package."""
sys.path.insert(0, str(SRC))
from vocalinux.utils.vosk_model_info import VOSK_MODEL_INFO
from vocalinux.utils.whispercpp_model_info import WHISPERCPP_MODEL_INFO, whispercpp_model_file
from vocalinux.utils.whispercpp_model_info import WHISPERCPP_MODEL_INFO
vosk = {
name for info in VOSK_MODEL_INFO.values() for name in info["languages"].values() if name
}
whispercpp = {whispercpp_model_file(name) for name in WHISPERCPP_MODEL_INFO}
return sorted(whispercpp), sorted(vosk)
return sorted(WHISPERCPP_MODEL_INFO), sorted(vosk)
def _parakeet_entries() -> list[Entry]:
@@ -205,7 +204,27 @@ def _faster_whisper_entries() -> list[Entry]:
return entries
def _hf_lfs_digests(repo: str, revision: str) -> dict[str, tuple[str, int]]:
"""sha256/size per LFS file in ``repo`` at ``revision``, from the HF API."""
payload = _fetch_json(
f"https://huggingface.co/api/models/{repo}/revision/{revision}?blobs=true"
)
published = {}
for sibling in payload.get("siblings", []):
lfs = sibling.get("lfs") or {}
digest = lfs.get("sha256") or lfs.get("oid")
if digest:
published[sibling["rfilename"]] = (digest, int(lfs.get("size", 0)))
return published
def _whispercpp_entries(wanted: Iterable[str]) -> tuple[list[Entry], str]:
sys.path.insert(0, str(SRC))
from vocalinux.utils.whispercpp_model_info import (
whispercpp_model_file,
whispercpp_model_source,
)
payload = _fetch_json(HF_API)
revision = payload.get("sha")
if not revision:
@@ -218,20 +237,36 @@ def _whispercpp_entries(wanted: Iterable[str]) -> tuple[list[Entry], str]:
if digest:
by_name[sibling["rfilename"]] = (digest, int(lfs.get("size", 0)))
# Catalog entries hosted outside the main repository pin their own commit
# in whispercpp_model_info.py (same contract parakeet bundles follow).
side_repo_digests: dict[str, dict[str, tuple[str, int]]] = {}
entries, missing = [], []
for filename in wanted:
if filename not in by_name:
for model_name in wanted:
filename = whispercpp_model_file(model_name)
repo, pinned_revision = whispercpp_model_source(model_name)
if repo != HF_REPO:
if repo not in side_repo_digests:
side_repo_digests[repo] = _hf_lfs_digests(repo, pinned_revision)
found = side_repo_digests[repo].get(filename)
if found is None:
missing.append(f"{filename} ({repo})")
continue
digest, size = found
elif filename not in by_name:
missing.append(filename)
continue
digest, size = by_name[filename]
else:
digest, size = by_name[filename]
entries.append(Entry(filename, "sha256", digest, size))
if missing:
raise SystemExit(
f"Hugging Face {HF_REPO}@{revision} publishes no sha256 for: {', '.join(missing)}.\n"
f"Hugging Face publishes no sha256 for: {', '.join(missing)}.\n"
"Either the model was renamed upstream or whispercpp_model_info.py lists "
"a file that does not exist."
)
entries.sort(key=lambda entry: entry.filename)
return entries, revision