fix(whispercpp): don't default to all CPU cores on hybrid processors (#492)

The CPU-only auto thread count was `min(cpu_count(), 8)`. On an 8-core
hybrid CPU (e.g. Intel Core Ultra, P+E cores) this selects 8 = every
core. whisper.cpp synchronises all worker threads at each decode step,
so pulling the slow efficiency cores into the pool makes every barrier
wait on the slowest core, and leaves nothing for the OS / mic capture.

Measured on a Core Ultra 7 355 (8 cores, no SMT), base model, 6s clip:

  n_threads=6 -> RTF 0.14x
  n_threads=8 -> RTF 2.13x   (~15x slower, i.e. slower than real time)

Default to `max(1, min(cpu_count() - 2, 8))`, which leaves headroom and
picks 6 on that machine (optimal in a sweep) while staying <= the
previous value everywhere else. Users can still override via the
`whispercpp_n_threads` config setting.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Jatin K Malik <jatinkrmalik@gmail.com>
This commit is contained in:
only21mil
2026-07-13 14:44:49 -07:00
committed by GitHub
co-authored by Claude Opus 4.8 Jatin K Malik
parent b3605bf3c6
commit 6c8aae86c0
@@ -1266,7 +1266,16 @@ class SpeechRecognitionManager:
elif has_gpu_libs:
n_threads = max(1, multiprocessing.cpu_count() // 4)
else:
n_threads = min(multiprocessing.cpu_count(), 8)
# Leave a couple of cores for the OS and audio capture, and avoid
# oversubscribing hybrid CPUs. Using *every* core drags the slow
# efficiency cores into the pool; whisper.cpp synchronises all threads
# at each decode step, so the barrier stalls on the slowest core and
# throughput collapses. On an 8-core hybrid CPU (Intel Core Ultra,
# P+E cores) the old `min(cpu_count, 8)` picked 8 = every core and
# measured RTF jumped from ~0.14x (6 threads) to ~2.1x (8 threads) —
# a ~15x slowdown. Cap at 8; base/small models rarely scale past that.
# Users can still override with the `whispercpp_n_threads` setting.
n_threads = max(1, min(multiprocessing.cpu_count() - 2, 8))
load_start_time = time.time()
loaded_backend = backend