fix(whispercpp): don't default to all CPU cores on hybrid processors (#492)
The CPU-only auto thread count was `min(cpu_count(), 8)`. On an 8-core hybrid CPU (e.g. Intel Core Ultra, P+E cores) this selects 8 = every core. whisper.cpp synchronises all worker threads at each decode step, so pulling the slow efficiency cores into the pool makes every barrier wait on the slowest core, and leaves nothing for the OS / mic capture. Measured on a Core Ultra 7 355 (8 cores, no SMT), base model, 6s clip: n_threads=6 -> RTF 0.14x n_threads=8 -> RTF 2.13x (~15x slower, i.e. slower than real time) Default to `max(1, min(cpu_count() - 2, 8))`, which leaves headroom and picks 6 on that machine (optimal in a sweep) while staying <= the previous value everywhere else. Users can still override via the `whispercpp_n_threads` config setting. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Jatin K Malik <jatinkrmalik@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
Jatin K Malik
parent
b3605bf3c6
commit
6c8aae86c0
@@ -1266,7 +1266,16 @@ class SpeechRecognitionManager:
|
||||
elif has_gpu_libs:
|
||||
n_threads = max(1, multiprocessing.cpu_count() // 4)
|
||||
else:
|
||||
n_threads = min(multiprocessing.cpu_count(), 8)
|
||||
# Leave a couple of cores for the OS and audio capture, and avoid
|
||||
# oversubscribing hybrid CPUs. Using *every* core drags the slow
|
||||
# efficiency cores into the pool; whisper.cpp synchronises all threads
|
||||
# at each decode step, so the barrier stalls on the slowest core and
|
||||
# throughput collapses. On an 8-core hybrid CPU (Intel Core Ultra,
|
||||
# P+E cores) the old `min(cpu_count, 8)` picked 8 = every core and
|
||||
# measured RTF jumped from ~0.14x (6 threads) to ~2.1x (8 threads) —
|
||||
# a ~15x slowdown. Cap at 8; base/small models rarely scale past that.
|
||||
# Users can still override with the `whispercpp_n_threads` setting.
|
||||
n_threads = max(1, min(multiprocessing.cpu_count() - 2, 8))
|
||||
|
||||
load_start_time = time.time()
|
||||
loaded_backend = backend
|
||||
|
||||
Reference in New Issue
Block a user