fix(whisper): use CUDA device 0 when pywhispercpp is CUDA-backed (#636)

* fix(whisper): use CUDA device 0 when pywhispercpp is CUDA-backed

Vulkan GPU enumeration on hybrid laptops lists the iGPU as GPU0 and the
dGPU as GPU1, but CUDA ordinals always place the first NVIDIA GPU at 0.
Auto mode and the settings UI were passing Vulkan indices into CUDA
pywhispercpp, which silently fell back to CPU inference.

Detect the actual pywhispercpp backend before selecting gpu_device and
map CUDA installs to device 0. Vulkan-backed installs keep Vulkan indices.

* test(whisper): cover CUDA gpu_device selection branches for codecov

---------

Co-authored-by: Jatin K Malik <jatinkrmalik@gmail.com>
This commit is contained in:
Ariel Flesler
2026-08-05 21:53:23 -07:00
committed by GitHub
co-authored by Jatin K Malik
parent 7a5ddfde38
commit 5d06d21838
2 changed files with 215 additions and 8 deletions
@@ -1407,9 +1407,26 @@ class SpeechRecognitionManager:
f"whisper.cpp using {get_backend_display_name(backend)} backend: {backend_info}"
)
# Select Vulkan GPU via pywhispercpp context_params (GGML_VULKAN_DEVICE is ignored).
actual_gpu_backend = self._detect_pywhispercpp_gpu_backend()
# Select GPU device for pywhispercpp context_params.
selected_gpu_device = None
if backend == ComputeBackend.VULKAN:
if actual_gpu_backend == "cuda":
# pywhispercpp CUDA uses CUDA device ordinals (0 = first NVIDIA GPU).
# Vulkan enumeration on hybrid laptops lists iGPU as GPU0 and dGPU as
# GPU1; passing that Vulkan index to CUDA selects the CPU fallback.
vulkan_gpu_index = self.whispercpp_gpu_device
if vulkan_gpu_index is None or vulkan_gpu_index < 0:
vulkan_gpu_index = _prefer_discrete_vulkan_device()
if vulkan_gpu_index not in (None, 0):
logger.info(
"pywhispercpp is CUDA-backed; using CUDA device 0 "
f"(Vulkan GPU index {vulkan_gpu_index} does not map to CUDA ordinals)"
)
else:
logger.info("pywhispercpp is CUDA-backed; using CUDA device 0")
selected_gpu_device = 0
elif actual_gpu_backend == "vulkan":
gpu_device_index = self.whispercpp_gpu_device
if gpu_device_index is None or gpu_device_index < 0:
gpu_device_index = _prefer_discrete_vulkan_device()
@@ -1439,7 +1456,6 @@ class SpeechRecognitionManager:
logger.info(f"Loading whisper.cpp '{self.model_size}' model...")
self.model = None # Release previous model if re-initializing
actual_gpu_backend = self._detect_pywhispercpp_gpu_backend()
has_gpu_libs = actual_gpu_backend in ("vulkan", "cuda")
if self.whispercpp_n_threads is not None and self.whispercpp_n_threads > 0: