RTX 2050 CUDA model fit #3

New Issue

2026-11-05T15:23:03Z

xavierk commented

2026-11-05 15:23:03 +00:00

Part of #1

Question

Which whisper.cpp model and quantization should be the CUDA default on an RTX 2050 with 4 GB VRAM, and what backend order (CUDA, Vulkan, CPU) should Voculinux use on Nvidia hardware?

Part of #1 ## Question Which whisper.cpp model and quantization should be the CUDA default on an RTX 2050 with 4 GB VRAM, and what backend order (CUDA, Vulkan, CPU) should Voculinux use on Nvidia hardware?

xavierk added the wayfinder:research label 2026-11-05 15:23:03 +00:00

xavierk commented

2026-11-05 15:29:35 +00:00

RTX 2050 CUDA model fit — whisper.cpp model/quantization and backend order

Ticket: Gitea "RTX 2050 CUDA model fit", part of map "Voculinux port"
Question: Which whisper.cpp model and quantization should be the CUDA
default on an RTX 2050 4 GB, and what backend order (CUDA/Vulkan/CPU)
on Nvidia?
Scope: research only. No repo files were edited. Local state inspected in
src/vocalinux/utils/whispercpp_model_info.py, install.sh,
install.d/, and callers in src/vocalinux/ui/settings_dialog.py and
src/vocalinux/speech_recognition/recognition_manager.py.
Date: 2026-11-05. Primary sources are cited inline as [S1]…[S12];
full list at the end. Secondary sources used only where flagged.

Verdict

CUDA default model on RTX 2050 4 GB: large-v3-turbo-q8_0
(834 MiB weights), with large-v3-turbo-q5_0 (547 MiB) as the
automatic fallback when free VRAM at load time is low, and
small/small.en (465 MiB, current behaviour) as the final safe
fallback. Do not default to unquantized large-v3-turbo,
medium, or any large-v3 variant on a 4 GB card.
Backend order on Nvidia must be conditional, not a fixed flip:
- Nvidia GPU and a complete, arch-matching CUDA toolkit already
  installed → CUDA > Vulkan > CPU.
- Nvidia GPU without a usable CUDA toolkit (the common case,
  including Nouveau/NVK driver users) → Vulkan > CPU, which is
  what the installer effectively does today.
- A blind change to "CUDA first on Nvidia" would regress most Nvidia
  installs: the installer never installs the CUDA toolkit, so the
  CUDA build would be skipped or fail before the working Vulkan
  build is attempted (see §4).
Current code never reaches its own CUDA model recommendation on an
RTX 2050. detect_compute_backend() returns Vulkan first, so
get_recommended_model() takes the Vulkan branch (keyed to system
RAM, ignoring VRAM) and returns unquantized small. The CUDA/VRAM
branch is dead code on any Vulkan-capable Nvidia card (§2). Any fix
must key the model recommendation to the built backend plus
measured VRAM, not to the informational backend priority.

Confidence: high on the memory arithmetic and on the local-code
findings; medium on q8_0 vs unquantized-turbo speed on this exact
GPU (no primary benchmark exists for an RTX 2050 — validation plan in
§6). The q8_0 choice is made on guaranteed fit and headroom, not on
a claimed speed advantage.

1. Hardware envelope: what "RTX 2050 4 GB" actually means

The GeForce RTX 2050 is a laptop-only Ampere part (GA107, the same
die family as the RTX 3050 Laptop): 2,048 CUDA cores, 4 GB GDDR6 on
a 64-bit bus for 112 GB/s of memory bandwidth, 30–45 W TGP
[S1][S2]. Two facts dominate model fit: the 4 GB capacity and
the narrow 112 GB/s bus (one third of an RTX 2060's 336 GB/s
[S2]). Autoregressive decoding is memory-bandwidth-bound, so bytes
of weights read per generated token matter on this card.
NVIDIA's compute-capability table lists the GA107 consumer parts
(RTX 3050 / 3050 Ti) at CC 8.6; the RTX 2050 itself is not
listed on that page [S3]. Do not hard-code 8.6: read it from the
device. install.sh already does this correctly
(nvidia-smi --query-gpu=compute_cap, install.sh:889-910) and
refuses a toolkit that cannot target the detected arch
(install.sh:911-949). CC 8.6 implies Ampere tensor cores with
FP16 acceleration, which is what ggml's CUDA backend exploits via
cuBLAS and custom kernels [S4].
Usable VRAM is materially less than 4 GB. On a laptop the dGPU
may also drive the display (compositor framebuffers, browser GPU
processes) and every CUDA context carries fixed overhead. Plan
against free VRAM (nvidia-smi --query-gpu=memory.free), not
memory.total. Current code parses only total VRAM from the
backend-info string (whispercpp_model_info.py:544-559), which
overstates the budget on exactly the hybrid-graphics laptops that
ship this GPU.
NVIDIA lists the RTX 2050 as Vulkan-capable under its proprietary
driver [S5], so both backends are technically available on the
proprietary stack. On Nouveau/NVK there is no CUDA at all; Vulkan
is the only GPU path. Any Nvidia policy must keep that route.

# RTX 2050 CUDA model fit — whisper.cpp model/quantization and backend order - Ticket: Gitea "RTX 2050 CUDA model fit", part of map "Voculinux port" - Question: Which whisper.cpp model and quantization should be the CUDA default on an RTX 2050 4 GB, and what backend order (CUDA/Vulkan/CPU) on Nvidia? - Scope: research only. No repo files were edited. Local state inspected in `src/vocalinux/utils/whispercpp_model_info.py`, `install.sh`, `install.d/`, and callers in `src/vocalinux/ui/settings_dialog.py` and `src/vocalinux/speech_recognition/recognition_manager.py`. - Date: 2026-11-05. Primary sources are cited inline as [S1]…[S12]; full list at the end. Secondary sources used only where flagged. ## Verdict 1. **CUDA default model on RTX 2050 4 GB: `large-v3-turbo-q8_0`** (834 MiB weights), with `large-v3-turbo-q5_0` (547 MiB) as the automatic fallback when free VRAM at load time is low, and `small`/`small.en` (465 MiB, current behaviour) as the final safe fallback. Do not default to unquantized `large-v3-turbo`, `medium`, or any `large-v3` variant on a 4 GB card. 2. **Backend order on Nvidia must be conditional, not a fixed flip:** - Nvidia GPU **and** a complete, arch-matching CUDA toolkit already installed → **CUDA > Vulkan > CPU**. - Nvidia GPU **without** a usable CUDA toolkit (the common case, including Nouveau/NVK driver users) → **Vulkan > CPU**, which is what the installer effectively does today. - A blind change to "CUDA first on Nvidia" would regress most Nvidia installs: the installer never installs the CUDA toolkit, so the CUDA build would be skipped or fail before the working Vulkan build is attempted (see §4). 3. **Current code never reaches its own CUDA model recommendation on an RTX 2050.** `detect_compute_backend()` returns Vulkan first, so `get_recommended_model()` takes the Vulkan branch (keyed to system RAM, ignoring VRAM) and returns unquantized `small`. The CUDA/VRAM branch is dead code on any Vulkan-capable Nvidia card (§2). Any fix must key the model recommendation to the *built* backend plus measured VRAM, not to the informational backend priority. Confidence: high on the memory arithmetic and on the local-code findings; medium on `q8_0` vs unquantized-turbo *speed* on this exact GPU (no primary benchmark exists for an RTX 2050 — validation plan in §6). The `q8_0` choice is made on guaranteed fit and headroom, not on a claimed speed advantage. ## 1. Hardware envelope: what "RTX 2050 4 GB" actually means - The GeForce RTX 2050 is a laptop-only Ampere part (GA107, the same die family as the RTX 3050 Laptop): 2,048 CUDA cores, 4 GB GDDR6 on a 64-bit bus for 112 GB/s of memory bandwidth, 30–45 W TGP [S1][S2]. Two facts dominate model fit: the **4 GB capacity** and the **narrow 112 GB/s bus** (one third of an RTX 2060's 336 GB/s [S2]). Autoregressive decoding is memory-bandwidth-bound, so bytes of weights read per generated token matter on this card. - NVIDIA's compute-capability table lists the GA107 consumer parts (RTX 3050 / 3050 Ti) at **CC 8.6**; the RTX 2050 itself is not listed on that page [S3]. Do not hard-code 8.6: read it from the device. `install.sh` already does this correctly (`nvidia-smi --query-gpu=compute_cap`, `install.sh:889-910`) and refuses a toolkit that cannot target the detected arch (`install.sh:911-949`). CC 8.6 implies Ampere tensor cores with FP16 acceleration, which is what ggml's CUDA backend exploits via cuBLAS and custom kernels [S4]. - **Usable VRAM is materially less than 4 GB.** On a laptop the dGPU may also drive the display (compositor framebuffers, browser GPU processes) and every CUDA context carries fixed overhead. Plan against *free* VRAM (`nvidia-smi --query-gpu=memory.free`), not `memory.total`. Current code parses only total VRAM from the backend-info string (`whispercpp_model_info.py:544-559`), which overstates the budget on exactly the hybrid-graphics laptops that ship this GPU. - NVIDIA lists the RTX 2050 as Vulkan-capable under its proprietary driver [S5], so both backends are technically available on the proprietary stack. On Nouveau/NVK there is no CUDA at all; Vulkan is the only GPU path. Any Nvidia policy must keep that route.

xavierk closed this issue

2026-11-05 15:29:42 +00:00

xavierk referenced this issue

2026-11-05 15:36:13 +00:00

Voculinux port #1

xavierk referenced this issue

2026-11-05 16:21:40 +00:00

Model catalog for 4 GB VRAM #12

xavierk referenced this issue

2026-11-05 16:45:29 +00:00

Spec: Voculinux — Void Linux (glibc) Wayland/niri port of Vocalinux #16

xavierk referenced this issue from a commit