Commit Graph
2 Commits
Author SHA1 Message Date
soubarnak 987d8eec29 fix(models): use ticket 19 measured peaks for ladder thresholds
q8_0 threshold becomes 1644 MiB (1132+512) and q5_0 becomes 1358 MiB
(846+512) per the recorded baseline recommendation, replacing the
on-disk-size proxies. Tests pin the measured values and the 1644/1358
boundaries.
2026-11-06 09:00:47 +05:30
soubarnak df771d8fb1 feat(models): memory-aware whisper.cpp ladder with conditional backend order
Free memory picks the load-time rung (large-v3-turbo-q8_0 default,
q5_0 under 1086 MiB free, small/small.en floor) for unpinned default
configs; thresholds derive from catalog sizes plus the protocol's
512 MiB headroom. A manual pin is never overridden, and a failed load
steps one rung down and retries exactly once before surfacing.

CUDA ranks before Vulkan only with a complete architecture-matching
toolkit (nvidia-smi GPU, nvcc, compute capability >= 7.0); otherwise
Vulkan, then CPU. The driver-only CUDA recommendation now demotes to
CPU logic instead of naming an unreachable VRAM ladder.

Refs #20
2026-11-06 01:48:20 +05:30