Free-memory model ladder, conditional backend order, step-down retry #20

New Issue

2026-11-05T17:00:36Z

xavierk commented

2026-11-05 17:00:36 +00:00

Parent

Spec: Voculinux — Void Linux (glibc) Wayland/niri port of Vocalinux

What to build

Model selection becomes memory-aware end to end. On a 4 GB-class card the recommended model is chosen at load time from measured free memory (free VRAM on a discrete GPU, free system RAM otherwise): large-v3-turbo-q8_0 by default, large-v3-turbo-q5_0 when free memory is low, small or small.en (by language) as the floor, using the thresholds calibrated by the baseline ticket. Backend order becomes conditional: CUDA first only when a complete architecture-matching toolkit is already installed, otherwise Vulkan then CPU — fixing today's unreachable VRAM-aware recommendation. When a model load fails, the app steps one rung down and retries once before surfacing the error. Manual model selections are never overridden.

Acceptance criteria

With ample free memory the default rung is selected; with constrained free memory the next rung down is selected; thresholds match the baseline ticket's recorded recommendation.
With a complete matching CUDA toolkit present, CUDA is preferred; without one, selection goes straight to Vulkan and never attempts a CUDA build or prompts for a toolkit.
A forced model-load failure steps down exactly one rung and retries once; a second failure surfaces the engine error and does not loop.
English-only configuration pairs with the English-only floor variant; other languages pair with the multilingual variant.
A manually selected model loads as selected on every path, including models outside the recommended set.
Behaviour tests cover rung choice from mocked free-memory readings, both backend-order branches, and the single step-down retry, without real GPU hardware.

Blocked by

Measure the installed-build performance baseline

## Parent [Spec: Voculinux — Void Linux (glibc) Wayland/niri port of Vocalinux](https://git.bongbetic.com/xavierk/voculinux/issues/16) ## What to build Model selection becomes memory-aware end to end. On a 4 GB-class card the recommended model is chosen at load time from measured free memory (free VRAM on a discrete GPU, free system RAM otherwise): large-v3-turbo-q8_0 by default, large-v3-turbo-q5_0 when free memory is low, small or small.en (by language) as the floor, using the thresholds calibrated by the baseline ticket. Backend order becomes conditional: CUDA first only when a complete architecture-matching toolkit is already installed, otherwise Vulkan then CPU — fixing today's unreachable VRAM-aware recommendation. When a model load fails, the app steps one rung down and retries once before surfacing the error. Manual model selections are never overridden. ## Acceptance criteria - [ ] With ample free memory the default rung is selected; with constrained free memory the next rung down is selected; thresholds match the baseline ticket's recorded recommendation. - [ ] With a complete matching CUDA toolkit present, CUDA is preferred; without one, selection goes straight to Vulkan and never attempts a CUDA build or prompts for a toolkit. - [ ] A forced model-load failure steps down exactly one rung and retries once; a second failure surfaces the engine error and does not loop. - [ ] English-only configuration pairs with the English-only floor variant; other languages pair with the multilingual variant. - [ ] A manually selected model loads as selected on every path, including models outside the recommended set. - [ ] Behaviour tests cover rung choice from mocked free-memory readings, both backend-order branches, and the single step-down retry, without real GPU hardware. ## Blocked by - [Measure the installed-build performance baseline](https://git.bongbetic.com/xavierk/voculinux/issues/19)

xavierk added the ready-for-agent label 2026-11-05 17:00:36 +00:00

xavierk referenced this issue

2026-11-05 17:00:53 +00:00

Hardware acceptance run on Void, niri, Dank Material Shell, RTX 2050 #32

xavierk referenced this issue

2026-11-05 17:41:17 +00:00

Measure the installed-build performance baseline #19

xavierk commented

2026-11-06 04:15:48 +00:00

Merged in integration/spec16 1dd3986383 pushed to fork/integration/spec16 (https://github.com/soubarnak/vocalinux/tree/integration/spec16), review PASS, gates flake8 E9,F63,F7,F82 + black --check + isort --check + mypy src/ clean, pytest 37 passed, npm jest 11 suites/92 tests + lint 0 errors + tsc clean + build OK

Merged in integration/spec16 1dd398638312a3b6e27793d856662e60df73f252 pushed to fork/integration/spec16 (https://github.com/soubarnak/vocalinux/tree/integration/spec16), review PASS, gates flake8 E9,F63,F7,F82 + black --check + isort --check + mypy src/ clean, pytest 37 passed, npm jest 11 suites/92 tests + lint 0 errors + tsc clean + build OK

xavierk closed this issue

2026-11-06 04:15:48 +00:00

xavierk referenced this issue from a commit

2026-11-06 11:06:39 +00:00

Add Codecov integration for test results and coverage reports (#20)

xavierk referenced this issue from a commit

2026-11-06 11:06:41 +00:00

feat(perf): lock installed-build baseline protocol and take target-machine numbers

xavierk referenced this issue from a commit

2026-11-06 11:06:41 +00:00

feat(models): memory-aware whisper.cpp ladder with conditional backend order

xavierk referenced this issue from a commit