docs(cuda): record owner decisions and phase 0 result (#58)
CI / security (push) Successful in 1m11s
CI / lint (push) Successful in 22s
CI / ai-review (push) Skipped

This commit was merged in pull request #58.
This commit is contained in:
2026-11-07 03:24:37 +00:00
parent 594686d0f7
commit 61e8206c67
+38 -17
View File
@@ -1,6 +1,7 @@
# CUDA for Voculinux on Void Linux: integration plan # CUDA for Voculinux on Void Linux: integration plan
Status: plan, not started. Target release: 0.20.0. Owner decisions needed: section 9. Status: plan, not started. Target release: 0.20.0. Owner decisions taken 2026-11-07 (section 9);
phase 0 is complete.
This plan adds an optional CUDA backend for the whisper.cpp engine to the signed Void This plan adds an optional CUDA backend for the whisper.cpp engine to the signed Void
xbps repository, without changing what users who do not opt in receive. Each phase xbps repository, without changing what users who do not opt in receive. Each phase
@@ -75,7 +76,14 @@ Rules that come out of the EULA and xbps:
## 4. Phases ## 4. Phases
### Phase 0: preflight (maintainer, no code) ### Phase 0: preflight (maintainer, no code) — done 2026-11-07
Result: Gitea 1.27.1 on xServer accepted a 419,430,400-byte release asset (HTTP 201) in a
private scratch repository, and the authenticated download matched its sha256. The
scratch repository was deleted. No server limit change is needed, and the Coolify service
`gitea` carries no upload-limit overrides. The owner accepted the EULA terms (section 9).
Original steps, kept for re-checking after a Gitea upgrade:
1. Check the release upload limit on git.bongbetic.com: upload a 400 MiB file to a draft 1. Check the release upload limit on git.bongbetic.com: upload a 400 MiB file to a draft
release in a scratch repository with `tea`. If refused, raise release in a scratch repository with `tea`. If refused, raise
@@ -92,7 +100,8 @@ Work in a throwaway worktree, never on `main`.
1. In the pinned build image (`packaging/void/tool_checksums.txt`, `base-image`), unpack 1. In the pinned build image (`packaging/void/tool_checksums.txt`, `base-image`), unpack
the redistributable archives from section 5, build pywhispercpp 1.5.0 with: the redistributable archives from section 5, build pywhispercpp 1.5.0 with:
`GGML_CUDA=1`, `-DGGML_BACKEND_DL=ON -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF`, `GGML_CUDA=1`, `-DGGML_BACKEND_DL=ON -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF`,
`-DCMAKE_CUDA_ARCHITECTURES="75-real;80-real;86-real;89-real;90-real;100-real;120-real;120-virtual"` `-DCMAKE_CUDA_ARCHITECTURES="75-real;86-real;89-real;120-real;120-virtual"`
(consumer GPUs only, section 9)
and fatbin compression, if ggml's CMake offers it for this nvcc and fatbin compression, if ggml's CMake offers it for this nvcc
(`GGML_CUDA_COMPRESSION_MODE=size`; confirm the option exists in the vendored ggml). (`GGML_CUDA_COMPRESSION_MODE=size`; confirm the option exists in the vendored ggml).
2. Record: build time, `libggml-cuda.so` size (raw and zstd), and whether nvcc accepts 2. Record: build time, `libggml-cuda.so` size (raw and zstd), and whether nvcc accepts
@@ -300,22 +309,34 @@ both templates in one pull request, rebuild both packages together, and re-run p
|------|----------|--------| |------|----------|--------|
| `voculinux-cuda-libs` package | about 365 MiB compressed, 535 MiB installed | zstd of the 13.2 libraries | | `voculinux-cuda-libs` package | about 365 MiB compressed, 535 MiB installed | zstd of the 13.2 libraries |
| `python3-pywhispercpp-cuda` package | 36 MiB per architecture before fatbin compression; total measured in phase 1 | local sm_86 build | | `python3-pywhispercpp-cuda` package | 36 MiB per architecture before fatbin compression; total measured in phase 1 | local sm_86 build |
| Build time of the CUDA backend | measured in phase 1 | eight architectures with nvcc | | Build time of the CUDA backend | measured in phase 1 | four architectures plus PTX with nvcc |
| User download for `voculinux-cuda` | the two packages above | | | User download for `voculinux-cuda` | the two packages above | |
If the multi-architecture backend is too large, drop `90-real` and `100-real`, which are Architecture set (consumer GPUs only, owner decision):
data-center GPUs, keeping `120-virtual` so newer cards still JIT-compile from PTX.
| Target | GPUs |
|--------|------|
| `75-real` | Turing: GeForce GTX 16 series, RTX 20 series |
| `86-real` | Ampere consumer: RTX 30 series and RTX 2050 |
| `89-real` | Ada: RTX 40 series |
| `120-real` | Blackwell consumer: RTX 50 series |
| `120-virtual` | PTX, so a future consumer architecture JIT-compiles on first load |
Left out on purpose: `80` (A100), `87` (Jetson Orin), `90` (H100), `100` and `103`
(data-center Blackwell). Those GPUs are not Void desktop targets. An Ampere data-center
card would JIT from no matching PTX and fail to load the backend, so the app falls back
to Vulkan there.
## 7. Risks ## 7. Risks
| Risk | Mitigation | | Risk | Mitigation |
|------|------------| |------|------------|
| Release asset upload limit below the package size | phase 0 measures and raises it before any template work | | Release asset upload limit below the package size | closed: phase 0 uploaded 400 MiB; `publish-release.sh` still gets a size preflight (phase 6) |
| ggml backend ABI mismatch between the two pywhispercpp builds | exact `version_revision` dependency plus a template-equality test | | ggml backend ABI mismatch between the two pywhispercpp builds | exact `version_revision` dependency plus a template-equality test |
| nvcc 13.2 rejects the image's GCC | phase 1 finds out; if so, use a GCC the image provides and nvcc supports, never `-allow-unsupported-compiler` | | nvcc 13.2 rejects the image's GCC | phase 1 finds out; if so, use a GCC the image provides and nvcc supports, never `-allow-unsupported-compiler` |
| Driver too old for CUDA 13.2 | `INSTALL.msg` states the minimum driver from NVIDIA's CUDA 13.2 release notes; selection falls back when `cudaGetDeviceCount` fails | | Driver too old for CUDA 13.2 | `INSTALL.msg` states the minimum driver from NVIDIA's CUDA 13.2 release notes; selection falls back when `cudaGetDeviceCount` fails |
| Vulkan and CUDA both register the same NVIDIA GPU | phase 1 measures order; phase 5 hides the unused one with environment variables | | Vulkan and CUDA both register the same NVIDIA GPU | phase 1 measures order; phase 5 hides the unused one with environment variables |
| No speed gain on small GPUs | `auto` keeps CUDA first only where measured useful; the `vulkan` preference is one config key away; phase 7 records numbers | | No speed gain on small GPUs | owner chose CUDA first regardless; the `vulkan` preference is one config key away; phase 7 records numbers |
| Void's `nvidia-libs` upgrade changes the driver ABI | `libcuda.so.1` is the stable driver ABI; phase 7 re-run after major driver updates | | Void's `nvidia-libs` upgrade changes the driver ABI | `libcuda.so.1` is the stable driver ABI; phase 7 re-run after major driver updates |
## 8. Ticket breakdown ## 8. Ticket breakdown
@@ -324,7 +345,7 @@ Dependencies in brackets.
| # | Ticket | Type | | # | Ticket | Type |
|---|--------|------| |---|--------|------|
| 1 | Phase 0: release upload limit and EULA decision | human / infra | | 1 | Phase 0: release upload limit and EULA decision (done 2026-11-07) | human / infra |
| 2 | Phase 1: CUDA backend spike and measurements [1] | spike | | 2 | Phase 1: CUDA backend spike and measurements [1] | spike |
| 3 | Phase 2: `voculinux-cuda-libs` template [2] | AFK | | 3 | Phase 2: `voculinux-cuda-libs` template [2] | AFK |
| 4 | Phase 3: `python3-pywhispercpp-cuda` template [2, 3] | AFK | | 4 | Phase 3: `python3-pywhispercpp-cuda` template [2, 3] | AFK |
@@ -333,12 +354,12 @@ Dependencies in brackets.
| 7 | Phase 6: release pipeline [3, 4, 5] | AFK | | 7 | Phase 6: release pipeline [3, 4, 5] | AFK |
| 8 | Phase 7: docs, 0.20.0 release and hardware acceptance [6, 7] | human-in-the-loop | | 8 | Phase 7: docs, 0.20.0 release and hardware acceptance [6, 7] | human-in-the-loop |
## 9. Decisions for the owner ## 9. Owner decisions (taken 2026-11-07)
1. Accept redistributing `libcudart`, `libcublas` and `libcublasLt` under the NVIDIA CUDA 1. **EULA: accepted.** `libcudart`, `libcublas` and `libcublasLt` are redistributed from
EULA from this repository (required). this repository under the NVIDIA CUDA EULA, unmodified, with the license shipped.
2. Raise the Gitea release upload limit if phase 0 needs it (infra change on the Gitea 2. **Gitea upload limit: may be raised.** Phase 0 showed no change is needed.
host). 3. **Default preference: CUDA first.** `auto` selects CUDA whenever it is ready, then
3. Default preference: `auto` (CUDA first when ready, as the plan says) or `vulkan` Vulkan, then CPU, even though Vulkan matched CUDA on the RTX 2050.
first, given that Vulkan matched CUDA on the RTX 2050. 4. **Architectures: consumer GPUs only.** `75-real;86-real;89-real;120-real;120-virtual`
4. Architecture list: the eight in phase 1, or a smaller consumer-only set. (section 6).