diff --git a/docs/cuda-vocu.md b/docs/cuda-vocu.md index 2fd2b81..fc4ac74 100644 --- a/docs/cuda-vocu.md +++ b/docs/cuda-vocu.md @@ -1,6 +1,7 @@ # CUDA for Voculinux on Void Linux: integration plan -Status: plan, not started. Target release: 0.20.0. Owner decisions needed: section 9. +Status: plan, not started. Target release: 0.20.0. Owner decisions taken 2026-11-07 (section 9); +phase 0 is complete. This plan adds an optional CUDA backend for the whisper.cpp engine to the signed Void xbps repository, without changing what users who do not opt in receive. Each phase @@ -75,7 +76,14 @@ Rules that come out of the EULA and xbps: ## 4. Phases -### Phase 0: preflight (maintainer, no code) +### Phase 0: preflight (maintainer, no code) — done 2026-11-07 + +Result: Gitea 1.27.1 on xServer accepted a 419,430,400-byte release asset (HTTP 201) in a +private scratch repository, and the authenticated download matched its sha256. The +scratch repository was deleted. No server limit change is needed, and the Coolify service +`gitea` carries no upload-limit overrides. The owner accepted the EULA terms (section 9). + +Original steps, kept for re-checking after a Gitea upgrade: 1. Check the release upload limit on git.bongbetic.com: upload a 400 MiB file to a draft release in a scratch repository with `tea`. If refused, raise @@ -92,7 +100,8 @@ Work in a throwaway worktree, never on `main`. 1. In the pinned build image (`packaging/void/tool_checksums.txt`, `base-image`), unpack the redistributable archives from section 5, build pywhispercpp 1.5.0 with: `GGML_CUDA=1`, `-DGGML_BACKEND_DL=ON -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF`, - `-DCMAKE_CUDA_ARCHITECTURES="75-real;80-real;86-real;89-real;90-real;100-real;120-real;120-virtual"` + `-DCMAKE_CUDA_ARCHITECTURES="75-real;86-real;89-real;120-real;120-virtual"` + (consumer GPUs only, section 9) and fatbin compression, if ggml's CMake offers it for this nvcc (`GGML_CUDA_COMPRESSION_MODE=size`; confirm the option exists in the vendored ggml). 2. Record: build time, `libggml-cuda.so` size (raw and zstd), and whether nvcc accepts @@ -300,22 +309,34 @@ both templates in one pull request, rebuild both packages together, and re-run p |------|----------|--------| | `voculinux-cuda-libs` package | about 365 MiB compressed, 535 MiB installed | zstd of the 13.2 libraries | | `python3-pywhispercpp-cuda` package | 36 MiB per architecture before fatbin compression; total measured in phase 1 | local sm_86 build | -| Build time of the CUDA backend | measured in phase 1 | eight architectures with nvcc | +| Build time of the CUDA backend | measured in phase 1 | four architectures plus PTX with nvcc | | User download for `voculinux-cuda` | the two packages above | | -If the multi-architecture backend is too large, drop `90-real` and `100-real`, which are -data-center GPUs, keeping `120-virtual` so newer cards still JIT-compile from PTX. +Architecture set (consumer GPUs only, owner decision): + +| Target | GPUs | +|--------|------| +| `75-real` | Turing: GeForce GTX 16 series, RTX 20 series | +| `86-real` | Ampere consumer: RTX 30 series and RTX 2050 | +| `89-real` | Ada: RTX 40 series | +| `120-real` | Blackwell consumer: RTX 50 series | +| `120-virtual` | PTX, so a future consumer architecture JIT-compiles on first load | + +Left out on purpose: `80` (A100), `87` (Jetson Orin), `90` (H100), `100` and `103` +(data-center Blackwell). Those GPUs are not Void desktop targets. An Ampere data-center +card would JIT from no matching PTX and fail to load the backend, so the app falls back +to Vulkan there. ## 7. Risks | Risk | Mitigation | |------|------------| -| Release asset upload limit below the package size | phase 0 measures and raises it before any template work | +| Release asset upload limit below the package size | closed: phase 0 uploaded 400 MiB; `publish-release.sh` still gets a size preflight (phase 6) | | ggml backend ABI mismatch between the two pywhispercpp builds | exact `version_revision` dependency plus a template-equality test | | nvcc 13.2 rejects the image's GCC | phase 1 finds out; if so, use a GCC the image provides and nvcc supports, never `-allow-unsupported-compiler` | | Driver too old for CUDA 13.2 | `INSTALL.msg` states the minimum driver from NVIDIA's CUDA 13.2 release notes; selection falls back when `cudaGetDeviceCount` fails | | Vulkan and CUDA both register the same NVIDIA GPU | phase 1 measures order; phase 5 hides the unused one with environment variables | -| No speed gain on small GPUs | `auto` keeps CUDA first only where measured useful; the `vulkan` preference is one config key away; phase 7 records numbers | +| No speed gain on small GPUs | owner chose CUDA first regardless; the `vulkan` preference is one config key away; phase 7 records numbers | | Void's `nvidia-libs` upgrade changes the driver ABI | `libcuda.so.1` is the stable driver ABI; phase 7 re-run after major driver updates | ## 8. Ticket breakdown @@ -324,7 +345,7 @@ Dependencies in brackets. | # | Ticket | Type | |---|--------|------| -| 1 | Phase 0: release upload limit and EULA decision | human / infra | +| 1 | Phase 0: release upload limit and EULA decision (done 2026-11-07) | human / infra | | 2 | Phase 1: CUDA backend spike and measurements [1] | spike | | 3 | Phase 2: `voculinux-cuda-libs` template [2] | AFK | | 4 | Phase 3: `python3-pywhispercpp-cuda` template [2, 3] | AFK | @@ -333,12 +354,12 @@ Dependencies in brackets. | 7 | Phase 6: release pipeline [3, 4, 5] | AFK | | 8 | Phase 7: docs, 0.20.0 release and hardware acceptance [6, 7] | human-in-the-loop | -## 9. Decisions for the owner +## 9. Owner decisions (taken 2026-11-07) -1. Accept redistributing `libcudart`, `libcublas` and `libcublasLt` under the NVIDIA CUDA - EULA from this repository (required). -2. Raise the Gitea release upload limit if phase 0 needs it (infra change on the Gitea - host). -3. Default preference: `auto` (CUDA first when ready, as the plan says) or `vulkan` - first, given that Vulkan matched CUDA on the RTX 2050. -4. Architecture list: the eight in phase 1, or a smaller consumer-only set. +1. **EULA: accepted.** `libcudart`, `libcublas` and `libcublasLt` are redistributed from + this repository under the NVIDIA CUDA EULA, unmodified, with the license shipped. +2. **Gitea upload limit: may be raised.** Phase 0 showed no change is needed. +3. **Default preference: CUDA first.** `auto` selects CUDA whenever it is ready, then + Vulkan, then CPU, even though Vulkan matched CUDA on the RTX 2050. +4. **Architectures: consumer GPUs only.** `75-real;86-real;89-real;120-real;120-virtual` + (section 6).