xavierk/voculinux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

docs(cuda): record owner decisions and phase 0 result #58

Merged
xavierk merged 1 commits from docs/cuda-decisions into main 2026-11-07 03:24:37 +00:00
Conversation 1 Commits 1 Files Changed 1
+38 -17
1 changed files with 38 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cuda-vocu.md
+38 -17
View File
@@ -1,6 +1,7 @@
# CUDA for Voculinux on Void Linux: integration plan
Status: plan, not started. Target release: 0.20.0. Owner decisions needed: section 9.
Status: plan, not started. Target release: 0.20.0. Owner decisions taken 2026-11-07 (section 9);
phase 0 is complete.
This plan adds an optional CUDA backend for the whisper.cpp engine to the signed Void
xbps repository, without changing what users who do not opt in receive. Each phase
@@ -75,7 +76,14 @@ Rules that come out of the EULA and xbps:
## 4. Phases
### Phase 0: preflight (maintainer, no code)
### Phase 0: preflight (maintainer, no code) — done 2026-11-07
Result: Gitea 1.27.1 on xServer accepted a 419,430,400-byte release asset (HTTP 201) in a
private scratch repository, and the authenticated download matched its sha256. The
scratch repository was deleted. No server limit change is needed, and the Coolify service
`gitea` carries no upload-limit overrides. The owner accepted the EULA terms (section 9).
Original steps, kept for re-checking after a Gitea upgrade:
1. Check the release upload limit on git.bongbetic.com: upload a 400 MiB file to a draft
release in a scratch repository with `tea`. If refused, raise
@@ -92,7 +100,8 @@ Work in a throwaway worktree, never on `main`.
1. In the pinned build image (`packaging/void/tool_checksums.txt`, `base-image`), unpack
the redistributable archives from section 5, build pywhispercpp 1.5.0 with:
`GGML_CUDA=1`, `-DGGML_BACKEND_DL=ON -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF`,
`-DCMAKE_CUDA_ARCHITECTURES="75-real;80-real;86-real;89-real;90-real;100-real;120-real;120-virtual"`
`-DCMAKE_CUDA_ARCHITECTURES="75-real;86-real;89-real;120-real;120-virtual"`
(consumer GPUs only, section 9)
and fatbin compression, if ggml's CMake offers it for this nvcc
(`GGML_CUDA_COMPRESSION_MODE=size`; confirm the option exists in the vendored ggml).
2. Record: build time, `libggml-cuda.so` size (raw and zstd), and whether nvcc accepts
@@ -300,22 +309,34 @@ both templates in one pull request, rebuild both packages together, and re-run p
|------|----------|--------|
| `voculinux-cuda-libs` package | about 365 MiB compressed, 535 MiB installed | zstd of the 13.2 libraries |
| `python3-pywhispercpp-cuda` package | 36 MiB per architecture before fatbin compression; total measured in phase 1 | local sm_86 build |
| Build time of the CUDA backend | measured in phase 1 | eight architectures with nvcc |
| Build time of the CUDA backend | measured in phase 1 | four architectures plus PTX with nvcc |
| User download for `voculinux-cuda` | the two packages above | |
If the multi-architecture backend is too large, drop `90-real` and `100-real`, which are
data-center GPUs, keeping `120-virtual` so newer cards still JIT-compile from PTX.
Architecture set (consumer GPUs only, owner decision):
| Target | GPUs |
|--------|------|
| `75-real` | Turing: GeForce GTX 16 series, RTX 20 series |
| `86-real` | Ampere consumer: RTX 30 series and RTX 2050 |
| `89-real` | Ada: RTX 40 series |
| `120-real` | Blackwell consumer: RTX 50 series |
| `120-virtual` | PTX, so a future consumer architecture JIT-compiles on first load |
Left out on purpose: `80` (A100), `87` (Jetson Orin), `90` (H100), `100` and `103`
(data-center Blackwell). Those GPUs are not Void desktop targets. An Ampere data-center
card would JIT from no matching PTX and fail to load the backend, so the app falls back
to Vulkan there.
## 7. Risks
| Risk | Mitigation |
|------|------------|
| Release asset upload limit below the package size | phase 0 measures and raises it before any template work |
| Release asset upload limit below the package size | closed: phase 0 uploaded 400 MiB; `publish-release.sh` still gets a size preflight (phase 6) |
| ggml backend ABI mismatch between the two pywhispercpp builds | exact `version_revision` dependency plus a template-equality test |
| nvcc 13.2 rejects the image's GCC | phase 1 finds out; if so, use a GCC the image provides and nvcc supports, never `-allow-unsupported-compiler` |
| Driver too old for CUDA 13.2 | `INSTALL.msg` states the minimum driver from NVIDIA's CUDA 13.2 release notes; selection falls back when `cudaGetDeviceCount` fails |
| Vulkan and CUDA both register the same NVIDIA GPU | phase 1 measures order; phase 5 hides the unused one with environment variables |
| No speed gain on small GPUs | `auto` keeps CUDA first only where measured useful; the `vulkan` preference is one config key away; phase 7 records numbers |
| No speed gain on small GPUs | owner chose CUDA first regardless; the `vulkan` preference is one config key away; phase 7 records numbers |
| Void's `nvidia-libs` upgrade changes the driver ABI | `libcuda.so.1` is the stable driver ABI; phase 7 re-run after major driver updates |
## 8. Ticket breakdown
@@ -324,7 +345,7 @@ Dependencies in brackets.
| # | Ticket | Type |
|---|--------|------|
| 1 | Phase 0: release upload limit and EULA decision | human / infra |
| 1 | Phase 0: release upload limit and EULA decision (done 2026-11-07) | human / infra |
| 2 | Phase 1: CUDA backend spike and measurements [1] | spike |
| 3 | Phase 2: `voculinux-cuda-libs` template [2] | AFK |
| 4 | Phase 3: `python3-pywhispercpp-cuda` template [2, 3] | AFK |
@@ -333,12 +354,12 @@ Dependencies in brackets.
| 7 | Phase 6: release pipeline [3, 4, 5] | AFK |
| 8 | Phase 7: docs, 0.20.0 release and hardware acceptance [6, 7] | human-in-the-loop |
## 9. Decisions for the owner
## 9. Owner decisions (taken 2026-11-07)
1. Accept redistributing `libcudart`, `libcublas` and `libcublasLt` under the NVIDIA CUDA
EULA from this repository (required).
2. Raise the Gitea release upload limit if phase 0 needs it (infra change on the Gitea
host).
3. Default preference: `auto` (CUDA first when ready, as the plan says) or `vulkan`
first, given that Vulkan matched CUDA on the RTX 2050.
4. Architecture list: the eight in phase 1, or a smaller consumer-only set.
1. **EULA: accepted.** `libcudart`, `libcublas` and `libcublasLt` are redistributed from
this repository under the NVIDIA CUDA EULA, unmodified, with the license shipped.
2. **Gitea upload limit: may be raised.** Phase 0 showed no change is needed.
3. **Default preference: CUDA first.** `auto` selects CUDA whenever it is ready, then
Vulkan, then CPU, even though Vulkan matched CUDA on the RTX 2050.
4. **Architectures: consumer GPUs only.** `75-real;86-real;89-real;120-real;120-virtual`
(section 6).