docs(cuda): record owner decisions and phase 0 result (#58)
This commit was merged in pull request #58.
This commit is contained in:
+38
-17
@@ -1,6 +1,7 @@
|
|||||||
# CUDA for Voculinux on Void Linux: integration plan
|
# CUDA for Voculinux on Void Linux: integration plan
|
||||||
|
|
||||||
Status: plan, not started. Target release: 0.20.0. Owner decisions needed: section 9.
|
Status: plan, not started. Target release: 0.20.0. Owner decisions taken 2026-11-07 (section 9);
|
||||||
|
phase 0 is complete.
|
||||||
|
|
||||||
This plan adds an optional CUDA backend for the whisper.cpp engine to the signed Void
|
This plan adds an optional CUDA backend for the whisper.cpp engine to the signed Void
|
||||||
xbps repository, without changing what users who do not opt in receive. Each phase
|
xbps repository, without changing what users who do not opt in receive. Each phase
|
||||||
@@ -75,7 +76,14 @@ Rules that come out of the EULA and xbps:
|
|||||||
|
|
||||||
## 4. Phases
|
## 4. Phases
|
||||||
|
|
||||||
### Phase 0: preflight (maintainer, no code)
|
### Phase 0: preflight (maintainer, no code) — done 2026-11-07
|
||||||
|
|
||||||
|
Result: Gitea 1.27.1 on xServer accepted a 419,430,400-byte release asset (HTTP 201) in a
|
||||||
|
private scratch repository, and the authenticated download matched its sha256. The
|
||||||
|
scratch repository was deleted. No server limit change is needed, and the Coolify service
|
||||||
|
`gitea` carries no upload-limit overrides. The owner accepted the EULA terms (section 9).
|
||||||
|
|
||||||
|
Original steps, kept for re-checking after a Gitea upgrade:
|
||||||
|
|
||||||
1. Check the release upload limit on git.bongbetic.com: upload a 400 MiB file to a draft
|
1. Check the release upload limit on git.bongbetic.com: upload a 400 MiB file to a draft
|
||||||
release in a scratch repository with `tea`. If refused, raise
|
release in a scratch repository with `tea`. If refused, raise
|
||||||
@@ -92,7 +100,8 @@ Work in a throwaway worktree, never on `main`.
|
|||||||
1. In the pinned build image (`packaging/void/tool_checksums.txt`, `base-image`), unpack
|
1. In the pinned build image (`packaging/void/tool_checksums.txt`, `base-image`), unpack
|
||||||
the redistributable archives from section 5, build pywhispercpp 1.5.0 with:
|
the redistributable archives from section 5, build pywhispercpp 1.5.0 with:
|
||||||
`GGML_CUDA=1`, `-DGGML_BACKEND_DL=ON -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF`,
|
`GGML_CUDA=1`, `-DGGML_BACKEND_DL=ON -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF`,
|
||||||
`-DCMAKE_CUDA_ARCHITECTURES="75-real;80-real;86-real;89-real;90-real;100-real;120-real;120-virtual"`
|
`-DCMAKE_CUDA_ARCHITECTURES="75-real;86-real;89-real;120-real;120-virtual"`
|
||||||
|
(consumer GPUs only, section 9)
|
||||||
and fatbin compression, if ggml's CMake offers it for this nvcc
|
and fatbin compression, if ggml's CMake offers it for this nvcc
|
||||||
(`GGML_CUDA_COMPRESSION_MODE=size`; confirm the option exists in the vendored ggml).
|
(`GGML_CUDA_COMPRESSION_MODE=size`; confirm the option exists in the vendored ggml).
|
||||||
2. Record: build time, `libggml-cuda.so` size (raw and zstd), and whether nvcc accepts
|
2. Record: build time, `libggml-cuda.so` size (raw and zstd), and whether nvcc accepts
|
||||||
@@ -300,22 +309,34 @@ both templates in one pull request, rebuild both packages together, and re-run p
|
|||||||
|------|----------|--------|
|
|------|----------|--------|
|
||||||
| `voculinux-cuda-libs` package | about 365 MiB compressed, 535 MiB installed | zstd of the 13.2 libraries |
|
| `voculinux-cuda-libs` package | about 365 MiB compressed, 535 MiB installed | zstd of the 13.2 libraries |
|
||||||
| `python3-pywhispercpp-cuda` package | 36 MiB per architecture before fatbin compression; total measured in phase 1 | local sm_86 build |
|
| `python3-pywhispercpp-cuda` package | 36 MiB per architecture before fatbin compression; total measured in phase 1 | local sm_86 build |
|
||||||
| Build time of the CUDA backend | measured in phase 1 | eight architectures with nvcc |
|
| Build time of the CUDA backend | measured in phase 1 | four architectures plus PTX with nvcc |
|
||||||
| User download for `voculinux-cuda` | the two packages above | |
|
| User download for `voculinux-cuda` | the two packages above | |
|
||||||
|
|
||||||
If the multi-architecture backend is too large, drop `90-real` and `100-real`, which are
|
Architecture set (consumer GPUs only, owner decision):
|
||||||
data-center GPUs, keeping `120-virtual` so newer cards still JIT-compile from PTX.
|
|
||||||
|
| Target | GPUs |
|
||||||
|
|--------|------|
|
||||||
|
| `75-real` | Turing: GeForce GTX 16 series, RTX 20 series |
|
||||||
|
| `86-real` | Ampere consumer: RTX 30 series and RTX 2050 |
|
||||||
|
| `89-real` | Ada: RTX 40 series |
|
||||||
|
| `120-real` | Blackwell consumer: RTX 50 series |
|
||||||
|
| `120-virtual` | PTX, so a future consumer architecture JIT-compiles on first load |
|
||||||
|
|
||||||
|
Left out on purpose: `80` (A100), `87` (Jetson Orin), `90` (H100), `100` and `103`
|
||||||
|
(data-center Blackwell). Those GPUs are not Void desktop targets. An Ampere data-center
|
||||||
|
card would JIT from no matching PTX and fail to load the backend, so the app falls back
|
||||||
|
to Vulkan there.
|
||||||
|
|
||||||
## 7. Risks
|
## 7. Risks
|
||||||
|
|
||||||
| Risk | Mitigation |
|
| Risk | Mitigation |
|
||||||
|------|------------|
|
|------|------------|
|
||||||
| Release asset upload limit below the package size | phase 0 measures and raises it before any template work |
|
| Release asset upload limit below the package size | closed: phase 0 uploaded 400 MiB; `publish-release.sh` still gets a size preflight (phase 6) |
|
||||||
| ggml backend ABI mismatch between the two pywhispercpp builds | exact `version_revision` dependency plus a template-equality test |
|
| ggml backend ABI mismatch between the two pywhispercpp builds | exact `version_revision` dependency plus a template-equality test |
|
||||||
| nvcc 13.2 rejects the image's GCC | phase 1 finds out; if so, use a GCC the image provides and nvcc supports, never `-allow-unsupported-compiler` |
|
| nvcc 13.2 rejects the image's GCC | phase 1 finds out; if so, use a GCC the image provides and nvcc supports, never `-allow-unsupported-compiler` |
|
||||||
| Driver too old for CUDA 13.2 | `INSTALL.msg` states the minimum driver from NVIDIA's CUDA 13.2 release notes; selection falls back when `cudaGetDeviceCount` fails |
|
| Driver too old for CUDA 13.2 | `INSTALL.msg` states the minimum driver from NVIDIA's CUDA 13.2 release notes; selection falls back when `cudaGetDeviceCount` fails |
|
||||||
| Vulkan and CUDA both register the same NVIDIA GPU | phase 1 measures order; phase 5 hides the unused one with environment variables |
|
| Vulkan and CUDA both register the same NVIDIA GPU | phase 1 measures order; phase 5 hides the unused one with environment variables |
|
||||||
| No speed gain on small GPUs | `auto` keeps CUDA first only where measured useful; the `vulkan` preference is one config key away; phase 7 records numbers |
|
| No speed gain on small GPUs | owner chose CUDA first regardless; the `vulkan` preference is one config key away; phase 7 records numbers |
|
||||||
| Void's `nvidia-libs` upgrade changes the driver ABI | `libcuda.so.1` is the stable driver ABI; phase 7 re-run after major driver updates |
|
| Void's `nvidia-libs` upgrade changes the driver ABI | `libcuda.so.1` is the stable driver ABI; phase 7 re-run after major driver updates |
|
||||||
|
|
||||||
## 8. Ticket breakdown
|
## 8. Ticket breakdown
|
||||||
@@ -324,7 +345,7 @@ Dependencies in brackets.
|
|||||||
|
|
||||||
| # | Ticket | Type |
|
| # | Ticket | Type |
|
||||||
|---|--------|------|
|
|---|--------|------|
|
||||||
| 1 | Phase 0: release upload limit and EULA decision | human / infra |
|
| 1 | Phase 0: release upload limit and EULA decision (done 2026-11-07) | human / infra |
|
||||||
| 2 | Phase 1: CUDA backend spike and measurements [1] | spike |
|
| 2 | Phase 1: CUDA backend spike and measurements [1] | spike |
|
||||||
| 3 | Phase 2: `voculinux-cuda-libs` template [2] | AFK |
|
| 3 | Phase 2: `voculinux-cuda-libs` template [2] | AFK |
|
||||||
| 4 | Phase 3: `python3-pywhispercpp-cuda` template [2, 3] | AFK |
|
| 4 | Phase 3: `python3-pywhispercpp-cuda` template [2, 3] | AFK |
|
||||||
@@ -333,12 +354,12 @@ Dependencies in brackets.
|
|||||||
| 7 | Phase 6: release pipeline [3, 4, 5] | AFK |
|
| 7 | Phase 6: release pipeline [3, 4, 5] | AFK |
|
||||||
| 8 | Phase 7: docs, 0.20.0 release and hardware acceptance [6, 7] | human-in-the-loop |
|
| 8 | Phase 7: docs, 0.20.0 release and hardware acceptance [6, 7] | human-in-the-loop |
|
||||||
|
|
||||||
## 9. Decisions for the owner
|
## 9. Owner decisions (taken 2026-11-07)
|
||||||
|
|
||||||
1. Accept redistributing `libcudart`, `libcublas` and `libcublasLt` under the NVIDIA CUDA
|
1. **EULA: accepted.** `libcudart`, `libcublas` and `libcublasLt` are redistributed from
|
||||||
EULA from this repository (required).
|
this repository under the NVIDIA CUDA EULA, unmodified, with the license shipped.
|
||||||
2. Raise the Gitea release upload limit if phase 0 needs it (infra change on the Gitea
|
2. **Gitea upload limit: may be raised.** Phase 0 showed no change is needed.
|
||||||
host).
|
3. **Default preference: CUDA first.** `auto` selects CUDA whenever it is ready, then
|
||||||
3. Default preference: `auto` (CUDA first when ready, as the plan says) or `vulkan`
|
Vulkan, then CPU, even though Vulkan matched CUDA on the RTX 2050.
|
||||||
first, given that Vulkan matched CUDA on the RTX 2050.
|
4. **Architectures: consumer GPUs only.** `75-real;86-real;89-real;120-real;120-virtual`
|
||||||
4. Architecture list: the eight in phase 1, or a smaller consumer-only set.
|
(section 6).
|
||||||
|
|||||||
Reference in New Issue
Block a user