The canonical sdist is built from the tag, but a second build of the same commit differs in tar and gzip timestamps, so its digest is recorded from the published asset, not from a rebuild. Correct the comments and docs that said "reproducible sdist". No behaviour change.
294 lines
15 KiB
Markdown
294 lines
15 KiB
Markdown
# Void packaging
|
|
|
|
Void Linux (glibc, x86_64) templates in void-packages layout, built with the
|
|
standard Void source tooling. Submission to upstream void-packages is out of
|
|
scope for spec #16; these templates serve the signed personal xbps
|
|
repository first.
|
|
|
|
## Layout
|
|
|
|
```
|
|
packaging/void/
|
|
├── README.md this file
|
|
├── build-release.sh the release path: gate, sdist, signed repo, install smoke
|
|
├── container-release.sh the container half: build, sign, install (one per phase)
|
|
├── render-voculinux.sh pins version, _tag and checksum in the build tree only
|
|
├── smoke.sh ticket #28 gate: the release path with a throwaway key
|
|
├── xbps-fingerprint.py the repository key's fingerprint and client plist
|
|
├── repository-key.plist the repository's PUBLIC key (never a private key)
|
|
├── tool_checksums.txt pinned base image and void-packages commit
|
|
└── srcpkgs/
|
|
├── python3-pywhispercpp/ ticket #26: whisper.cpp Python binding
|
|
├── python3-pynput/ ticket #25: the input library Void does not ship
|
|
├── python3-PyAudio/ ticket #25: the audio binding, updated past Void's copy
|
|
└── voculinux ticket #27: the application
|
|
```
|
|
|
|
`srcpkgs/` mirrors void-packages so the tree overlays cleanly:
|
|
`cp -r packaging/void/srcpkgs/. /path/to/void-packages/srcpkgs/`
|
|
(the release path does this inside the container).
|
|
|
|
## Templates
|
|
|
|
| Template | Ticket | Notes |
|
|
|---|---|---|
|
|
| `python3-pywhispercpp` | #26 | Vulkan backend on by default; see below |
|
|
| `python3-pynput`, `python3-PyAudio` | #25 | Dependencies Void does not ship at the needed version |
|
|
| `voculinux` | #27 | The app, from the canonical release sdist; see below |
|
|
|
|
## python3-pywhispercpp
|
|
|
|
Version `1.5.0`, pinned to the same version `install.sh`
|
|
(`PYWHISPERCPP_VERSION`) and `uv.lock` pin — never newest upstream.
|
|
The sdist checksum in the template must equal the `sha256` uv.lock records
|
|
for the sdist; `tests/test_void_pywhispercpp_template.py` fails otherwise.
|
|
|
|
- **Vulkan** is a build option, on by default. Rebuild CPU-only in a
|
|
void-packages checkout with the overlay applied: `./xbps-src -o ~vulkan pkg python3-pywhispercpp`.
|
|
There is no CUDA build: Void ships no CUDA toolkit package. Rebuilding
|
|
locally with a hand-installed toolkit (`GGML_CUDA=1`) is a documented,
|
|
unsupported escape hatch.
|
|
- **Portable CPU flags**: every ggml SIMD option is pinned off
|
|
(`-DGGML_NATIVE=OFF -DGGML_SSE42=OFF ...`), the plain x86-64 baseline
|
|
already used by the Flatpak (`#938`, `#957`). The install smoke scans the
|
|
shipped `libggml-cpu` (`.GCC.command.line`) for `-march=native`,
|
|
`-mcpu=native` and `-mtune=native` and fails on a match. The scan is a
|
|
tripwire, not a proof: the proof is the pinned configuration.
|
|
- **Shared libraries**: `BUILD_SHARED_LIBS=ON` with an `$ORIGIN` rpath
|
|
(as `install.sh` does), so the extension resolves the whisper.cpp
|
|
libraries installed beside it with no `LD_LIBRARY_PATH`.
|
|
- **Offline build**: the sdist vendors whisper.cpp, pybind11, and the
|
|
ggml sources; the build fetches nothing beyond the declared distfiles.
|
|
`PYWHISPERCPP_VERSION` is injected because the sdist carries no
|
|
`version.txt` (its version otherwise comes from git metadata).
|
|
|
|
## voculinux
|
|
|
|
The template builds the fork's canonical release source distribution. It is
|
|
built from the release tag, never attached by hand, and never fetched from
|
|
upstream. Read [Version mapping](#version-mapping) and [Checksum](#checksum)
|
|
before changing `version`, `_tag` or `checksum`.
|
|
|
|
- **Runtime deps** are Void's own interpreter, GTK and desktop bindings,
|
|
the packaged Python libraries, the three templates from tickets #25
|
|
and #26 (`python3-pynput`, `python3-PyAudio`, `python3-pywhispercpp`),
|
|
and the helper tools the app shells out to: PipeWire (capture), `wtype`
|
|
(direct typing on niri), `wl-clipboard`, `xdotool` and
|
|
`pulseaudio-utils`. `ibus` is not a dependency: it cannot inject on
|
|
niri. Every name was verified to exist in current void-packages; the
|
|
container build re-verifies via dependency resolution.
|
|
- **No payload beyond the app**: no speech models (they download on
|
|
first run), no vendored wheels, no pip at build or install time. The
|
|
install smoke fails the release if the payload carries model weights
|
|
or wheels.
|
|
- **Installed names**: the `voculinux.desktop` entry, the hicolor icons,
|
|
and a `usr/share/voculinux/packaged` marker the app reads to present
|
|
package-manager updates. The fork's sdist already carries the final
|
|
names, so the template installs them as they are; the payload smoke
|
|
rejects any pre-rename leftovers in the OS-visible paths.
|
|
- **INSTALL.msg** covers what xbps cannot express: the `input` group for
|
|
the evdev hotkey backend, first-run model downloads, optional engines
|
|
as manual additions (vosk, OpenAI whisper, Parakeet, faster-whisper,
|
|
onnxruntime), the niri first run, the absent `ydotool` rung, and
|
|
updating through xbps.
|
|
- **Update checker**: packaged installs keep background checks as a
|
|
signal, but every update affordance (tray entry, notification, About
|
|
page) presents the package-manager path
|
|
(`src/vocalinux/utils/packaging.py`).
|
|
|
|
### Version mapping
|
|
|
|
xbps versions cannot contain `-`. The xbps spelling of a release is
|
|
`__version__` with the hyphen removed. `build-release.sh names TAG` prints it.
|
|
|
|
| `__version__` (tag `v` + it) | xbps `version` | distfile and release asset |
|
|
|---|---|---|
|
|
| `0.18.5` | `0.18.5` | `voculinux-0.18.5.tar.gz` |
|
|
| `0.19.0-alpha` | `0.19.0alpha` | `voculinux-0.19.0alpha.tar.gz` |
|
|
| `0.19.0-beta` | `0.19.0beta` | `voculinux-0.19.0beta.tar.gz` |
|
|
| `0.19.0-rc1` | `0.19.0rc1` | `voculinux-0.19.0rc1.tar.gz` |
|
|
|
|
Only `X.Y.Z` and `X.Y.Z-alpha`, `-beta` or `-rcN` are release forms; anything
|
|
else is refused. xbps sorts these spellings below the final release
|
|
(`xbps-uhelper cmpver`, checked in `tests/test_void_release_build.py`), so a
|
|
pre-release never looks newer than the release it precedes. The sdist keeps its
|
|
PEP 440 name inside (`voculinux-0.19.0b0/`), which xbps-src's extract hook
|
|
renames to the package's `wrksrc`, so the two spellings need not match.
|
|
|
|
The distfile URL is `releases/download/${_tag}/voculinux-${version}.tar.gz`.
|
|
`_tag` keeps the hyphen (`v0.19.0-beta`), so the URL names the tag that exists.
|
|
|
|
## Checksum
|
|
|
|
The template's `checksum=` is a placeholder of 64 zeros in the committed
|
|
template until the first fork release has a recorded digest. That placeholder is
|
|
never a real digest. Every release build replaces it, in its **build tree only**,
|
|
with the sha256 of the sdist it builds (`render-voculinux.sh`). The release run
|
|
refuses to continue unless exactly one `checksum=` line exists and reads back as
|
|
the computed digest, and it refuses if the committed template changed.
|
|
|
|
After a release, the digest of the published sdist is recorded in the template
|
|
by a **pull request**: the workflow log and the release notes both print
|
|
`checksum=<sha256>`, and the pull request copies it into
|
|
`packaging/void/srcpkgs/voculinux/template`. Nothing commits to `main`.
|
|
|
|
## Building
|
|
|
|
```sh
|
|
just void-smoke # the gate: build, sign with a throwaway key, install, smoke
|
|
just void-release v0.18.5 # a dry run of the release, with XBPS_SIGNING_KEY_FILE set
|
|
packaging/void/build-release.sh names v0.19.0-beta # offline: the names a tag maps to
|
|
packaging/void/build-release.sh check-tag v0.19.0-beta # offline: the tag gate
|
|
```
|
|
|
|
Podman only, never Docker. The base image digest and the void-packages commit
|
|
come from `tool_checksums.txt`; bump them there, never in a script.
|
|
|
|
## Smoke gate
|
|
|
|
`packaging/void/smoke.sh` runs the release path for the current version with a
|
|
throwaway RSA 4096 key generated in a temporary directory. The gate needs no
|
|
host Void install and nothing is published. Stages, in order, each fails the run:
|
|
|
|
1. the tag gate (`check-tag`) and the committed template's version and `_tag`;
|
|
2. the canonical sdist, built from the working tree;
|
|
3. `build`: every template in `PKGS` order (`smoke.sh` owns the list), linted
|
|
with `xlint`, built with `xbps-src`, the sdist resolved by `xbps-src show`;
|
|
4. `sign`: the packages indexed and signed, with no network;
|
|
5. `install`: in a clean image with no key, an unsigned install is refused, the
|
|
signed repository is trusted, `voculinux` installs, `voculinux --version`
|
|
shows the version, the payload checks pass (desktop entry, icon, marker, no
|
|
models, no pre-rename names, the Vulkan backend, no host-native CPU tuning),
|
|
and the imports load (`pynput` through its dummy backend, `evdev`, `gi` with
|
|
Gtk 3, `pywhispercpp.model`, `pyaudio`, and the app's own modules, which keep their retained internal name).
|
|
|
|
`tests/test_void_smoke_gate.py` asserts offline that the gate stays wired to
|
|
every `srcpkgs` template.
|
|
|
|
## Signed repository
|
|
|
|
`packaging/void/build-release.sh release TAG --key-file FILE --out-dir DIR`
|
|
is the whole release, and the only path that signs anything. It is the path the
|
|
tag workflow runs, and `just void-release` runs it locally.
|
|
|
|
Fail-closed, in this order, before any container starts:
|
|
|
|
- the tag must be exactly `v` plus `__version__`, and the committed template must
|
|
name that version and tag;
|
|
- the key must be non-empty, valid, RSA 4096, and its public half must equal
|
|
`repository-key.plist` byte for byte (a different key never publishes);
|
|
- the canonical sdist must build with Python 3.11 and uv 0.12/0.13. Only its
|
|
wheel is compared byte for byte: a second build of the same tag has identical
|
|
contents, but the tar and gzip timestamps differ, so the release records the
|
|
digest of the sdist it publishes, not of a rebuild.
|
|
|
|
Then three containers, each with only what it needs:
|
|
|
|
| phase | privileges | what runs |
|
|
|---|---|---|
|
|
| build | root, network on, **no key** | void-packages at the pinned commit, the build tree rendered from the tag, every template built as the `builder` user |
|
|
| sign | root, **no network**, key mounted read-only | `xbps-rindex` indexes and signs; no build code runs |
|
|
| install | root, network on (dependencies), **no key** | an unsigned install is refused; the signed repository installs `voculinux`; the smoke above |
|
|
|
|
The committed templates must be unchanged after the build; the run fails if they
|
|
are not. `DIR` must be absent or empty, and the run writes `DIR/sdist/`,
|
|
`DIR/xbps/repo/` (packages, signatures, `x86_64-repodata`), `DIR/xbps/keys/`
|
|
(the client `.plist` named by its fingerprint, and `FINGERPRINT`) and
|
|
`DIR/SDIST_SHA256`. A final check accepts only that set: nothing from musl, no
|
|
other architecture, every package signed, the key the pinned one.
|
|
|
|
### The repository key
|
|
|
|
One RSA 4096 key signs everything. Its fingerprint, which clients pin:
|
|
|
|
```text
|
|
fa:d1:00:32:ce:b1:b9:fd:73:05:05:76:75:54:37:25
|
|
```
|
|
|
|
- **Public half**: committed as `packaging/void/repository-key.plist`
|
|
(the same file xbps keeps in `/var/db/xbps/keys/`, named by the
|
|
fingerprint). Signer identity in the file: `Bongbetic <soubarna@live.in>`.
|
|
`tests/test_void_signed_repo.py` derives the fingerprint from this file
|
|
and fails if either README shows a different one.
|
|
- **Private half**: never committed, never in the repository, never on a
|
|
build host's disk beyond the run. It exists as the `XBPS_SIGNING_KEY`
|
|
CI secret (PEM text, newlines intact) and in the maintainer's own backup.
|
|
- **Generating a new key** (only if the key is ever replaced; clients then
|
|
have to re-pin, so this is a release-notes event):
|
|
|
|
```sh
|
|
openssl genrsa -out xbps-sign-key.pem 4096
|
|
# back the key up somewhere safe, then store it as XBPS_SIGNING_KEY
|
|
openssl rsa -in xbps-sign-key.pem -pubout # the public half, to build the .plist from
|
|
```
|
|
|
|
- **Checking a private key against the committed public key**, before it is
|
|
stored as the secret (both lines must print the same digest):
|
|
|
|
```sh
|
|
openssl rsa -in xbps-sign-key.pem -pubout 2>/dev/null | openssl md5
|
|
python3 -c "import plistlib,sys; sys.stdout.buffer.write(plistlib.load(open('packaging/void/repository-key.plist','rb'))['public-key'])" | openssl md5
|
|
```
|
|
|
|
### Releasing: the Gitea workflow
|
|
|
|
A human pushes a version tag; `.gitea/workflows/xbps-release.yml` does the
|
|
rest. It never runs on a schedule or a manual dispatch.
|
|
|
|
1. **Tag gate.** `build-release.sh check-tag` (see above). Nothing else runs first.
|
|
2. **Key gate.** The CI secret `XBPS_SIGNING_KEY` must be non-empty. The key is
|
|
written with noclobber to a `0600` file on tmpfs (`/dev/shm`), never to disk.
|
|
3. **Runner check.** The tools in [Runner requirements](#runner-requirements)
|
|
must be present, or the job stops and names what is missing.
|
|
4. **Release.** `build-release.sh release`, with the key file (above).
|
|
5. **Publish.** The staged set goes to the tag's release and to the rolling
|
|
`latest` release, which the client registration URL serves. Assets:
|
|
`sdist/voculinux-<version>.tar.gz`, the packages, their `.sig2` signatures,
|
|
`x86_64-repodata` (last), `FINGERPRINT` and the `.plist`. Nothing is built or
|
|
uploaded by hand.
|
|
6. **Cleanup.** The key file is removed by a step that always runs.
|
|
|
|
Then the template's checksum is recorded by a pull request (above), and the
|
|
release notes show the sdist digest to copy from.
|
|
|
|
Secret setup, once: Gitea repository settings, Actions, Secrets, add
|
|
`XBPS_SIGNING_KEY` with the PEM private key. The job token must be allowed to
|
|
create releases and upload release assets.
|
|
|
|
### Runner requirements
|
|
|
|
The job runs on the `bongbetic-ci` label as a host job. It is not a `container:`
|
|
job, and it never uses Docker. It needs:
|
|
|
|
| requirement | why | check |
|
|
|---|---|---|
|
|
| `podman` (rootless with `subuid`/`subgid` ranges, or rootful) | the three phases | `podman run --rm alpine true` |
|
|
| unprivileged user namespaces | `xbps-src` builds in a user-namespace chroot (`uunshare`) | the build phase fails with a clear error otherwise |
|
|
| `/dev/shm` mounted as tmpfs | the key must never touch disk | `stat -f -c %T /dev/shm` prints `tmpfs` |
|
|
| `python3.11` on `PATH` | the sdist is built with Python 3.11, as CI does | `python3.11 --version` |
|
|
| `uv` 0.12.x or 0.13.x on `PATH` | the locked release-build tools | `uv --version` |
|
|
| `openssl`, `git`, `sha256sum`, `curl`, `python3` | key checks, checkout, digests, Gitea API | `command -v` |
|
|
| about 25 GiB free disk | the void-packages bootstrap and the whisper.cpp build | `df -h` |
|
|
| egress to `github.com`, `repo-default.voidlinux.org`, `ghcr.io`, `pypi.org`, `files.pythonhosted.org`, `git.bongbetic.com` | clone, bootstrap, base image, locked tools, distfiles, Gitea API | the build phase |
|
|
|
|
The job's `timeout-minutes` is 240. A cold build takes most of that: the
|
|
whisper.cpp Vulkan build is the long step.
|
|
|
|
## Client setup
|
|
|
|
```sh
|
|
# as root on the Void system
|
|
curl -fsSL -o '/var/db/xbps/keys/fa:d1:00:32:ce:b1:b9:fd:73:05:05:76:75:54:37:25.plist' \
|
|
https://git.bongbetic.com/xavierk/voculinux/raw/branch/main/packaging/void/repository-key.plist
|
|
echo 'repository=https://git.bongbetic.com/xavierk/voculinux/releases/download/latest' > /etc/xbps.d/voculinux.conf
|
|
xbps-install -S voculinux
|
|
```
|
|
|
|
The key file is named by its fingerprint, so xbps finds it on the first
|
|
sync. If xbps shows a fingerprint prompt anyway, answer `N` unless it
|
|
reads exactly the fingerprint above and matches the `FINGERPRINT` file on
|
|
the release; do not add `-y` to the first sync. `tests/test_void_signed_repo.py`
|
|
asserts offline that the registration text stays correct, and
|
|
`tests/test_void_release_build.py` asserts the release path refuses to
|
|
publish under any other key.
|