chore(release): prepare v0.19.0 (#53)
CI / security (push) Successful in 1m11s
CI / lint (push) Successful in 21s
CI / ai-review (push) Skipped

Version 0.19.0 across version.py, the Void template and the synced docs; changelog entry; Gitea project URLs in pyproject.toml; per-backend ladder thresholds measured in #19; the xbps release workflow is manual-only, and packaging/void/publish-release.sh publishes a locally built, verified repository to the Gitea releases.
This commit was merged in pull request #53.
This commit is contained in:
2026-11-06 20:23:14 +00:00
parent 85cbc37842
commit 731b919865
25 changed files with 1408 additions and 189 deletions
+27 -19
View File
@@ -1,22 +1,32 @@
name: xbps-release name: xbps-release
# Signed Void xbps repository release (tickets #28 and #29). A human pushes a # Signed Void xbps repository release (tickets #28 and #29). Manual dispatch only.
# version tag; this job does everything else through packaging/void/
# build-release.sh, the same script a local dry run and the smoke gate call:
# #
# 1. the tag must EXACTLY equal "v" + __version__ (src/vocalinux/version.py), # A tag push does NOT start this workflow. Releases are currently built and
# and the committed Void template must already name that version; # published from the maintainer's own x86_64 Void host, not from CI: the only
# 2. the XBPS_SIGNING_KEY secret must be non-empty and RSA 4096, and its public # runner with the bongbetic-ci label is ARM64 Docker on another server, and it
# half must be packaging/void/repository-key.plist; # cannot build the glibc x86_64 release. The local sequence is in
# 3. the canonical sdist is built from the tag (Python 3.11, # docs/RELEASE_PROCESS.md (Step 7 and the Void section of Step 8) and
# uv 0.12/0.13, scripts/build_reproducible_wheel.py); # packaging/void/README.md ("Releasing"):
# 4. every template builds in the pinned Void glibc image with podman (no # 1. create and push the annotated tag on the verified origin/main commit;
# signing key in that phase), then the repository is signed in a second # 2. packaging/void/build-release.sh release TAG --key-file FILE --out-dir DIR,
# container with no network, then voculinux is installed and smoked in a # in a clean checkout of the tag;
# clean image with no key; # 3. packaging/void/publish-release.sh TAG --out-dir DIR, after --dry-run.
# 5. the sdist, the signed repository and the client key are published to the #
# tag's release and to the rolling `latest` release that the client # This job is kept for a future x86_64 host-mode runner with rootless podman. It
# registration URL serves (README.md). # runs the same release path as the local run, and its publish step makes the
# same API calls as publish-release.sh. A dispatch's ref is a branch, so the tag
# gate below refuses it before anything is built or signed.
#
# The job's steps: the tag must EXACTLY equal "v" + __version__
# (src/vocalinux/version.py), and the committed Void template must already name
# that version; the XBPS_SIGNING_KEY secret must be non-empty and RSA 4096, and
# its public half must be packaging/void/repository-key.plist; the canonical sdist
# is built from the tag; every template builds in the pinned Void glibc image with
# podman, then the repository is signed in a second container with no network,
# then voculinux is installed and smoked in a clean image with no key; and the
# staged set is published to the tag's release and to the rolling `latest`
# release that the client registration URL serves (README.md).
# #
# Nothing here is built or uploaded by hand, nothing commits or pushes to any # Nothing here is built or uploaded by hand, nothing commits or pushes to any
# branch, and no template is bumped: the post-release template bump is a normal # branch, and no template is bumped: the post-release template bump is a normal
@@ -25,9 +35,7 @@ name: xbps-release
# only, never Docker. Runner requirements: packaging/void/README.md. # only, never Docker. Runner requirements: packaging/void/README.md.
on: on:
push: workflow_dispatch:
tags:
- "v*"
permissions: permissions:
contents: write contents: write
+17
View File
@@ -12,6 +12,23 @@ Release history for Vocalinux.
## Current stable ## Current stable
**[v0.19.0](https://git.bongbetic.com/xavierk/voculinux/releases/tag/v0.19.0)** (2026-11-07)
The first Voculinux release: a hard fork of 0.18.5 for Void Linux (glibc, x86_64), published as a signed xbps repository from the fork's Gitea host. Changes since v0.18.5, from the commit subjects and PR titles on `main`:
- **Void Linux packaging:** signed xbps repository and release pipeline, a package smoke gate across all templates, the `voculinux` template built from the release tag (#37), and carried templates for `python3-pywhispercpp` 1.5.0 with CPU variants loaded at runtime (#49), `python3-pynput`, `python3-PyAudio` and `python3-pysocks`, with the audio and SOCKS version floors constrained (#39). The Void package is the only published install channel, and upstream install references are gone (#38, #40).
- **Identity:** user-visible names are Voculinux, with the identity centralised in one module (tickets #17, #18). An existing configuration and custom dictionary are imported once on first run, and the migrated dictionary points at the fork (#36).
- **Model default:** a memory-aware whisper.cpp ladder with a conditional backend order (CUDA only with a complete toolkit, then Vulkan, then CPU). Free-memory thresholds are per backend, from the installed-build measurement in #19. A load failure steps down one rung and retries once (ticket #20).
- **Dictation on niri:** a niri keybind pack and Dank Material Shell notes (ticket #22), a niri-aware inject probe and conflict-free example keybinds (#47). The injection fallback ladder is portal, direct typing, virtual-input tool, XWayland, then clipboard (ticket #31).
- **Audio:** system audio is captured from PipeWire sink monitors. A configured microphone is never swapped, and the microphone is never opened for system audio (#48).
- **Interface and scripting:** a dictation state signal and property on D-Bus (ticket #21), a brand accent hook in the GTK colour scheme (ticket #23), and the installed-build performance baseline protocol (ticket #19).
- **Website:** rebranded to Voculinux (ticket #24), with Google Analytics removed and site links pointing at the fork (#35).
- **Fixes:** CUDA detection outside `PATH`, an IBus warning on niri, update checks against the fork's Gitea repository (#36), and the review findings for the Void port (#34).
Carried from upstream after 0.18.5 (VocaHQ/vocalinux PR numbers): overlay Wayland detection (#1030, #1035), quitting when the dock closes the overlay window (#1028, #1036), the selected microphone's full name in the device tooltip (#1029, #1037), escaped device names in the audio status label (#1043), Parakeet Underdog Bark 3.0 models (#1031), the snap build from locked inputs (#1041), the GitHub Actions dependency bump (#1042), and the website's 404 page and SEO work (#1038, #1039, #1040).
See [docs/UPDATE.md](docs/UPDATE.md#whats-new-in-v0190) for the highlight table.
**[v0.18.5](https://github.com/VocaHQ/vocalinux/releases/tag/v0.18.5)** (2026-10-27) **[v0.18.5](https://github.com/VocaHQ/vocalinux/releases/tag/v0.18.5)** (2026-10-27)
Patch on the stable line: a stop that races a recording start can no longer leave the microphone recording with no way to halt it, and a stalled microphone read can no longer hang Stop forever, keyboards survive suspend and the GNOME lock screen again because the evdev grab is released when forwarding cannot run, Dictation Pad and Settings no longer bind to other windows or travel workspaces together on Wayland, Ctrl+mouse-wheel zoom works again, the installer gains an upgrade mode that keeps your existing configuration, paste-injection copies are marked sensitive on wl-clipboard 2.2+ so clipboard managers that honor the hint keep them out of history, and AUR source publication is gated on attested release sdists. Patch on the stable line: a stop that races a recording start can no longer leave the microphone recording with no way to halt it, and a stalled microphone read can no longer hang Stop forever, keyboards survive suspend and the GNOME lock screen again because the evdev grab is released when forwarding cannot run, Dictation Pad and Settings no longer bind to other windows or travel workspaces together on Wayland, Ctrl+mouse-wheel zoom works again, the installer gains an upgrade mode that keeps your existing configuration, paste-injection copies are marked sensitive on wl-clipboard 2.2+ so clipboard managers that honor the hint keep them out of history, and AUR source publication is gated on attested release sdists.
+2 -2
View File
@@ -11,7 +11,7 @@
[![License: AGPL-3.0](https://img.shields.io/badge/License-AGPL--3.0-blue.svg)](https://www.gnu.org/licenses/agpl-3.0) [![License: AGPL-3.0](https://img.shields.io/badge/License-AGPL--3.0-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)
[![Privacy: on-device](https://img.shields.io/badge/privacy-on--device-success)](#privacy-and-security) [![Privacy: on-device](https://img.shields.io/badge/privacy-on--device-success)](#privacy-and-security)
[![X11 & Wayland](https://img.shields.io/badge/display-X11%20%7C%20Wayland-lightgrey)](#features) [![X11 & Wayland](https://img.shields.io/badge/display-X11%20%7C%20Wayland-lightgrey)](#features)
[![Release](https://img.shields.io/badge/release-v0.18.5-478061)](https://git.bongbetic.com/xavierk/voculinux/releases/tag/v0.18.5) [![Release](https://img.shields.io/badge/release-v0.19.0-478061)](https://git.bongbetic.com/xavierk/voculinux/releases/tag/v0.19.0)
[Website](https://voculinux.com) · [Install](#install) · [Docs](#documentation) · [Releases](https://git.bongbetic.com/xavierk/voculinux/releases) · [Issues](https://git.bongbetic.com/xavierk/voculinux/issues) [Website](https://voculinux.com) · [Install](#install) · [Docs](#documentation) · [Releases](https://git.bongbetic.com/xavierk/voculinux/releases) · [Issues](https://git.bongbetic.com/xavierk/voculinux/issues)
@@ -21,7 +21,7 @@ Voculinux turns speech into typed text in whatever app has focus. It is a free,
No account is required. Models download once. After that, speech-to-text stays on your machine. No account is required. Models download once. After that, speech-to-text stays on your machine.
**Current release:** [v0.18.5](https://git.bongbetic.com/xavierk/voculinux/releases/tag/v0.18.5). Patch: a stop that races a recording start can no longer leave the mic recording forever, keyboards stay alive through suspend and the GNOME lock screen, Ctrl+mouse-wheel zoom works again, Dictation Pad and Settings stop travelling workspaces together, and paste-injection copies are marked sensitive for clipboard history on wl-clipboard 2.2+. Details: [docs/UPDATE.md](docs/UPDATE.md). **Current release:** [v0.19.0](https://git.bongbetic.com/xavierk/voculinux/releases/tag/v0.19.0). The first Voculinux release: a Void Linux package from the [signed xbps repository](#void-linux-signed-xbps-repo), niri and Dank Material Shell support, a model default chosen by free GPU memory, and a fix that keeps system-audio capture from ever opening the microphone. Details: [docs/UPDATE.md](docs/UPDATE.md).
## Features ## Features
+2 -1
View File
@@ -6,7 +6,8 @@ Only the current stable minor line receives security fixes:
| Version | Supported | | Version | Supported |
| ------- | --------- | | ------- | --------- |
| 0.18.x | Yes | | 0.19.x | Yes |
| 0.18.x | No |
| 0.17.x | No | | 0.17.x | No |
| 0.16.x | No | | 0.16.x | No |
| 0.15.x | No | | 0.15.x | No |
+52 -45
View File
@@ -39,23 +39,25 @@ publish succeeds.
### Void xbps repository (required for a Void release) ### Void xbps repository (required for a Void release)
The signed Void repository is released by `.gitea/workflows/xbps-release.yml` The signed Void repository is built and published from the maintainer's own
on the fork's Gitea host, not by `release.yml`. The whole release is one script, x86_64 Void host, not from CI. The whole build is one script,
`packaging/void/build-release.sh`, which the workflow and a local dry run both `packaging/void/build-release.sh`, and the publish is one more,
call. Set it up once: `packaging/void/publish-release.sh`. The Gitea workflow
`.gitea/workflows/xbps-release.yml` is manual dispatch only: a tag push does not
start it, because its only runner (label `bongbetic-ci`) is ARM64 and cannot build
the glibc x86_64 release. Set this up once on the release host:
| what | value | | what | value |
|---|---| |---|---|
| Secret | `XBPS_SIGNING_KEY`: the PEM-encoded RSA 4096 repository private key (Gitea repository settings, Actions, Secrets) | | Signing key | the PEM-encoded RSA 4096 repository private key, kept in the maintainer's own backup. It is passed to `build-release.sh` as a file on tmpfs (`/dev/shm`) and removed after the run; it is never committed |
| Public key | `packaging/void/repository-key.plist`, committed, fingerprint `fa:d1:00:32:ce:b1:b9:fd:73:05:05:76:75:54:37:25` | | Public key | `packaging/void/repository-key.plist`, committed, fingerprint `fa:d1:00:32:ce:b1:b9:fd:73:05:05:76:75:54:37:25` |
| Runner | `bongbetic-ci`, a host job with podman (rootless, with user namespaces), python3.11, uv 0.12/0.13 and tmpfs `/dev/shm`. Requirements: `packaging/void/README.md`, "Runner requirements". Never Docker | | Host | x86_64 Void with rootless podman (user namespaces), python3.11, uv 0.12/0.13 and tmpfs `/dev/shm`. Requirements: `packaging/void/README.md`, "Runner requirements". Never Docker |
| Token | the job token must be allowed to create releases and upload release assets | | Gitea login | a `tea` login for `https://git.bongbetic.com` whose token may create releases and upload assets, or `GITEA_TOKEN` in the environment for `publish-release.sh` |
Without the secret the workflow fails and publishes nothing; it never signs Without a valid key the build stops before any container starts. It never signs
with a different key either: the release path refuses any key whose public half with a different key: the release path refuses any key whose public half is not
is not `repository-key.plist`. The private key is never committed; only the `repository-key.plist`. Only the public `.plist` is committed. Key generation and the
public `.plist` is. Key generation and the check that a private key matches the check that a private key matches the committed public key are in `packaging/void/README.md`.
committed public key are in `packaging/void/README.md`.
The carried Void templates in `packaging/void/srcpkgs/` track the project's own The carried Void templates in `packaging/void/srcpkgs/` track the project's own
pins. `python3-PyAudio` and `python3-pysocks` replace stock Void packages below pins. `python3-PyAudio` and `python3-pysocks` replace stock Void packages below
@@ -109,7 +111,7 @@ Install `just` from https://just.systems or the distro package `just`.
- [ ] Commit all version changes - [ ] Commit all version changes
- [ ] Push branch and create PR - [ ] Push branch and create PR
- [ ] After PR merge, create tag: `git tag -a vX.Y.Z-PHASE -m "Release X.Y.Z-PHASE"` - [ ] After PR merge, create tag: `git tag -a vX.Y.Z-PHASE -m "Release X.Y.Z-PHASE"`
- [ ] Push tag: `git push origin vX.Y.Z-PHASE` (this also starts the Gitea xbps workflow, see Step 8) - [ ] Create and push the annotated tag on the verified `origin/main` commit: `git push origin refs/tags/vX.Y.Z-PHASE`, never `--tags`. Pushing a tag starts no workflow; the Void release is built and published by hand (Step 8)
### After the release run finishes ### After the release run finishes
- [ ] Run `just verify-release` - the published release verifies as published - [ ] Run `just verify-release` - the published release verifies as published
@@ -351,8 +353,8 @@ It has two parts, handled at two times, and both go through a normal pull reques
release, so a stale template stops the tag, before any build. release, so a stale template stops the tag, before any build.
- **Checksum: after the tag, in a pull request.** The release build replaces - **Checksum: after the tag, in a pull request.** The release build replaces
`checksum=` in its build tree only, with the sha256 of the sdist it builds, and `checksum=` in its build tree only, with the sha256 of the sdist it builds, and
prints `checksum=<sha256>` in the workflow log and in the release notes prints `checksum=<sha256>`, and the published release notes carry it as
(`sdist sha256:`). Copy that value into the template in a pull request through `sdist sha256:`. Copy that value into the template in a pull request through
the normal test, lint and type gates. Never commit it directly to `main`, and the normal test, lint and type gates. Never commit it directly to `main`, and
never hand-edit a release asset. never hand-edit a release asset.
@@ -515,10 +517,15 @@ Key changes since 0.4.1-alpha:
Full changelog: https://github.com/VocaHQ/vocalinux/releases" Full changelog: https://github.com/VocaHQ/vocalinux/releases"
# Push tag # Push the one tag by its ref, never with --tags
git push origin v0.5.0-beta git push origin refs/tags/v0.5.0-beta
``` ```
**Void release (this fork):** tag the verified `origin/main` commit by its full SHA, then push
that one ref: `git tag -a v0.19.0 <sha> -m "Voculinux 0.19.0"` and
`git push origin refs/tags/v0.19.0`. Never `git push --tags`: this clone holds upstream tags
that must not reach the fork's Gitea host. Then follow the Void section of Step 8.
### Step 8: Monitor Release ### Step 8: Monitor Release
After pushing the tag, the GitHub Actions workflow will automatically: After pushing the tag, the GitHub Actions workflow will automatically:
@@ -551,36 +558,35 @@ After pushing the tag, the GitHub Actions workflow will automatically:
Monitor at: https://github.com/VocaHQ/vocalinux/actions Monitor at: https://github.com/VocaHQ/vocalinux/actions
#### Void xbps repository (Gitea tag workflow) #### Void xbps repository (local release)
Pushing the tag to `origin` (the fork's Gitea repository) also starts Pushing the tag starts nothing. The Void release is run by hand on the release host,
`.gitea/workflows/xbps-release.yml`. Pushing a tag is a human act and is from a clean checkout of the tag, in this order. Replace `vX.Y.Z` with the release.
approved each time: the workflow has no schedule and no manual trigger.
1. **Tag gate.** `build-release.sh check-tag`: the tag must be exactly `v` plus 1. **Merge the release pull request** to `main`, then confirm which commit `origin/main` is:
`__version__`, and the committed template must name that release. Any other `git fetch origin && git rev-parse origin/main`.
tag exits 1 and publishes nothing. 2. **Create and push the annotated tag** on that commit (Step 7).
2. **Key gate.** The `XBPS_SIGNING_KEY` secret must be non-empty. It is written 3. **Check out the tag cleanly:** `git worktree add /tmp/voculinux-vX.Y.Z vX.Y.Z`. A worktree
with noclobber to a `0600` file on tmpfs and removed by a final step that at a tag creates no tag. Work only in that worktree.
always runs. 4. **Build:** from that checkout, run
3. **Runner check.** podman, python3.11, uv 0.12/0.13, openssl, git and curl `packaging/void/build-release.sh release vX.Y.Z --key-file /dev/shm/xbps-sign-key.pem --out-dir /path/to/out`.
must be on the runner, or the job names what is missing and stops. The out-dir must be absent or empty. The run refuses an uncommitted checkout and a key
4. **Release.** `build-release.sh release` builds the canonical sdist from the that is not the committed one, both before any container starts. It writes `sdist/`,
tag, builds every template in the pinned Void glibc container `xbps/`, `SDIST_SHA256` and `BUILT_FROM` (the tag and the commit it was built from), and
without the key, signs the repository in a second container with no network, prints `checksum=<sha256>`. Remove the key file afterwards.
and installs `voculinux` from that repository in a clean image without the key. 5. **Dry-run the publish:** `packaging/void/publish-release.sh vX.Y.Z --out-dir /path/to/out --dry-run`.
A local dry run is `just void-release vX.Y.Z` with `XBPS_SIGNING_KEY_FILE` It runs every check (the tag is on `origin` at the built commit, the digest, the key)
set: the same script, the same containers, nothing published. and prints the API calls it would make. It sends no request and reads no token.
5. **Publish.** The staged set (sdist, packages, signatures, `x86_64-repodata`, 6. **Publish:** `packaging/void/publish-release.sh vX.Y.Z --out-dir /path/to/out`. It attaches the
`FINGERPRINT`, the client `.plist`) attaches to the tag's release on Gitea and sdist, packages, signatures, `FINGERPRINT` and the client `.plist` to the tag's release, then
replaces the assets of the rolling `latest` release, which the client replaces the assets of the rolling `latest` release that the client registration URL serves.
registration URL The repodata is uploaded last. The token comes from `GITEA_TOKEN`, or from the `tea` login
(`https://git.bongbetic.com/xavierk/voculinux/releases/download/latest`) for `https://git.bongbetic.com`.
serves. Push tags in version order: pushing an older tag refreshes `latest` 7. **Record the checksum** in a post-release pull request (section 3.5).
with the older repository.
Nothing in this run is built or uploaded by hand, and the workflow never commits Neither script commits or pushes to any branch or tag, and nothing is built or uploaded by
or pushes to any branch. The post-release checksum pull request is section 3.5. hand. Publish tags in version order: publishing an older tag refreshes `latest` with the
older repository.
### Step 9: Post-Release Tasks ### Step 9: Post-Release Tasks
@@ -601,7 +607,7 @@ or pushes to any branch. The post-release checksum pull request is section 3.5.
- [ ] Verify PyPI package was published (if applicable), and that its wheel sha256 - [ ] Verify PyPI package was published (if applicable), and that its wheel sha256
matches the line for that wheel in `SHA256SUMS` matches the line for that wheel in `SHA256SUMS`
- [ ] Verify website was deployed (check voculinux.com) - [ ] Verify website was deployed (check voculinux.com)
- [ ] Verify the Gitea xbps run succeeded, and that both the tag release and - [ ] Verify the Void publish succeeded, and that both the tag release and
`latest` carry the full signed set (`sdist/voculinux-*.tar.gz`, `latest` carry the full signed set (`sdist/voculinux-*.tar.gz`,
`*.x86_64.xbps`, `*.sig2`, `x86_64-repodata`, `FINGERPRINT`, the `.plist`), `*.x86_64.xbps`, `*.sig2`, `x86_64-repodata`, `FINGERPRINT`, the `.plist`),
that the fingerprint matches the one in `README.md`, and that the sdist that the fingerprint matches the one in `README.md`, and that the sdist
@@ -636,6 +642,7 @@ git push origin v0.5.1-beta
| Version | Date | Phase | Notes | | Version | Date | Phase | Notes |
|---------|------|-------|-------| |---------|------|-------|-------|
| 0.19.0 | 2026-11-07 | Stable | First Voculinux release: a hard fork for Void Linux (glibc, x86_64) with a signed xbps repository, niri and Dank Material Shell support, a per-backend memory-aware model default, and PipeWire system audio that never falls back to the microphone |
| 0.2.0-alpha | 2024 | Alpha | Initial alpha | | 0.2.0-alpha | 2024 | Alpha | Initial alpha |
| 0.3.0-alpha | 2024 | Alpha | - | | 0.3.0-alpha | 2024 | Alpha | - |
| 0.4.0-alpha | 2024 | Alpha | Multi-language support | | 0.4.0-alpha | 2024 | Alpha | Multi-language support |
+24 -1
View File
@@ -25,7 +25,7 @@ installer as below.
```bash ```bash
cd voculinux cd voculinux
git fetch origin git fetch origin
git checkout v0.18.5 git checkout v0.19.0
./install.sh ./install.sh
``` ```
@@ -90,6 +90,29 @@ Missing system packages: see [INSTALL.md](INSTALL.md) or [DISTRO_COMPATIBILITY.m
--- ---
## What's New in v0.19.0
0.19.0 is the first Voculinux release: a hard fork of the 0.18.5 line for Void Linux
(glibc, x86_64), installed and updated through the signed xbps repository. The default
engine is still whisper.cpp, and the speech models still download on first run against
pinned checksums.
### 0.19.0 highlights
| Area | Change |
|------|--------|
| Packaging | Signed Void xbps repository with the `voculinux` package. Carried templates for pywhispercpp 1.5.0 (with CPU variants), pynput, PyAudio and PySocks. Install and update with `xbps-install`. |
| Names and first run | User-visible names are Voculinux. An existing configuration and custom dictionary are copied once on first run; the Voculinux configuration always wins after that. |
| Model default | The whisper.cpp model is chosen at load time from free memory: VRAM on a GPU build, system RAM otherwise. Thresholds are per backend, measured on a 4 GB RTX 2050. A load failure steps down one size and retries once. |
| niri and Dank Material Shell | A niri keybind pack for dictation toggles. On niri, typing goes straight to the cursor with no permission prompt. The tray is a standard StatusNotifier item. |
| Audio | System-audio sources come from PipeWire. A saved system-audio source that disappears is an error, never a silent switch to the microphone. |
| Scripting | The D-Bus interface reports dictation state (idle, listening, processing) for bars and scripts. |
| Fixes | CUDA detection outside `PATH`, the IBus warning on niri, and update checks against this fork's Gitea repository. |
Full change list: [CHANGELOG.md](../CHANGELOG.md).
---
## What's New in v0.18.5 ## What's New in v0.18.5
0.18.5 is a **patch** on the stable line. Default engine is still whisper.cpp. It closes the reports filed after 0.18.4: a stop pressed while a recording is starting can no longer leave the microphone recording forever, microphones that stall mid-read can no longer wedge dictation, keyboards survive suspend and the GNOME lock screen, Ctrl+mouse-wheel zoom works again, and the installer learns an upgrade mode that keeps your settings. 0.18.5 is a **patch** on the stable line. Default engine is still whisper.cpp. It closes the reports filed after 0.18.4: a stop pressed while a recording is starting can no longer leave the microphone recording forever, microphones that stall mid-read can no longer wedge dictation, keyboards survive suspend and the GNOME lock screen, Ctrl+mouse-wheel zoom works again, and the installer learns an upgrade mode that keeps your settings.
+6 -6
View File
@@ -134,12 +134,12 @@ native-smoke distro="debian:12":
void-smoke: void-smoke:
bash packaging/void/smoke.sh bash packaging/void/smoke.sh
# Dry-run or perform the signed Void release for TAG (vX.Y.Z[-phase]) exactly # Build the signed Void release for TAG (vX.Y.Z[-phase]) exactly as the release
# as the Gitea tag workflow does: packaging/void/build-release.sh release. # host does: packaging/void/build-release.sh release, from a clean checkout of
# Stages the complete set under publish/ and publishes nothing. Needs # the tag. Stages the complete set under publish/ and publishes nothing; publish
# XBPS_SIGNING_KEY_FILE (an RSA 4096 private key whose public half is # it with packaging/void/publish-release.sh. Needs XBPS_SIGNING_KEY_FILE (an RSA
# packaging/void/repository-key.plist) and the release tools in # 4096 private key whose public half is packaging/void/repository-key.plist) and
# packaging/void/README.md. Podman only, never Docker. # the release tools in packaging/void/README.md. Podman only, never Docker.
void-release TAG: void-release TAG:
bash packaging/void/build-release.sh release "{{TAG}}" \ bash packaging/void/build-release.sh release "{{TAG}}" \
--key-file "${XBPS_SIGNING_KEY_FILE:?set XBPS_SIGNING_KEY_FILE to the RSA 4096 key}" \ --key-file "${XBPS_SIGNING_KEY_FILE:?set XBPS_SIGNING_KEY_FILE to the RSA 4096 key}" \
+55 -35
View File
@@ -10,7 +10,8 @@ repository first.
``` ```
packaging/void/ packaging/void/
├── README.md this file ├── README.md this file
├── build-release.sh the release path: gate, sdist, signed repo, install smoke ├── build-release.sh the release path: gate, clean checkout, sdist, signed repo, install smoke
├── publish-release.sh publishes a built out-dir to the Gitea release API (dry run first)
├── container-release.sh the container half: build, sign, install (one per phase) ├── container-release.sh the container half: build, sign, install (one per phase)
├── render-voculinux.sh pins version, _tag and checksum in the build tree only ├── render-voculinux.sh pins version, _tag and checksum in the build tree only
├── smoke.sh ticket #28 gate: the release path with a throwaway key ├── smoke.sh ticket #28 gate: the release path with a throwaway key
@@ -161,7 +162,7 @@ xbps versions cannot contain `-`. The xbps spelling of a release is
| `__version__` (tag `v` + it) | xbps `version` | distfile and release asset | | `__version__` (tag `v` + it) | xbps `version` | distfile and release asset |
|---|---|---| |---|---|---|
| `0.18.5` | `0.18.5` | `voculinux-0.18.5.tar.gz` | | `0.19.0` | `0.19.0` | `voculinux-0.19.0.tar.gz` |
| `0.19.0-alpha` | `0.19.0alpha` | `voculinux-0.19.0alpha.tar.gz` | | `0.19.0-alpha` | `0.19.0alpha` | `voculinux-0.19.0alpha.tar.gz` |
| `0.19.0-beta` | `0.19.0beta` | `voculinux-0.19.0beta.tar.gz` | | `0.19.0-beta` | `0.19.0beta` | `voculinux-0.19.0beta.tar.gz` |
| `0.19.0-rc1` | `0.19.0rc1` | `voculinux-0.19.0rc1.tar.gz` | | `0.19.0-rc1` | `0.19.0rc1` | `voculinux-0.19.0rc1.tar.gz` |
@@ -186,7 +187,7 @@ refuses to continue unless exactly one `checksum=` line exists and reads back as
the computed digest, and it refuses if the committed template changed. the computed digest, and it refuses if the committed template changed.
After a release, the digest of the published sdist is recorded in the template After a release, the digest of the published sdist is recorded in the template
by a **pull request**: the workflow log and the release notes both print by a **pull request**: the build output and the published release notes both carry
`checksum=<sha256>`, and the pull request copies it into `checksum=<sha256>`, and the pull request copies it into
`packaging/void/srcpkgs/voculinux/template`. Nothing commits to `main`. `packaging/void/srcpkgs/voculinux/template`. Nothing commits to `main`.
@@ -194,9 +195,10 @@ by a **pull request**: the workflow log and the release notes both print
```sh ```sh
just void-smoke # the gate: build, sign with a throwaway key, install, smoke just void-smoke # the gate: build, sign with a throwaway key, install, smoke
just void-release v0.18.5 # a dry run of the release, with XBPS_SIGNING_KEY_FILE set just void-release v0.19.0 # build the release set, with XBPS_SIGNING_KEY_FILE set
packaging/void/build-release.sh names v0.19.0-beta # offline: the names a tag maps to packaging/void/build-release.sh names v0.19.0-beta # offline: the names a tag maps to
packaging/void/build-release.sh check-tag v0.19.0-beta # offline: the tag gate packaging/void/build-release.sh check-tag v0.19.0-beta # offline: the tag gate
packaging/void/publish-release.sh v0.19.0 --out-dir DIR --dry-run # every check, the planned calls, no request
``` ```
Podman only, never Docker. The base image digest and the void-packages commit Podman only, never Docker. The base image digest and the void-packages commit
@@ -230,13 +232,17 @@ every `srcpkgs` template.
## Signed repository ## Signed repository
`packaging/void/build-release.sh release TAG --key-file FILE --out-dir DIR` `packaging/void/build-release.sh release TAG --key-file FILE --out-dir DIR`
is the whole release, and the only path that signs anything. It is the path the is the whole release, and the only path that signs anything. The maintainer runs
tag workflow runs, and `just void-release` runs it locally. it on the release host (see "Releasing" below); `just void-release` runs the same
command with `publish/` as the out-dir.
Fail-closed, in this order, before any container starts: Fail-closed, in this order, before any container starts:
- the tag must be exactly `v` plus `__version__`, and the committed template must - the tag must be exactly `v` plus `__version__`, and the committed template must
name that version and tag; name that version and tag;
- the checkout must be clean: no uncommitted or untracked change. The sdist is built
from the working tree, so anything else is not the tag. The commit it is built from
is recorded in `BUILT_FROM`;
- the key must be non-empty, valid, RSA 4096, and its public half must equal - the key must be non-empty, valid, RSA 4096, and its public half must equal
`repository-key.plist` byte for byte (a different key never publishes); `repository-key.plist` byte for byte (a different key never publishes);
- the canonical sdist must build with Python 3.11 and uv 0.12/0.13. Only its - the canonical sdist must build with Python 3.11 and uv 0.12/0.13. Only its
@@ -256,8 +262,9 @@ The committed templates must be unchanged after the build; the run fails if they
are not. `DIR` must be absent or empty, and the run writes `DIR/sdist/`, are not. `DIR` must be absent or empty, and the run writes `DIR/sdist/`,
`DIR/xbps/repo/` (packages, signatures, `x86_64-repodata`), `DIR/xbps/keys/` `DIR/xbps/repo/` (packages, signatures, `x86_64-repodata`), `DIR/xbps/keys/`
(the client `.plist` named by its fingerprint, and `FINGERPRINT`) and (the client `.plist` named by its fingerprint, and `FINGERPRINT`) and
`DIR/SDIST_SHA256`. A final check accepts only that set: nothing from musl, no `DIR/SDIST_SHA256` and `DIR/BUILT_FROM` (the tag and the commit it was built
other architecture, every package signed, the key the pinned one. from). A final check accepts only that set: nothing from musl, no other
architecture, every package signed, the key the pinned one.
### The repository key ### The repository key
@@ -273,8 +280,10 @@ fa:d1:00:32:ce:b1:b9:fd:73:05:05:76:75:54:37:25
`tests/test_void_signed_repo.py` derives the fingerprint from this file `tests/test_void_signed_repo.py` derives the fingerprint from this file
and fails if either README shows a different one. and fails if either README shows a different one.
- **Private half**: never committed, never in the repository, never on a - **Private half**: never committed, never in the repository, never on a
build host's disk beyond the run. It exists as the `XBPS_SIGNING_KEY` build host's disk beyond the run. It is kept in the maintainer's own backup,
CI secret (PEM text, newlines intact) and in the maintainer's own backup. and the release reads it as a file on tmpfs (`/dev/shm`) that is removed
afterwards. The `XBPS_SIGNING_KEY` CI secret holds the same PEM text for the
workflow, which is manual dispatch only.
- **Generating a new key** (only if the key is ever replaced; clients then - **Generating a new key** (only if the key is ever replaced; clients then
have to re-pin, so this is a release-notes event): have to re-pin, so this is a release-notes event):
@@ -292,35 +301,47 @@ openssl rsa -in xbps-sign-key.pem -pubout 2>/dev/null | openssl md5
python3 -c "import plistlib,sys; sys.stdout.buffer.write(plistlib.load(open('packaging/void/repository-key.plist','rb'))['public-key'])" | openssl md5 python3 -c "import plistlib,sys; sys.stdout.buffer.write(plistlib.load(open('packaging/void/repository-key.plist','rb'))['public-key'])" | openssl md5
``` ```
### Releasing: the Gitea workflow ### Releasing: the local sequence
A human pushes a version tag; `.gitea/workflows/xbps-release.yml` does the Pushing a tag starts nothing. `.gitea/workflows/xbps-release.yml` is manual dispatch
rest. It never runs on a schedule or a manual dispatch. only, kept for a future x86_64 host-mode runner; its one runner today is ARM64 and
cannot build this release. The sequence is:
1. **Tag gate.** `build-release.sh check-tag` (see above). Nothing else runs first. 1. **Merge the release pull request** to `main`. `origin/main` is then the commit to release.
2. **Key gate.** The CI secret `XBPS_SIGNING_KEY` must be non-empty. The key is 2. **Create and push the annotated tag** on that commit:
written with noclobber to a `0600` file on tmpfs (`/dev/shm`), never to disk. `git tag -a vX.Y.Z <sha> -m "Voculinux X.Y.Z"`, then `git push origin refs/tags/vX.Y.Z`.
3. **Runner check.** The tools in [Runner requirements](#runner-requirements) Never `git push --tags`: the clone holds upstream tags.
must be present, or the job stops and names what is missing. 3. **Build in a clean checkout of the tag:** `git worktree add /tmp/voculinux-vX.Y.Z vX.Y.Z`,
4. **Release.** `build-release.sh release`, with the key file (above). then, in that worktree,
5. **Publish.** The staged set goes to the tag's release and to the rolling `packaging/void/build-release.sh release vX.Y.Z --key-file /dev/shm/xbps-sign-key.pem --out-dir /path/to/out`.
`latest` release, which the client registration URL serves. Assets: Remove the key file when the run ends.
`sdist/voculinux-<version>.tar.gz`, the packages, their `.sig2` signatures, 4. **Dry-run the publish:** `packaging/void/publish-release.sh vX.Y.Z --out-dir /path/to/out --dry-run`.
`x86_64-repodata` (last), `FINGERPRINT` and the `.plist`. Nothing is built or It checks that origin's `vX.Y.Z` peels to the commit in `BUILT_FROM`, that the sdist
uploaded by hand. matches `SDIST_SHA256`, and that the client key is the committed one, then prints the
6. **Cleanup.** The key file is removed by a step that always runs. calls it would make. It sends no request and reads no token.
5. **Publish:** `packaging/void/publish-release.sh vX.Y.Z --out-dir /path/to/out`. The tag's
release gets the sdist, packages, signatures, `FINGERPRINT` and the client `.plist`, and
the rolling `latest` release that the client registration URL serves is replaced, with the
repodata last. The token comes from `GITEA_TOKEN`, or from the `tea` login for
`https://git.bongbetic.com`. It goes into a 0600 header file that is removed on exit and is
never printed.
6. **Post-release pull request:** record the `checksum=` the build printed in the template
(section 3.5 of `docs/RELEASE_PROCESS.md`).
Then the template's checksum is recorded by a pull request (above), and the `publish-release.sh` and the workflow's publish step make the same API calls: the sdist,
release notes show the sdist digest to copy from. packages, signatures and client key, then the repodata, to the tag's release and then to
`latest`. Any non-2xx response stops the publish. Publishing an older tag refreshes `latest`
with the older repository, so publish in version order.
Secret setup, once: Gitea repository settings, Actions, Secrets, add Secret setup for the workflow, once: Gitea repository settings, Actions, Secrets, add
`XBPS_SIGNING_KEY` with the PEM private key. The job token must be allowed to `XBPS_SIGNING_KEY` with the PEM private key. The job token must be allowed to create
create releases and upload release assets. releases and upload release assets.
### Runner requirements ### Runner requirements (the release host)
The job runs on the `bongbetic-ci` label as a host job. It is not a `container:` The release runs on the maintainer's x86_64 Void host. The workflow's `runs-on` label
job, and it never uses Docker. It needs: (`bongbetic-ci`) is ARM64 today and cannot run it. Requirements, which a future x86_64
host-mode runner would also need. It never uses Docker:
| requirement | why | check | | requirement | why | check |
|---|---|---| |---|---|---|
@@ -333,8 +354,7 @@ job, and it never uses Docker. It needs:
| about 25 GiB free disk | the void-packages bootstrap and the whisper.cpp build | `df -h` | | about 25 GiB free disk | the void-packages bootstrap and the whisper.cpp build | `df -h` |
| egress to `github.com`, `repo-default.voidlinux.org`, `ghcr.io`, `pypi.org`, `files.pythonhosted.org`, `git.bongbetic.com` | clone, bootstrap, base image, locked tools, distfiles, Gitea API | the build phase | | egress to `github.com`, `repo-default.voidlinux.org`, `ghcr.io`, `pypi.org`, `files.pythonhosted.org`, `git.bongbetic.com` | clone, bootstrap, base image, locked tools, distfiles, Gitea API | the build phase |
The job's `timeout-minutes` is 240. A cold build takes most of that: the A cold build is long: the whisper.cpp Vulkan build is the long step.
whisper.cpp Vulkan build is the long step.
## Client setup ## Client setup
+26 -3
View File
@@ -12,9 +12,11 @@
# src/vocalinux/version.py, and the committed Void template must already # src/vocalinux/version.py, and the committed Void template must already
# name that version and tag. Exits 1 otherwise. # name that version and tag. Exits 1 otherwise.
# build-release.sh release TAG --key-file FILE --out-dir DIR # build-release.sh release TAG --key-file FILE --out-dir DIR
# the whole release: gate, canonical sdist, signed repository, # the whole release: gate, clean-checkout gate, canonical sdist, signed
# completeness check. Writes DIR/sdist and DIR/xbps. Publishes nothing; # repository, completeness check. Writes DIR/sdist, DIR/xbps,
# the workflow publishes DIR. DIR must be absent or empty. # DIR/SDIST_SHA256 and DIR/BUILT_FROM (the tag and the commit built from).
# Publishes nothing; packaging/void/publish-release.sh publishes DIR.
# DIR must be absent or empty, and the checkout must be clean.
# build-release.sh smoke # build-release.sh smoke
# the same pipeline for the current version, signed with a throwaway # the same pipeline for the current version, signed with a throwaway
# RSA4096 key in a temporary directory. Proves the path works; nothing is # RSA4096 key in a temporary directory. Proves the path works; nothing is
@@ -127,6 +129,19 @@ check_template() {
fail "committed template _tag is '$have', not '$TAG'; update packaging/void/srcpkgs/voculinux/template" fail "committed template _tag is '$have', not '$TAG'; update packaging/void/srcpkgs/voculinux/template"
} }
# release_checkout: a release is built from a clean git checkout, and the commit
# it is built from is recorded in DIR/BUILT_FROM. The sdist is built from the
# working tree (tracked edits and non-ignored new files included), so anything
# but a clean tree is not the tag. Sets SOURCE_COMMIT.
release_checkout() {
git -C "$ROOT" rev-parse --git-dir >/dev/null 2>&1 ||
fail "a release is built from a git checkout of $TAG, and records its commit; $ROOT is not one"
[ -z "$(git -C "$ROOT" status --porcelain --untracked-files=normal)" ] ||
fail "the working tree has uncommitted or untracked changes; build $TAG from a clean checkout"
SOURCE_COMMIT=$(git -C "$ROOT" rev-parse --verify HEAD) ||
fail "could not read the commit HEAD"
}
# check_key KEY_FILE: RSA 4096, and for a release the public half must be the # check_key KEY_FILE: RSA 4096, and for a release the public half must be the
# committed repository-key.plist key. Sets FINGERPRINT and writes the client # committed repository-key.plist key. Sets FINGERPRINT and writes the client
# key to $WORK/client.plist. # key to $WORK/client.plist.
@@ -281,6 +296,9 @@ verify_set() {
run_pipeline() { run_pipeline() {
derive "$1" derive "$1"
check_template check_template
if [ "$MODE" = release ]; then
release_checkout
fi
WORK=$(mktemp -d "${TMPDIR:-/tmp}/voculinux-release.XXXXXX") WORK=$(mktemp -d "${TMPDIR:-/tmp}/voculinux-release.XXXXXX")
cleanup() { cleanup() {
status=$? status=$?
@@ -317,6 +335,11 @@ run_pipeline() {
phase_install phase_install
verify_set verify_set
printf '%s\n' "$SDIST_SHA256" >"$OUT/SDIST_SHA256" printf '%s\n' "$SDIST_SHA256" >"$OUT/SDIST_SHA256"
if [ "$MODE" = release ]; then
# The commit this set was built from: publish-release.sh refuses to publish
# unless origin's tag peels to it.
printf 'tag=%s\ncommit=%s\n' "$TAG" "$SOURCE_COMMIT" >"$OUT/BUILT_FROM"
fi
echo "template bump for the post-release pull request: checksum=$SDIST_SHA256" echo "template bump for the post-release pull request: checksum=$SDIST_SHA256"
} }
+362
View File
@@ -0,0 +1,362 @@
#!/usr/bin/env bash
# Publish an already-built Void release to the fork's Gitea release API (spec #16).
#
# A release is built and verified by packaging/void/build-release.sh, locally on
# the maintainer's x86_64 Void host (packaging/void/README.md, "Releasing"). This
# script publishes that out-dir and nothing else:
#
# publish-release.sh TAG --out-dir DIR [--dry-run]
#
# Before any API request it checks that:
# 1. TAG passes the tag gate (build-release.sh check-tag): exactly v plus
# __version__, and the committed template names this release;
# 2. DIR/BUILT_FROM names TAG and the commit the set was built from;
# 3. origin's refs/tags/TAG resolves to that commit. An annotated tag is compared
# by the commit it peels to; a lightweight tag is compared directly;
# 4. the sdist matches DIR/SDIST_SHA256, the client key is the committed
# packaging/void/repository-key.plist, and the repository has its packages,
# signatures and index.
# It then publishes the staged set the way the Gitea workflow's publish step does
# (.gitea/workflows/xbps-release.yml): the tag's release first, then the rolling
# `latest` release, with the repodata uploaded last.
#
# --dry-run runs every check above, reads no token, makes no API call, and prints
# the calls it would make, in order.
#
# Token: GITEA_TOKEN when it is set. Otherwise the token of the tea login for
# GITEA_HOST in $TEA_CONFIG (default ${XDG_CONFIG_HOME:-$HOME/.config}/tea/config.yml).
# The token is written to a 0600 header file in a private temporary directory,
# which is removed on exit. It is never printed.
#
# Environment: GITEA_HOST, GITEA_API (the repository's API root), REMOTE (default
# origin), TEA_CONFIG. Tests point them at a stub; a release uses the defaults.
set -euo pipefail
CDPATH=""
HERE=$(cd -- "$(dirname -- "$0")" && pwd)
ROOT=$(cd -- "$HERE/../.." && pwd)
PLIST="$HERE/repository-key.plist"
GITEA_HOST=${GITEA_HOST:-https://git.bongbetic.com}
GITEA_API=${GITEA_API:-https://git.bongbetic.com/api/v1/repos/xavierk/voculinux}
REMOTE=${REMOTE:-origin}
TEA_CONFIG=${TEA_CONFIG:-${XDG_CONFIG_HOME:-$HOME/.config}/tea/config.yml}
usage() {
echo "usage: publish-release.sh TAG --out-dir DIR [--dry-run]" >&2
exit 2
}
fail() {
echo "publish-release.sh: $*" >&2
exit 1
}
plan() {
printf ' %s\n' "$*"
}
urlencode() {
local s=$1 out="" c i
for ((i = 0; i < ${#s}; i++)); do
c=${s:i:1}
case $c in
[a-zA-Z0-9._~-]) out+=$c ;;
*) out+=$(printf '%%%02X' "'$c") ;;
esac
done
printf '%s' "$out"
}
# Reads the token of the tea login for GITEA_HOST and writes the header file
# named by argv[3]. Prints nothing else; a failure names the config, never a token.
TEA_READER=$(cat <<'PY'
import re
import sys
config, host, out = sys.argv[1:4]
try:
with open(config, encoding="utf-8") as handle:
text = handle.read()
except OSError:
sys.exit(f"no tea config at {config}")
# tea keeps a YAML list of logins; each block opens with "- name:".
for block in re.split(r"^\s*-\s+name:", text, flags=re.M)[1:]:
fields = {}
for line in block.splitlines():
match = re.match(r"\s*(url|token):\s*(.*?)\s*$", line)
if match:
fields[match.group(1)] = match.group(2).strip("'\"")
if fields.get("url", "").rstrip("/") == host and fields.get("token"):
with open(out, "w", encoding="utf-8") as handle:
handle.write("Authorization: token " + fields["token"] + "\n")
sys.exit(0)
sys.exit(f"no tea login for {host} with a token in {config}")
PY
)
TAG=""
OUT=""
DRY_RUN=0
[ "$#" -ge 1 ] || usage
TAG=$1
shift
while [ "$#" -gt 0 ]; do
case $1 in
--out-dir)
[ "$#" -ge 2 ] || usage
OUT=$2
shift 2
;;
--dry-run)
DRY_RUN=1
shift
;;
*) usage ;;
esac
done
[ -n "$OUT" ] || usage
# Gate 1: the tag gate, offline. Exactly v plus __version__, and the template names it.
"$HERE/build-release.sh" check-tag "$TAG" >/dev/null ||
fail "tag $TAG fails the tag gate (build-release.sh check-tag); refusing to publish"
names=$("$HERE/build-release.sh" names "$TAG") || fail "could not derive the names of $TAG"
SDIST_NAME=$(printf '%s\n' "$names" | sed -n 's/^SDIST_NAME=//p')
# Gate 2: the out-dir is the set build-release.sh built for this tag.
[ -d "$OUT" ] || fail "$OUT is not a directory"
OUT=$(cd -- "$OUT" && pwd)
[ -f "$OUT/BUILT_FROM" ] ||
fail "$OUT has no BUILT_FROM record: it was not written by build-release.sh release"
BUILT_TAG=$(sed -n 's/^tag=//p' "$OUT/BUILT_FROM")
BUILT_COMMIT=$(sed -n 's/^commit=//p' "$OUT/BUILT_FROM")
[ "$BUILT_TAG" = "$TAG" ] || fail "$OUT was built for '$BUILT_TAG', not '$TAG'"
printf '%s\n' "$BUILT_COMMIT" | grep -Eq '^[0-9a-f]{40}$' ||
fail "BUILT_FROM in $OUT names no commit"
shopt -s nullglob
[ -f "$OUT/sdist/$SDIST_NAME" ] && [ "$(find "$OUT/sdist" -mindepth 1 -maxdepth 1 | wc -l)" -eq 1 ] ||
fail "$OUT/sdist must hold exactly $SDIST_NAME"
[ -f "$OUT/SDIST_SHA256" ] || fail "$OUT has no SDIST_SHA256"
want_digest=$(cat "$OUT/SDIST_SHA256")
got_digest=$(sha256sum "$OUT/sdist/$SDIST_NAME" | cut -d' ' -f1)
[ "$got_digest" = "$want_digest" ] || fail "sdist/$SDIST_NAME does not match SDIST_SHA256"
[ -f "$OUT/xbps/keys/FINGERPRINT" ] || fail "$OUT has no FINGERPRINT record"
FINGERPRINT=$(sed -n 's/^fingerprint: //p' "$OUT/xbps/keys/FINGERPRINT")
[ -n "$FINGERPRINT" ] || fail "FINGERPRINT in $OUT names no fingerprint"
cmp -s "$PLIST" "$OUT/xbps/keys/$FINGERPRINT.plist" ||
fail "the client key in $OUT is not the committed packaging/void/repository-key.plist"
packages=("$OUT"/xbps/repo/*.x86_64.xbps)
signatures=("$OUT"/xbps/repo/*.sig2)
index=("$OUT"/xbps/repo/*-repodata*)
[ "${#packages[@]}" -gt 0 ] || fail "$OUT holds no packages"
[ "${#signatures[@]}" -eq "${#packages[@]}" ] || fail "every package needs exactly one signature"
[ "${#index[@]}" -gt 0 ] || fail "$OUT has no repository index (x86_64-repodata)"
# Gate 3: origin's tag is the commit the set was built from. Both refs are asked
# for in one call: the tag object, and the commit it peels to (`^{}`).
remote_refs=$(git -C "$ROOT" ls-remote --tags "$REMOTE" "refs/tags/$TAG" "refs/tags/$TAG^{}") ||
fail "could not read the tags of $REMOTE"
direct=$(printf '%s\n' "$remote_refs" | awk -v ref="refs/tags/$TAG" '$2 == ref { print $1 }')
peeled=$(printf '%s\n' "$remote_refs" | awk -v ref="refs/tags/$TAG^{}" '$2 == ref { print $1 }')
[ -n "$direct" ] ||
fail "tag $TAG is not on $REMOTE; push the annotated tag first (packaging/void/README.md)"
[ "$(printf '%s\n' "$direct" | wc -l)" -eq 1 ] || fail "$REMOTE has more than one $TAG"
if [ -n "$peeled" ]; then
REMOTE_COMMIT=$peeled
else
REMOTE_COMMIT=$direct
fi
[ "$REMOTE_COMMIT" = "$BUILT_COMMIT" ] ||
fail "$REMOTE's $TAG points at $REMOTE_COMMIT, but $OUT was built from $BUILT_COMMIT; refusing to publish"
echo "gates passed: $TAG on $REMOTE points at the commit $OUT was built from ($BUILT_COMMIT)"
# The staged set, in upload order: the sdist, packages, signatures and the client
# key first; the repodata last, so no client sees an index for packages not yet up.
cd "$OUT"
FILES=(
sdist/*.tar.gz
xbps/repo/*.x86_64.xbps
xbps/repo/*.sig2
xbps/keys/FINGERPRINT
xbps/keys/*.plist
xbps/repo/*-repodata*
)
NAMES=$(for f in "${FILES[@]}"; do basename "$f"; done)
SDIST_DIGEST=$want_digest
NOTES=$(printf '%s\n\n%s\n%s\n' \
"Signed xbps repository, Void Linux glibc x86_64, built from ${TAG} by packaging/void/build-release.sh and published by packaging/void/publish-release.sh. This is a third-party repository published by this project, not by upstream Void Linux. Registration: README.md." \
"$(cat xbps/keys/FINGERPRINT)" \
"sdist sha256: ${SDIST_DIGEST} (voculinux-*.tar.gz, the source the packages are built from)")
# Every request goes through api(), which is the only place curl runs. In a dry
# run each helper prints its planned call instead and no request is made.
api() {
local method=$1 url=$2 code
shift 2
code=$(curl -sS -o "$BODY" -w '%{http_code}' -X "$method" -H "@$AUTH" "$@" "$url") ||
fail "request failed: $method $url"
case $code in
2[0-9][0-9]) ;;
*)
cat "$BODY" >&2
echo >&2
fail "$method $url returned HTTP $code"
;;
esac
}
json_field() {
python3 -I -c 'import json,sys; print(json.load(open(sys.argv[1]))[sys.argv[2]])' "$BODY" "$1"
}
release_payload() {
python3 -I -c 'import json,sys; print(json.dumps({"tag_name": sys.argv[1], "target_commitish": sys.argv[2], "name": sys.argv[3], "body": sys.argv[4], "draft": False, "prerelease": False}))' "$@"
}
# release_id TAG: sets REL_ID to the release's id, or to "" when the tag has no
# release. Sets a variable rather than printing: a failure inside $(...) in an
# argument position would otherwise be swallowed.
release_id() {
local code
REL_ID=""
code=$(curl -sS -o "$BODY" -w '%{http_code}' -H "@$AUTH" \
"$GITEA_API/releases/tags/$(urlencode "$1")") || fail "request failed: GET releases/tags/$1"
case $code in
200) REL_ID=$(json_field id) ;;
404) ;;
*)
cat "$BODY" >&2
echo >&2
fail "GET releases/tags/$1 returned HTTP $code"
;;
esac
}
# ensure_release TAG NAME NOTES: sets REL_ID, creating the release at the built
# commit when the tag has none.
ensure_release() {
local payload
if [ "$DRY_RUN" -eq 1 ]; then
plan "GET $GITEA_API/releases/tags/$1"
plan "POST $GITEA_API/releases only when $1 has no release: tag_name=$1 target_commitish=$BUILT_COMMIT"
REL_ID="<$1 release id>"
return 0
fi
release_id "$1"
if [ -z "$REL_ID" ]; then
payload=$(release_payload "$1" "$BUILT_COMMIT" "$2" "$3")
api POST "$GITEA_API/releases" -H 'Content-Type: application/json' -d "$payload"
REL_ID=$(json_field id)
fi
}
# delete_assets ID MODE: MODE is "all", or "same" for only the assets whose names
# this run uploads again (a rerun on a tag release leaves nothing stale behind).
delete_assets() {
local listing aid aname
if [ "$DRY_RUN" -eq 1 ]; then
plan "GET $GITEA_API/releases/$1/assets"
if [ "$2" = all ]; then
plan "DELETE $GITEA_API/releases/$1/assets/<asset id> for every asset"
else
plan "DELETE $GITEA_API/releases/$1/assets/<asset id> for each asset this set uploads again"
fi
return 0
fi
api GET "$GITEA_API/releases/$1/assets"
listing=$(python3 -I -c 'import json,sys; [print(a["id"], a["name"]) for a in json.load(open(sys.argv[1]))]' "$BODY")
[ -n "$listing" ] || return 0
while read -r aid aname; do
if [ "$2" = all ] || printf '%s\n' "$NAMES" | grep -qxF -- "$aname"; then
api DELETE "$GITEA_API/releases/$1/assets/$aid"
echo "deleted: $aname"
fi
done <<<"$listing"
}
upload_assets() {
local f name
for f in "${FILES[@]}"; do
name=$(basename "$f")
if [ "$DRY_RUN" -eq 1 ]; then
plan "POST $GITEA_API/releases/$1/assets?name=$(urlencode "$name") attachment=$f"
continue
fi
api POST "$GITEA_API/releases/$1/assets?name=$(urlencode "$name")" \
-F "attachment=@${f};filename=${name}"
echo "uploaded: $name"
done
}
# verify_assets ID: every file this run uploaded is on the release.
verify_assets() {
local have missing
if [ "$DRY_RUN" -eq 1 ]; then
plan "GET $GITEA_API/releases/$1/assets (every uploaded name must be listed)"
return 0
fi
api GET "$GITEA_API/releases/$1/assets"
have=$(python3 -I -c 'import json,sys; [print(a["name"]) for a in json.load(open(sys.argv[1]))]' "$BODY" | sort)
missing=$(printf '%s\n' "$NAMES" | sort | comm -23 - <(printf '%s\n' "$have"))
[ -z "$missing" ] || fail "release $1 is missing uploaded assets: $missing"
}
if [ "$DRY_RUN" -eq 1 ]; then
echo "dry run: no request is sent and no token is read. Planned calls, in order:"
echo " Authorization: token <redacted>, API $GITEA_API"
echo "tag release $TAG (commit $BUILT_COMMIT):"
REL_ID=""
BODY=""
ensure_release "$TAG" "$TAG" "$NOTES"
TAG_ID=$REL_ID
delete_assets "$TAG_ID" same
upload_assets "$TAG_ID"
verify_assets "$TAG_ID"
echo "latest release (rolling, the registration URL serves it):"
ensure_release latest latest "$NOTES"
LATEST_ID=$REL_ID
plan "PATCH $GITEA_API/releases/$LATEST_ID name=latest, body=the same notes, with ${TAG} named"
delete_assets "$LATEST_ID" all
upload_assets "$LATEST_ID"
verify_assets "$LATEST_ID"
echo "dry run complete: nothing was published"
exit 0
fi
# Token: GITEA_TOKEN first (the workflow's job token), then the tea login.
AUTH_DIR=$(mktemp -d "${TMPDIR:-/tmp}/publish-release.XXXXXX")
trap 'rm -rf "$AUTH_DIR"' EXIT
AUTH="$AUTH_DIR/authorization.h"
BODY="$AUTH_DIR/body.json"
umask 077
if [ -n "${GITEA_TOKEN:-}" ]; then
printf 'Authorization: token %s\n' "$GITEA_TOKEN" >"$AUTH"
else
python3 -I -c "$TEA_READER" "$TEA_CONFIG" "$GITEA_HOST" "$AUTH" ||
fail "no tea login for $GITEA_HOST with a token (set GITEA_TOKEN, or log in with tea)"
fi
echo "publishing ${TAG} (built from ${BUILT_COMMIT}) to ${GITEA_API}"
echo "== tag release ${TAG}"
ensure_release "$TAG" "$TAG" "$NOTES"
TAG_ID=$REL_ID
delete_assets "$TAG_ID" same
upload_assets "$TAG_ID"
verify_assets "$TAG_ID"
# The rolling release the registration URL points at. The tag it was created
# with stays where it was first made; its assets are what clients download.
echo "== latest release"
LATEST_NOTES=$(printf '%s\n\n%s\n' \
"Rolling signed xbps repository, currently built from ${TAG} (${BUILT_COMMIT})." \
"$NOTES")
ensure_release latest latest "$LATEST_NOTES"
LATEST_ID=$REL_ID
PATCH_PAYLOAD=$(python3 -I -c 'import json,sys; print(json.dumps({"name": sys.argv[1], "body": sys.argv[2]}))' latest "$LATEST_NOTES")
api PATCH "$GITEA_API/releases/$LATEST_ID" -H 'Content-Type: application/json' -d "$PATCH_PAYLOAD"
delete_assets "$LATEST_ID" all
upload_assets "$LATEST_ID"
verify_assets "$LATEST_ID"
echo "OK: signed repository published to ${TAG} and latest"
+2 -2
View File
@@ -1,11 +1,11 @@
# Template file for 'voculinux' # Template file for 'voculinux'
pkgname=voculinux pkgname=voculinux
version=0.18.5 version=0.19.0
revision=1 revision=1
# The release tag this template names: v + __version__, with the pre-release # The release tag this template names: v + __version__, with the pre-release
# hyphen kept (the tag is v0.19.0-beta while version is 0.19.0beta). The release # hyphen kept (the tag is v0.19.0-beta while version is 0.19.0beta). The release
# gate (packaging/void/build-release.sh check-tag) refuses any mismatch. # gate (packaging/void/build-release.sh check-tag) refuses any mismatch.
_tag=v0.18.5 _tag=v0.19.0
# The release sdist ships no setup.py (setuptools pyproject-only), so the # The release sdist ships no setup.py (setuptools pyproject-only), so the
# build goes through the PEP 517 wheel (cf. Solaar, PhotoCollage). # build goes through the PEP 517 wheel (cf. Solaar, PhotoCollage).
build_style=python3-pep517 build_style=python3-pep517
+5 -5
View File
@@ -123,11 +123,11 @@ voculinux = "vocalinux.main:main"
voculinux-gui = "vocalinux.main:main" voculinux-gui = "vocalinux.main:main"
[project.urls] [project.urls]
Homepage = "https://github.com/VocaHQ/vocalinux" Homepage = "https://git.bongbetic.com/xavierk/voculinux"
Documentation = "https://github.com/VocaHQ/vocalinux/tree/main/docs" Documentation = "https://git.bongbetic.com/xavierk/voculinux/src/branch/main/docs"
Repository = "https://github.com/VocaHQ/vocalinux" Repository = "https://git.bongbetic.com/xavierk/voculinux"
Issues = "https://github.com/VocaHQ/vocalinux/issues" Issues = "https://git.bongbetic.com/xavierk/voculinux/issues"
Changelog = "https://github.com/VocaHQ/vocalinux/releases" Changelog = "https://git.bongbetic.com/xavierk/voculinux/releases"
[tool.uv] [tool.uv]
# Contributors may use uv 0.12 or 0.13. CI and lock/export generation use # Contributors may use uv 0.12 or 0.13. CI and lock/export generation use
@@ -8,14 +8,13 @@ Rungs (quantized Turbo first, small floor last)::
large-v3-turbo-q8_0 -> large-v3-turbo-q5_0 -> small.en / small large-v3-turbo-q8_0 -> large-v3-turbo-q5_0 -> small.en / small
The thresholds are PROVISIONAL. They are the rule The thresholds are hard-coded from the ticket #19 installed-build measurement
``scripts/measure_perf_baseline.py::recommend_thresholds`` applies (measured (issue #19, comment 12913): each rung's transcribe-time VRAM delta plus the
footprint plus the 512 MiB desktop headroom) to the transcribe peaks 512 MiB desktop headroom that ``scripts/measure_perf_baseline.py`` applies.
recorded for ticket #19 (1132 MiB for q8_0, 846 MiB for q5_0). Ticket #19 They are per GPU backend, because CUDA's transcribe footprint is larger than
is still open and its measurement is not recorded in this repository, so Vulkan's on the same card. ``tests/test_model_ladder.py`` pins the constants
these numbers are the calibration to replace, not evidence. A re-measure and the measured inputs they derive from, so a re-measure changes them on
changes the constants here, and ``tests/test_model_ladder.py`` pins them so purpose.
that change is deliberate.
Backend order follows the same readiness rule: CUDA ranks before Vulkan Backend order follows the same readiness rule: CUDA ranks before Vulkan
only when a complete architecture-matching CUDA toolkit is installed only when a complete architecture-matching CUDA toolkit is installed
@@ -47,14 +46,22 @@ LADDER_RUNGS: tuple[str, ...] = (DEFAULT_RUNG, LOW_MEMORY_RUNG)
#: the baseline protocol adds to measured footprints. #: the baseline protocol adds to measured footprints.
HEADROOM_MIB: float = 512.0 HEADROOM_MIB: float = 512.0
#: Provisional transcribe peaks (MiB) from ticket #19, not yet recorded in the repo. #: Free-memory thresholds (MiB) per GPU backend, from ticket #19 (comment 12913):
_MEASURED_Q8_0_PEAK_MIB: float = 1132.0 #: the measured transcribe-time VRAM delta plus HEADROOM_MIB.
_MEASURED_Q5_0_PEAK_MIB: float = 846.0 #: vulkan: q8_0 1132 + 512 = 1644, q5_0 847 + 512 = 1359
#: cuda: q8_0 1178 + 512 = 1690, q5_0 892 + 512 = 1404
_PREFER_ABOVE_MIB: dict[str, dict[str, float]] = {
"vulkan": {DEFAULT_RUNG: 1644.0, LOW_MEMORY_RUNG: 1359.0},
"cuda": {DEFAULT_RUNG: 1690.0, LOW_MEMORY_RUNG: 1404.0},
}
#: Free memory at or above this keeps the default q8_0 rung. #: Free memory at or above this keeps the default q8_0 rung on Vulkan.
PREFER_Q8_0_ABOVE_MIB: float = _MEASURED_Q8_0_PEAK_MIB + HEADROOM_MIB PREFER_Q8_0_ABOVE_MIB: float = _PREFER_ABOVE_MIB["vulkan"][DEFAULT_RUNG]
#: Free memory at or above this keeps the q5_0 rung; below it is the floor. #: Free memory at or above this keeps the q5_0 rung on Vulkan; below it is the floor.
PREFER_Q5_0_ABOVE_MIB: float = _MEASURED_Q5_0_PEAK_MIB + HEADROOM_MIB PREFER_Q5_0_ABOVE_MIB: float = _PREFER_ABOVE_MIB["vulkan"][LOW_MEMORY_RUNG]
#: The CUDA thresholds, named for callers and tests that compare against them.
PREFER_CUDA_Q8_0_ABOVE_MIB: float = _PREFER_ABOVE_MIB["cuda"][DEFAULT_RUNG]
PREFER_CUDA_Q5_0_ABOVE_MIB: float = _PREFER_ABOVE_MIB["cuda"][LOW_MEMORY_RUNG]
#: Minimum CUDA compute capability (major, minor) the bundled backend #: Minimum CUDA compute capability (major, minor) the bundled backend
#: targets. Older architectures cannot run its kernels. #: targets. Older architectures cannot run its kernels.
@@ -79,14 +86,31 @@ def _floor_rung(language_is_english: bool) -> str:
return FLOOR_RUNG_ENGLISH if language_is_english else FLOOR_RUNG_MULTILINGUAL return FLOOR_RUNG_ENGLISH if language_is_english else FLOOR_RUNG_MULTILINGUAL
def select_ladder_rung(language_is_english: bool, free_mib: float) -> str: def _thresholds_for(gpu_backend: str) -> tuple[float, float]:
"""Return ``(q8_0, q5_0)`` free-memory thresholds for the installed build's backend.
``cuda`` has its own pair. ``vulkan``, ``cpu`` and any unknown name share the
Vulkan pair: a CPU build reads system RAM against the same rule as before.
"""
table = _PREFER_ABOVE_MIB.get(gpu_backend, _PREFER_ABOVE_MIB["vulkan"])
return table[DEFAULT_RUNG], table[LOW_MEMORY_RUNG]
def select_ladder_rung(
language_is_english: bool,
free_mib: float,
gpu_backend: str = "vulkan",
) -> str:
"""Return the ladder rung that fits ``free_mib`` MiB of free memory. """Return the ladder rung that fits ``free_mib`` MiB of free memory.
Pure function of its inputs so the policy is testable without hardware. ``gpu_backend`` is the backend the installed build has (``vulkan``, ``cuda``
or ``cpu``); it selects the thresholds. Pure function of its inputs so the
policy is testable without hardware.
""" """
if free_mib >= PREFER_Q8_0_ABOVE_MIB: q8_threshold, q5_threshold = _thresholds_for(gpu_backend)
if free_mib >= q8_threshold:
return DEFAULT_RUNG return DEFAULT_RUNG
if free_mib >= PREFER_Q5_0_ABOVE_MIB: if free_mib >= q5_threshold:
return LOW_MEMORY_RUNG return LOW_MEMORY_RUNG
return _floor_rung(language_is_english) return _floor_rung(language_is_english)
@@ -95,15 +119,16 @@ def resolve_startup_rung(
pinned_variant: str, pinned_variant: str,
language_is_english: bool, language_is_english: bool,
free_mib: float, free_mib: float,
gpu_backend: str = "vulkan",
) -> str: ) -> str:
"""Return the rung to load: a manual pin wins verbatim, otherwise the ladder. """Return the rung to load: a manual pin wins verbatim, otherwise the ladder.
A manual selection is never overridden here; only unpinned loads derive A manual selection is never overridden here; only unpinned loads derive
their rung from free memory. their rung from free memory, on the thresholds of ``gpu_backend``.
""" """
if pinned_variant: if pinned_variant:
return pinned_variant return pinned_variant
return select_ladder_rung(language_is_english, free_mib) return select_ladder_rung(language_is_english, free_mib, gpu_backend)
def next_rung_down(rung: str, language_is_english: bool) -> str | None: def next_rung_down(rung: str, language_is_english: bool) -> str | None:
@@ -354,6 +379,8 @@ __all__ = [
"HEADROOM_MIB", "HEADROOM_MIB",
"PREFER_Q8_0_ABOVE_MIB", "PREFER_Q8_0_ABOVE_MIB",
"PREFER_Q5_0_ABOVE_MIB", "PREFER_Q5_0_ABOVE_MIB",
"PREFER_CUDA_Q8_0_ABOVE_MIB",
"PREFER_CUDA_Q5_0_ABOVE_MIB",
"MIN_CUDA_COMPUTE_CAPABILITY", "MIN_CUDA_COMPUTE_CAPABILITY",
"GPU_BACKENDS", "GPU_BACKENDS",
"select_ladder_rung", "select_ladder_rung",
+13 -3
View File
@@ -776,13 +776,23 @@ class ConfigManager:
# id, so first run loads large-v3-turbo-q8_0 where it fits (ticket # id, so first run loads large-v3-turbo-q8_0 where it fits (ticket
# #20). Any explicit size or pin keeps the behavior below untouched. # #20). Any explicit size or pin keeps the behavior below untouched.
if not pinned and str(saved or "tiny").lower() == "tiny": if not pinned and str(saved or "tiny").lower() == "tiny":
from ..speech_recognition.model_ladder import free_memory_mib, resolve_startup_rung from ..speech_recognition.model_ladder import (
free_memory_for_backend,
resolve_startup_rung,
)
from ..speech_recognition.recognition_manager import detect_pywhispercpp_gpu_backend
language_id = str(sr_config.get("language", "auto")) language_id = str(sr_config.get("language", "auto"))
language_info = cast(dict[str, str], SUPPORTED_LANGUAGES.get(language_id, {})) language_info = cast(dict[str, str], SUPPORTED_LANGUAGES.get(language_id, {}))
free_mib = free_memory_mib() # The memory the installed build uses (VRAM for a GPU build, RAM otherwise)
# and the thresholds of that same backend, as the recommendation does.
gpu_backend = detect_pywhispercpp_gpu_backend()
free_mib, _source = free_memory_for_backend(gpu_backend)
rung = resolve_startup_rung( rung = resolve_startup_rung(
"", language_info.get("whisper") == "en", free_mib if free_mib is not None else 0.0 "",
language_info.get("whisper") == "en",
free_mib if free_mib is not None else 0.0,
gpu_backend,
) )
logger.info( logger.info(
"whisper.cpp startup rung %s (free memory %.0f MiB)", "whisper.cpp startup rung %s (free memory %.0f MiB)",
+1 -1
View File
@@ -571,7 +571,7 @@ def get_recommended_model() -> tuple[str, str]:
if free_mib is None: if free_mib is None:
logger.info("Free memory unreadable; recommending the floor model") logger.info("Free memory unreadable; recommending the floor model")
return FLOOR_RUNG_MULTILINGUAL, "Free memory unreadable; floor model" return FLOOR_RUNG_MULTILINGUAL, "Free memory unreadable; floor model"
rung = resolve_startup_rung("", False, free_mib) rung = resolve_startup_rung("", False, free_mib, gpu_backend)
return rung, f"{rung}: {free_mib:.0f} MiB free {source} ({gpu_backend} build)" return rung, f"{rung}: {free_mib:.0f} MiB free {source} ({gpu_backend} build)"
+2 -2
View File
@@ -2,8 +2,8 @@
Version information for Voculinux. Version information for Voculinux.
""" """
__version__ = "0.18.5" __version__ = "0.19.0"
__version_info__ = (0, 18, 5) __version_info__ = (0, 19, 0)
__author__ = "Jatin K Malik" __author__ = "Jatin K Malik"
__email__ = "jatinkrmalik@gmail.com" __email__ = "jatinkrmalik@gmail.com"
__license__ = "AGPL-3.0-only" __license__ = "AGPL-3.0-only"
+10 -1
View File
@@ -527,7 +527,16 @@ class TestConfigManager(unittest.TestCase):
# The migrated whisper.cpp default ("tiny", unpinned) resolves through # The migrated whisper.cpp default ("tiny", unpinned) resolves through
# the memory-aware ladder instead of staying tiny (ticket #20): what # the memory-aware ladder instead of staying tiny (ticket #20): what
# matters here is that vosk's "large" does not leak across engines. # matters here is that vosk's "large" does not leak across engines.
with patch("vocalinux.speech_recognition.model_ladder.free_memory_mib", return_value=100.0): with (
patch(
"vocalinux.speech_recognition.model_ladder.free_memory_for_backend",
return_value=(100.0, "RAM"),
),
patch(
"vocalinux.speech_recognition.recognition_manager.detect_pywhispercpp_gpu_backend",
return_value="cpu",
),
):
# "auto" language pairs the floor with the multilingual rung. # "auto" language pairs the floor with the multilingual rung.
self.assertEqual(config_manager.get_model_size_for_engine("whisper_cpp"), "small") self.assertEqual(config_manager.get_model_size_for_engine("whisper_cpp"), "small")
self.assertEqual(config_manager.get_model_size_for_engine("whisper"), "tiny") self.assertEqual(config_manager.get_model_size_for_engine("whisper"), "tiny")
+92 -30
View File
@@ -1,14 +1,14 @@
"""Ticket #20: memory-aware model ladder, conditional backend order, retry. """Ticket #20: memory-aware model ladder, conditional backend order, retry.
All behavior is exercised with mocked probes and loaders: no GPU, no All behavior is exercised with mocked probes and loaders: no GPU, no
network, no hardware. The threshold-derivation test pins the rung network, no hardware. The threshold tests pin the per-backend rung thresholds
thresholds to the provisional transcribe peaks (1132 MiB for q8_0, 846 MiB to the ticket #19 installed-build measurement (issue #19, comment 12913): the
for q5_0, ticket #19, not yet recorded in the repo) plus the 512 MiB transcribe-time VRAM delta of each rung plus the 512 MiB protocol headroom.
protocol headroom. Replacing them with a recorded measurement is a Re-measuring changes those constants, and this file must change with them.
deliberate change to this test.
""" """
import os import os
from contextlib import ExitStack
from pathlib import Path from pathlib import Path
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
@@ -21,6 +21,8 @@ from vocalinux.speech_recognition.model_ladder import (
FLOOR_RUNG_MULTILINGUAL, FLOOR_RUNG_MULTILINGUAL,
HEADROOM_MIB, HEADROOM_MIB,
LOW_MEMORY_RUNG, LOW_MEMORY_RUNG,
PREFER_CUDA_Q5_0_ABOVE_MIB,
PREFER_CUDA_Q8_0_ABOVE_MIB,
PREFER_Q5_0_ABOVE_MIB, PREFER_Q5_0_ABOVE_MIB,
PREFER_Q8_0_ABOVE_MIB, PREFER_Q8_0_ABOVE_MIB,
cuda_toolkit_ready_from_probe, cuda_toolkit_ready_from_probe,
@@ -38,22 +40,40 @@ def _size_mb(name: str) -> float:
class TestThresholdDerivation: class TestThresholdDerivation:
"""Thresholds match the baseline ticket's recorded recommendation.""" """Thresholds are the ticket #19 recorded transcribe-time deltas plus headroom.
#: Ticket #19 measured transcribe peaks (MiB) per rung. The inputs are copied from issue #19, comment 12913 ("Per-rung load
Q8_0_MEASURED_MIB: float = 1132.0 footprint", transcribe delta column), not recomputed from the code.
Q5_0_MEASURED_MIB: float = 846.0 """
def test_q8_threshold_is_measured_footprint_plus_headroom(self) -> None: #: Transcribe-time VRAM delta (MiB), peak minus before, per backend and rung.
assert PREFER_Q8_0_ABOVE_MIB == self.Q8_0_MEASURED_MIB + HEADROOM_MIB VULKAN_Q8_0_DELTA_MIB: float = 1132.0
VULKAN_Q5_0_DELTA_MIB: float = 847.0
CUDA_Q8_0_DELTA_MIB: float = 1178.0
CUDA_Q5_0_DELTA_MIB: float = 892.0
def test_headroom_is_the_protocol_value(self) -> None:
assert HEADROOM_MIB == 512.0
def test_vulkan_q8_threshold_is_measured_delta_plus_headroom(self) -> None:
assert PREFER_Q8_0_ABOVE_MIB == self.VULKAN_Q8_0_DELTA_MIB + HEADROOM_MIB
assert PREFER_Q8_0_ABOVE_MIB == 1644.0 assert PREFER_Q8_0_ABOVE_MIB == 1644.0
def test_q5_threshold_is_measured_footprint_plus_headroom(self) -> None: def test_vulkan_q5_threshold_is_measured_delta_plus_headroom(self) -> None:
assert PREFER_Q5_0_ABOVE_MIB == self.Q5_0_MEASURED_MIB + HEADROOM_MIB assert PREFER_Q5_0_ABOVE_MIB == self.VULKAN_Q5_0_DELTA_MIB + HEADROOM_MIB
assert PREFER_Q5_0_ABOVE_MIB == 1358.0 assert PREFER_Q5_0_ABOVE_MIB == 1359.0
def test_rungs_are_ordered(self) -> None: def test_cuda_q8_threshold_is_measured_delta_plus_headroom(self) -> None:
assert PREFER_CUDA_Q8_0_ABOVE_MIB == self.CUDA_Q8_0_DELTA_MIB + HEADROOM_MIB
assert PREFER_CUDA_Q8_0_ABOVE_MIB == 1690.0
def test_cuda_q5_threshold_is_measured_delta_plus_headroom(self) -> None:
assert PREFER_CUDA_Q5_0_ABOVE_MIB == self.CUDA_Q5_0_DELTA_MIB + HEADROOM_MIB
assert PREFER_CUDA_Q5_0_ABOVE_MIB == 1404.0
def test_rungs_are_ordered_on_each_backend(self) -> None:
assert PREFER_Q8_0_ABOVE_MIB > PREFER_Q5_0_ABOVE_MIB assert PREFER_Q8_0_ABOVE_MIB > PREFER_Q5_0_ABOVE_MIB
assert PREFER_CUDA_Q8_0_ABOVE_MIB > PREFER_CUDA_Q5_0_ABOVE_MIB
assert _size_mb(DEFAULT_RUNG) > _size_mb(LOW_MEMORY_RUNG) assert _size_mb(DEFAULT_RUNG) > _size_mb(LOW_MEMORY_RUNG)
@@ -77,11 +97,44 @@ class TestSelectLadderRung:
def test_recorded_boundaries_select_recorded_rungs(self) -> None: def test_recorded_boundaries_select_recorded_rungs(self) -> None:
assert select_ladder_rung(True, 1644.0) == DEFAULT_RUNG assert select_ladder_rung(True, 1644.0) == DEFAULT_RUNG
assert select_ladder_rung(True, 1500.0) == LOW_MEMORY_RUNG assert select_ladder_rung(True, 1500.0) == LOW_MEMORY_RUNG
assert select_ladder_rung(True, 1358.0) == LOW_MEMORY_RUNG assert select_ladder_rung(True, 1359.0) == LOW_MEMORY_RUNG
assert select_ladder_rung(True, 1358.0) == FLOOR_RUNG_ENGLISH
assert select_ladder_rung(True, 1200.0) == FLOOR_RUNG_ENGLISH assert select_ladder_rung(True, 1200.0) == FLOOR_RUNG_ENGLISH
assert select_ladder_rung(False, 1200.0) == FLOOR_RUNG_MULTILINGUAL assert select_ladder_rung(False, 1200.0) == FLOOR_RUNG_MULTILINGUAL
class TestBackendThresholds:
"""Each GPU backend keys to its own measured thresholds (ticket #19)."""
def test_cuda_keeps_default_only_above_its_own_q8_threshold(self) -> None:
assert select_ladder_rung(True, 1690.0, "cuda") == DEFAULT_RUNG
assert select_ladder_rung(True, 1689.0, "cuda") == LOW_MEMORY_RUNG
# 1644 MiB is enough for Vulkan's q8_0 but not for CUDA's.
assert select_ladder_rung(True, 1644.0, "cuda") == LOW_MEMORY_RUNG
assert select_ladder_rung(True, 1644.0, "vulkan") == DEFAULT_RUNG
def test_cuda_q5_threshold_is_its_own(self) -> None:
assert select_ladder_rung(True, 1404.0, "cuda") == LOW_MEMORY_RUNG
assert select_ladder_rung(True, 1403.0, "cuda") == FLOOR_RUNG_ENGLISH
assert select_ladder_rung(True, 1359.0, "vulkan") == LOW_MEMORY_RUNG
assert select_ladder_rung(True, 1359.0, "cuda") == FLOOR_RUNG_ENGLISH
def test_cpu_and_unknown_backends_use_the_vulkan_thresholds(self) -> None:
for backend in ("cpu", "unknown"):
assert select_ladder_rung(True, 1644.0, backend) == DEFAULT_RUNG
assert select_ladder_rung(True, 1359.0, backend) == LOW_MEMORY_RUNG
assert select_ladder_rung(True, 1358.0, backend) == FLOOR_RUNG_ENGLISH
def test_default_backend_is_vulkan(self) -> None:
assert select_ladder_rung(True, 1690.0) == DEFAULT_RUNG
assert select_ladder_rung(True, 1644.0) == DEFAULT_RUNG
def test_resolve_passes_the_backend_through(self) -> None:
assert resolve_startup_rung("", True, 1644.0, "cuda") == LOW_MEMORY_RUNG
assert resolve_startup_rung("", True, 1644.0, "vulkan") == DEFAULT_RUNG
assert resolve_startup_rung("tiny", True, 0.0, "cuda") == "tiny"
class TestResolveStartupRung: class TestResolveStartupRung:
"""A manual pin wins verbatim; only unpinned loads use the ladder.""" """A manual pin wins verbatim; only unpinned loads use the ladder."""
@@ -372,6 +425,24 @@ class TestStepDownRetry:
assert loader.call_count == 1 assert loader.call_count == 1
def _probe_memory(free_mib: float, backend: str = "cpu"):
"""Stub the startup probes: free memory of the installed build, and its backend."""
stack = ExitStack()
stack.enter_context(
patch(
"vocalinux.speech_recognition.model_ladder.free_memory_for_backend",
return_value=(free_mib, "RAM"),
)
)
stack.enter_context(
patch(
"vocalinux.speech_recognition.recognition_manager.detect_pywhispercpp_gpu_backend",
return_value=backend,
)
)
return stack
class TestStartupRungSelection: class TestStartupRungSelection:
"""Unpinned defaults resolve through the ladder; explicit picks do not.""" """Unpinned defaults resolve through the ladder; explicit picks do not."""
@@ -390,9 +461,7 @@ class TestStartupRungSelection:
def test_unpinned_default_selects_rung_by_memory(self) -> None: def test_unpinned_default_selects_rung_by_memory(self) -> None:
manager = self._manager_with("tiny", "", "en-us") manager = self._manager_with("tiny", "", "en-us")
with patch( with _probe_memory(99999.0):
"vocalinux.speech_recognition.model_ladder.free_memory_mib", return_value=99999.0
):
with patch( with patch(
"vocalinux.utils.whispercpp_model_info.is_model_downloaded", return_value=False "vocalinux.utils.whispercpp_model_info.is_model_downloaded", return_value=False
): ):
@@ -400,10 +469,7 @@ class TestStartupRungSelection:
def test_unpinned_default_low_memory_selects_q5(self) -> None: def test_unpinned_default_low_memory_selects_q5(self) -> None:
manager = self._manager_with("tiny", "", "en-us") manager = self._manager_with("tiny", "", "en-us")
with patch( with _probe_memory(PREFER_Q5_0_ABOVE_MIB):
"vocalinux.speech_recognition.model_ladder.free_memory_mib",
return_value=PREFER_Q5_0_ABOVE_MIB,
):
with patch( with patch(
"vocalinux.utils.whispercpp_model_info.is_model_downloaded", return_value=False "vocalinux.utils.whispercpp_model_info.is_model_downloaded", return_value=False
): ):
@@ -411,7 +477,7 @@ class TestStartupRungSelection:
def test_unpinned_default_tight_memory_selects_floor(self) -> None: def test_unpinned_default_tight_memory_selects_floor(self) -> None:
manager = self._manager_with("tiny", "", "pl") manager = self._manager_with("tiny", "", "pl")
with patch("vocalinux.speech_recognition.model_ladder.free_memory_mib", return_value=100.0): with _probe_memory(100.0):
with patch( with patch(
"vocalinux.utils.whispercpp_model_info.is_model_downloaded", return_value=False "vocalinux.utils.whispercpp_model_info.is_model_downloaded", return_value=False
): ):
@@ -419,16 +485,12 @@ class TestStartupRungSelection:
def test_pinned_variant_never_uses_ladder(self) -> None: def test_pinned_variant_never_uses_ladder(self) -> None:
manager = self._manager_with("tiny", LOW_MEMORY_RUNG, "en-us") manager = self._manager_with("tiny", LOW_MEMORY_RUNG, "en-us")
with patch( with _probe_memory(99999.0):
"vocalinux.speech_recognition.model_ladder.free_memory_mib", return_value=99999.0
):
assert manager.get_model_size_for_engine("whisper_cpp") == LOW_MEMORY_RUNG assert manager.get_model_size_for_engine("whisper_cpp") == LOW_MEMORY_RUNG
def test_explicit_size_keeps_legacy_derivation(self) -> None: def test_explicit_size_keeps_legacy_derivation(self) -> None:
manager = self._manager_with("medium", "", "en-us") manager = self._manager_with("medium", "", "en-us")
with patch( with _probe_memory(99999.0):
"vocalinux.speech_recognition.model_ladder.free_memory_mib", return_value=99999.0
):
with patch( with patch(
"vocalinux.utils.whispercpp_model_info.is_model_downloaded", return_value=False "vocalinux.utils.whispercpp_model_info.is_model_downloaded", return_value=False
): ):
+576
View File
@@ -0,0 +1,576 @@
"""Offline guards for publishing a built Void release (spec #16; packaging/void/publish-release.sh).
``publish-release.sh`` publishes an out-dir that ``build-release.sh release``
already built and verified. These tests run the real script in a scratch git
checkout whose ``origin`` is a local bare repository, so the tag gate and the
``ls-remote`` comparison are exercised for real. The Gitea API is the stateful
``curl`` stand-in from ``test_xbps_release_workflow.py`` (``FakeGitea``): every
request, its order and its authentication are recorded, and nothing leaves the
machine. The release itself is never run.
The expected values come from the documented rules (the tag must be on origin
at the built commit, the repodata goes last, the token never leaves the header
file), not from the script.
"""
import hashlib
import os
import re
import shutil
import subprocess
from pathlib import Path
from typing import List, Optional
import pytest
from test_xbps_release_workflow import FakeGitea
REPO_ROOT = Path(__file__).resolve().parents[1]
VOID = REPO_ROOT / "packaging" / "void"
BUILD_RELEASE = VOID / "build-release.sh"
PUBLISH_RELEASE = VOID / "publish-release.sh"
SMOKE_SH = VOID / "smoke.sh"
PINS = VOID / "tool_checksums.txt"
PLIST = VOID / "repository-key.plist"
TEMPLATE = VOID / "srcpkgs" / "voculinux" / "template"
TAG = "v0.19.0"
XBPS = "0.19.0"
SDIST = "voculinux-0.19.0.tar.gz"
HOST = "https://git.bongbetic.com"
API = f"{HOST}/api/v1/repos/xavierk/voculinux"
TOKEN = "gitea-token-from-the-environment"
TEA_TOKEN = "tea-token-from-the-config-file"
FINGERPRINT = "fa:d1:00:32:ce:b1:b9:fd:73:05:05:76:75:54:37:25"
GIT_ENV = {
"GIT_CONFIG_GLOBAL": os.devnull,
"GIT_CONFIG_NOSYSTEM": "1",
"GIT_AUTHOR_NAME": "fixture",
"GIT_AUTHOR_EMAIL": "fixture@example.invalid",
"GIT_COMMITTER_NAME": "fixture",
"GIT_COMMITTER_EMAIL": "fixture@example.invalid",
"GIT_TERMINAL_PROMPT": "0",
}
def _git(cwd: Path, *args: str) -> str:
result = subprocess.run(
["git", *args],
cwd=cwd,
env={**os.environ, **GIT_ENV},
capture_output=True,
text=True,
check=True,
timeout=60,
)
return result.stdout.strip()
def _smoke_packages() -> List[str]:
match = re.search(r'^PKGS="([^"]+)"', SMOKE_SH.read_text(encoding="utf-8"), re.M)
assert match, "smoke.sh must declare its ordered PKGS list"
return match.group(1).split()
class Release:
"""A scratch checkout with an origin, a built out-dir, and a fake Gitea on PATH."""
def __init__(self, root: Path) -> None:
self.root = root
self.tree = root / "checkout"
self.origin = root / "origin.git"
self.out = root / "publish"
self.home = root / "home"
self.tmp = root / "tmp"
for directory in (self.home, self.tmp):
directory.mkdir()
self._write_tree()
_git(root, "init", "-q", "--bare", str(self.origin))
_git(self.tree, "remote", "add", "origin", str(self.origin))
self.gitea = FakeGitea(root)
def _write_tree(self) -> None:
pkg = self.tree / "packaging" / "void"
pkg.mkdir(parents=True)
for source in (BUILD_RELEASE, PUBLISH_RELEASE, SMOKE_SH, PINS, PLIST):
shutil.copy2(source, pkg / source.name)
(pkg / "srcpkgs" / "voculinux").mkdir(parents=True)
text = TEMPLATE.read_text(encoding="utf-8")
text = re.sub(r"^version=.*$", f"version={XBPS}", text, flags=re.M)
text = re.sub(r"^_tag=.*$", f"_tag={TAG}", text, flags=re.M)
(pkg / "srcpkgs" / "voculinux" / "template").write_text(text, encoding="utf-8")
src = self.tree / "src" / "vocalinux"
src.mkdir(parents=True)
(src / "version.py").write_text('__version__ = "0.19.0"\n', encoding="utf-8")
_git(self.tree, "init", "-q")
_git(self.tree, "add", "-A")
_git(self.tree, "commit", "-q", "-m", "release fixture")
def commit(self, message: str) -> str:
(self.tree / "CHANGELOG.fixture").write_text(message + "\n", encoding="utf-8")
_git(self.tree, "add", "-A")
_git(self.tree, "commit", "-q", "-m", message)
return _git(self.tree, "rev-parse", "HEAD")
def head(self) -> str:
return _git(self.tree, "rev-parse", "HEAD")
def push_tag(self, tag: str = TAG, annotated: bool = True, at: Optional[str] = None) -> None:
target = at or "HEAD"
if annotated:
_git(self.tree, "tag", "-a", tag, target, "-m", f"Voculinux {tag}")
else:
_git(self.tree, "tag", tag, target)
_git(self.tree, "push", "-q", "origin", f"refs/tags/{tag}")
def stage(self, built_commit: str, tag: str = TAG) -> None:
"""Lay out publish/ the way build-release.sh writes it, for ``built_commit``."""
sdist = self.out / "sdist"
repo = self.out / "xbps" / "repo"
keys = self.out / "xbps" / "keys"
for directory in (sdist, repo, keys):
directory.mkdir(parents=True)
payload = b"canonical sdist bytes"
(sdist / SDIST).write_bytes(payload)
(self.out / "SDIST_SHA256").write_text(hashlib.sha256(payload).hexdigest() + "\n")
for name in _smoke_packages():
pkg = f"{name}-{XBPS}_1"
(repo / f"{pkg}.x86_64.xbps").write_bytes(b"package " + pkg.encode())
(repo / f"{pkg}.x86_64.xbps.sig2").write_bytes(b"signature " + pkg.encode())
(repo / "x86_64-repodata").write_bytes(b"signed index")
(keys / "FINGERPRINT").write_text(
f"signed-by: Voculinux <xbps@voculinux.com>\nfingerprint: {FINGERPRINT}\n",
encoding="utf-8",
)
shutil.copy2(PLIST, keys / f"{FINGERPRINT}.plist")
(self.out / "BUILT_FROM").write_text(f"tag={tag}\ncommit={built_commit}\n")
def staged_names(self) -> List[str]:
return sorted(
p.name
for sub in ("sdist", "xbps/repo", "xbps/keys")
for p in (self.out / sub).iterdir()
)
def run(
self,
*args: str,
tag: str = TAG,
token: Optional[str] = TOKEN,
tea_config: Optional[Path] = None,
fail: str = "",
) -> subprocess.CompletedProcess:
env = {
"PATH": f"{self.gitea.bin}{os.pathsep}{os.environ['PATH']}",
"HOME": str(self.home),
"TMPDIR": str(self.tmp),
"GITEA_HOST": HOST,
"GITEA_API": API,
"TEA_CONFIG": str(tea_config or self.home / "no-tea-config.yml"),
"FAKE_GITEA_STATE": str(self.gitea.state_file),
"FAKE_GITEA_BASE": API,
"FAKE_GITEA_FAIL": fail,
**GIT_ENV,
}
if token is not None:
env["GITEA_TOKEN"] = token
return subprocess.run(
["bash", str(self.tree / "packaging" / "void" / "publish-release.sh"), tag, *args],
cwd=self.tree,
env=env,
capture_output=True,
text=True,
timeout=60,
check=False,
)
def publish(self, *extra: str, **kwargs) -> subprocess.CompletedProcess:
return self.run("--out-dir", str(self.out), *extra, **kwargs)
@pytest.fixture()
def release(tmp_path: Path) -> Release:
if shutil.which("git") is None or shutil.which("bash") is None:
pytest.skip("git and bash are required")
return Release(tmp_path)
def _published(release: Release) -> Release:
"""A tree whose origin holds v0.19.0 at HEAD, and a staged out-dir for HEAD."""
release.push_tag()
release.stage(release.head())
return release
def _uploads(gitea: FakeGitea, tag: str) -> List[str]:
return gitea.uploads(tag)
# ---------------------------------------------------------------------------
# The gates: nothing is published unless origin's tag is the built commit
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("annotated", [True, False], ids=["annotated", "lightweight"])
def test_a_tag_at_the_built_commit_publishes_both_releases(
release: Release, annotated: bool
) -> None:
release.push_tag(annotated=annotated)
release.stage(release.head())
result = release.publish()
assert result.returncode == 0, result.stdout + result.stderr
assert "gates passed" in result.stdout
expected = release.staged_names()
assert SDIST in expected
for tag in (TAG, "latest"):
assert sorted(release.gitea.asset_names(tag)) == expected, tag
uploaded = _uploads(release.gitea, tag)
assert sorted(uploaded) == expected, f"{tag}: {uploaded}"
assert uploaded[-1] == "x86_64-repodata", "the index goes last, after its packages"
def test_a_tag_that_is_not_on_origin_is_refused(release: Release) -> None:
release.stage(release.head())
result = release.publish()
assert result.returncode != 0
assert f"tag {TAG} is not on origin" in result.stderr
assert not release.gitea.state["log"], "no request may be made for an unpublished tag"
def test_a_tag_that_only_shares_a_prefix_with_the_release_is_refused(release: Release) -> None:
"""origin holds v0.19.0x; the release is v0.19.0. A prefix match must not pass."""
_git(release.tree, "tag", "-a", "v0.19.0x", "-m", "other", "HEAD")
_git(release.tree, "push", "-q", "origin", "refs/tags/v0.19.0x")
release.stage(release.head())
result = release.publish()
assert result.returncode != 0
assert f"tag {TAG} is not on origin" in result.stderr
assert not release.gitea.state["log"]
def test_a_tag_on_another_commit_is_refused(release: Release) -> None:
built = release.head()
release.stage(built)
later = release.commit("a commit after the build")
release.push_tag(at=later)
result = release.publish()
assert result.returncode != 0
assert f"points at {later}, but" in result.stderr
assert f"built from {built}" in result.stderr
assert not release.gitea.state["log"]
def test_an_out_dir_without_a_build_record_is_refused(release: Release) -> None:
_published(release)
(release.out / "BUILT_FROM").unlink()
result = release.publish()
assert result.returncode != 0
assert "no BUILT_FROM record" in result.stderr
assert not release.gitea.state["log"]
def test_an_out_dir_built_for_another_tag_is_refused(release: Release) -> None:
release.push_tag()
release.stage(release.head(), tag="v0.18.5")
result = release.publish()
assert result.returncode != 0
assert "was built for 'v0.18.5'" in result.stderr
assert not release.gitea.state["log"]
def test_a_sdist_that_does_not_match_its_digest_is_refused(release: Release) -> None:
_published(release)
(release.out / "sdist" / SDIST).write_bytes(b"tampered after the build")
result = release.publish()
assert result.returncode != 0
assert "does not match SDIST_SHA256" in result.stderr
assert not release.gitea.state["log"]
def test_a_client_key_other_than_the_committed_plist_is_refused(release: Release) -> None:
_published(release)
(release.out / "xbps" / "keys" / f"{FINGERPRINT}.plist").write_text("<plist/>\n")
result = release.publish()
assert result.returncode != 0
assert "not the committed packaging/void/repository-key.plist" in result.stderr
assert not release.gitea.state["log"]
def test_a_repository_without_its_index_is_refused(release: Release) -> None:
_published(release)
(release.out / "xbps" / "repo" / "x86_64-repodata").unlink()
result = release.publish()
assert result.returncode != 0
assert "has no repository index" in result.stderr
assert not release.gitea.state["log"]
def test_a_tag_that_fails_the_tag_gate_is_refused(release: Release) -> None:
"""v0.19.0-beta is not the release: the tag gate refuses it before origin is read."""
_published(release)
result = release.publish(tag="v0.19.0-beta")
assert result.returncode != 0
assert "fails the tag gate" in result.stderr
assert not release.gitea.state["log"]
def test_usage_errors_exit_2(release: Release) -> None:
"""No --out-dir is a usage error, and so is a call with no tag at all."""
no_out_dir = release.run()
assert no_out_dir.returncode == 2, no_out_dir.stderr
assert "usage: publish-release.sh" in no_out_dir.stderr
bare = subprocess.run(
["bash", str(release.tree / "packaging" / "void" / "publish-release.sh")],
capture_output=True,
text=True,
check=False,
)
assert bare.returncode == 2, bare.stderr
# ---------------------------------------------------------------------------
# Dry run: every gate, the planned calls, and no request at all
# ---------------------------------------------------------------------------
def test_a_dry_run_plans_every_call_and_makes_none(release: Release) -> None:
_published(release)
result = release.publish("--dry-run")
assert result.returncode == 0, result.stdout + result.stderr
assert "dry run" in result.stdout
assert release.gitea.state["log"] == [], "a dry run must not reach the API"
lines = result.stdout.splitlines()
tag_section = lines.index(f"tag release {TAG} (commit {release.head()}):")
latest_section = lines.index("latest release (rolling, the registration URL serves it):")
assert tag_section < latest_section
for start, end in ((tag_section, latest_section), (latest_section, len(lines))):
uploads = [line for line in lines[start:end] if "/assets?name=" in line]
assert uploads, "a section plans its uploads"
assert uploads[-1].endswith("attachment=xbps/repo/x86_64-repodata"), uploads[-1]
assert f"POST {API}/releases only when {TAG} has no release" in result.stdout
def test_a_dry_run_never_prints_or_reads_a_token(release: Release) -> None:
_published(release)
result = release.publish("--dry-run", token=TOKEN)
assert result.returncode == 0, result.stdout + result.stderr
assert TOKEN not in result.stdout + result.stderr
assert "Authorization: token <redacted>" in result.stdout
def test_a_dry_run_needs_no_token_at_all(release: Release) -> None:
_published(release)
result = release.publish("--dry-run", token=None)
assert result.returncode == 0, result.stdout + result.stderr
def test_a_dry_run_still_refuses_a_tag_that_is_not_on_origin(release: Release) -> None:
release.stage(release.head())
result = release.publish("--dry-run")
assert result.returncode != 0
assert "is not on origin" in result.stderr
assert not release.gitea.state["log"]
# ---------------------------------------------------------------------------
# The publish itself, against the stateful stand-in
# ---------------------------------------------------------------------------
def test_the_latest_release_is_created_at_the_built_commit(release: Release) -> None:
_published(release)
assert release.publish().returncode == 0
built = release.head()
latest = release.gitea.release("latest")
assert latest["target_commitish"] == built
tag_release = release.gitea.release(TAG)
assert tag_release["target_commitish"] == built
assert not tag_release["draft"] and not latest["draft"]
assert TAG in latest["body"], "latest says which tag it was built from"
assert built in latest["body"]
def test_every_request_is_to_the_fork_api_and_carries_the_token(release: Release) -> None:
_published(release)
assert release.publish().returncode == 0
log = release.gitea.state["log"]
assert log
for entry in log:
assert entry["url"].startswith(API + "/"), entry["url"]
assert entry["auth"] == [f"Authorization: token {TOKEN}"], entry
posts = [e["data"]["tag_name"] for e in log if e["url"] == f"{API}/releases"]
assert posts == [TAG, "latest"], "the tag release is filled before latest is touched"
def test_the_notes_state_the_sdist_digest_exactly(release: Release) -> None:
_published(release)
assert release.publish().returncode == 0
digest = (release.out / "SDIST_SHA256").read_text(encoding="utf-8").strip()
assert f"sdist sha256: {digest}" in release.gitea.release("latest")["body"]
assert FINGERPRINT in release.gitea.release(TAG)["body"]
def test_a_rerun_replaces_same_named_assets_on_the_tag_release(release: Release) -> None:
_published(release)
release.gitea.seed_release(TAG, [SDIST, "x86_64-repodata"])
before = {a["name"]: a["id"] for a in release.gitea.release(TAG)["assets"]}
assert release.publish().returncode == 0
after = {a["name"]: a["id"] for a in release.gitea.release(TAG)["assets"]}
assert after[SDIST] != before[SDIST]
assert after["x86_64-repodata"] != before["x86_64-repodata"]
assert sorted(after) == release.staged_names()
def test_latest_replaces_every_asset_it_already_held(release: Release) -> None:
_published(release)
release.gitea.seed_release("latest", ["voculinux-0.18.5_1.x86_64.xbps", "stale.txt"])
assert release.publish().returncode == 0
assert sorted(release.gitea.asset_names("latest")) == release.staged_names()
assert (
len([r for r in release.gitea.state["releases"].values() if r["tag_name"] == "latest"]) == 1
)
def test_uploads_come_only_from_the_staged_set(release: Release) -> None:
_published(release)
assert release.publish().returncode == 0
staged = set(release.staged_names())
for tag in (TAG, "latest"):
assert set(_uploads(release.gitea, tag)) == staged
for entry in release.gitea.state["log"]:
for form in entry["forms"]:
path = re.match(r"attachment=@(.*?);", form).group(1)
assert path.startswith(("sdist/", "xbps/")), f"not from the staged set: {path}"
def test_the_colon_in_the_fingerprint_name_is_percent_encoded(release: Release) -> None:
_published(release)
assert release.publish().returncode == 0
plist_urls = [e["url"] for e in release.gitea.state["log"] if ".plist" in e["url"]]
assert plist_urls and all(":" not in u.split("name=")[1] for u in plist_urls), plist_urls
assert f"{FINGERPRINT}.plist" in release.gitea.asset_names("latest")
@pytest.mark.parametrize(
"fail",
[
"POST|/releases|403", # the token may not create releases
"POST|/assets?|500", # an upload fails midway
"DELETE|/assets/|500",
"GET|/releases/tags/|500",
"PATCH|/releases/|500",
],
)
def test_any_non_2xx_response_fails_the_publish(release: Release, fail: str) -> None:
_published(release)
release.gitea.seed_release("latest", ["stale"])
release.gitea.seed_release(TAG, ["x86_64-repodata"])
result = release.publish(fail=fail)
assert result.returncode != 0, f"{fail} must fail the publish"
assert "HTTP" in result.stderr or "failed" in result.stderr
def test_a_failed_tag_release_never_touches_latest(release: Release) -> None:
_published(release)
result = release.publish(fail="POST|/assets?|500")
assert result.returncode != 0
assert not any(r["tag_name"] == "latest" for r in release.gitea.state["releases"].values())
# ---------------------------------------------------------------------------
# The token: from GITEA_TOKEN or the tea login, never printed, never left behind
# ---------------------------------------------------------------------------
def _tea_config(path: Path, token: str) -> Path:
path.write_text(
"logins:\n"
" - name: other\n"
" url: https://example.invalid\n"
" token: token-for-another-host\n"
" default: true\n"
" - name: xavierk\n"
f" url: {HOST}\n"
f" token: {token}\n"
" default: false\n"
" user: xavierk\n"
"preferences:\n"
" editor: false\n",
encoding="utf-8",
)
return path
def test_the_token_comes_from_the_tea_login_for_this_host_and_is_never_printed(
release: Release,
) -> None:
_published(release)
config = _tea_config(release.root / "tea.yml", TEA_TOKEN)
result = release.publish(token=None, tea_config=config)
assert result.returncode == 0, result.stdout + result.stderr
assert TEA_TOKEN not in result.stdout + result.stderr
for entry in release.gitea.state["log"]:
assert entry["auth"] == [f"Authorization: token {TEA_TOKEN}"], entry
def test_no_token_and_no_tea_login_refuses_before_any_request(release: Release) -> None:
_published(release)
config = _tea_config(release.root / "tea.yml", TEA_TOKEN)
other = release.root / "other-host.yml"
other.write_text(config.read_text(encoding="utf-8").replace(HOST, "https://elsewhere.invalid"))
result = release.publish(token=None, tea_config=other)
assert result.returncode != 0
assert f"no tea login for {HOST}" in result.stderr
assert not release.gitea.state["log"]
def test_the_header_file_is_removed_when_the_publish_ends(release: Release) -> None:
_published(release)
assert release.publish().returncode == 0
assert list(release.tmp.glob("publish-release.*")) == []
def test_a_failed_publish_also_removes_the_header_file(release: Release) -> None:
_published(release)
assert release.publish(fail="POST|/releases|403").returncode != 0
assert list(release.tmp.glob("publish-release.*")) == []
def test_the_token_never_appears_in_the_output_of_a_failed_request(release: Release) -> None:
_published(release)
result = release.publish(fail="POST|/releases|403")
assert TOKEN not in result.stdout + result.stderr
# ---------------------------------------------------------------------------
# The script itself
# ---------------------------------------------------------------------------
def test_the_script_is_executable_bash_that_fails_closed() -> None:
text = PUBLISH_RELEASE.read_text(encoding="utf-8")
assert text.startswith("#!/usr/bin/env bash\n")
assert "set -euo pipefail" in text
assert os.access(PUBLISH_RELEASE, os.X_OK)
def test_the_script_never_enables_tracing_or_echoes_the_header() -> None:
code = "\n".join(
line
for line in PUBLISH_RELEASE.read_text(encoding="utf-8").splitlines()
if not line.lstrip().startswith("#")
)
assert "set -x" not in code
assert 'cat "$AUTH"' not in code
assert 'echo "$GITEA_TOKEN"' not in code and "printf '%s' \"$GITEA_TOKEN\"" not in code
def test_the_workflow_publish_step_and_this_script_name_the_same_api() -> None:
"""Two copies of the publish logic: they must publish to the same place."""
workflow = (REPO_ROOT / ".gitea" / "workflows" / "xbps-release.yml").read_text(encoding="utf-8")
assert "GITEA_API: https://git.bongbetic.com/api/v1/repos/xavierk/voculinux" in workflow
assert "https://git.bongbetic.com/api/v1/repos/xavierk/voculinux" in PUBLISH_RELEASE.read_text(
encoding="utf-8"
)
script = PUBLISH_RELEASE.read_text(encoding="utf-8")
assert "xbps/repo/*-repodata*" in workflow and "xbps/repo/*-repodata*" in script
+62
View File
@@ -90,6 +90,17 @@ def make_tree(
version_py if version_py is not None else f'"""Version."""\n\n__version__ = "{version}"\n', version_py if version_py is not None else f'"""Version."""\n\n__version__ = "{version}"\n',
encoding="utf-8", encoding="utf-8",
) )
# A release is built from a clean git checkout, so the fixture is one: committed,
# with nothing left uncommitted. Tests that want a dirty tree change it after this.
git = ["git", "-c", "user.name=fixture", "-c", "user.email=fixture@example.invalid"]
subprocess.run(["git", "init", "-q"], cwd=root, check=True, capture_output=True)
subprocess.run(["git", "add", "-A"], cwd=root, check=True, capture_output=True)
subprocess.run(
[*git, "-c", "commit.gpgsign=false", "commit", "-q", "-m", "fixture"],
cwd=root,
check=True,
capture_output=True,
)
return root return root
@@ -466,6 +477,7 @@ def test_the_checks_run_before_the_sdist_and_before_any_container() -> None:
pipeline = _body(text, "run_pipeline") pipeline = _body(text, "run_pipeline")
order = [ order = [
pipeline.index("check_template"), pipeline.index("check_template"),
pipeline.index("release_checkout"),
pipeline.index("check_key"), pipeline.index("check_key"),
pipeline.index("build_sdist"), pipeline.index("build_sdist"),
pipeline.index("phase_build"), pipeline.index("phase_build"),
@@ -473,6 +485,56 @@ def test_the_checks_run_before_the_sdist_and_before_any_container() -> None:
assert order == sorted(order), order assert order == sorted(order), order
def test_a_release_records_the_commit_it_is_built_from() -> None:
"""publish-release.sh refuses unless origin's tag peels to this commit."""
text = _code(BUILD_RELEASE)
pipeline = _body(text, "run_pipeline")
assert 'printf \'tag=%s\\ncommit=%s\\n\' "$TAG" "$SOURCE_COMMIT" >"$OUT/BUILT_FROM"' in pipeline
assert "rev-parse --verify HEAD" in _body(text, "release_checkout")
@pytest.fixture()
def no_container(tmp_path: Path) -> tuple:
"""A podman that must never run, with a log that proves it did not."""
bindir = tmp_path / "nocontainer-bin"
bindir.mkdir()
log = tmp_path / "nocontainer-calls.log"
script = bindir / "podman"
script.write_text(f'#!/bin/sh\necho "$@" >> "{log}"\nexit 97\n', encoding="utf-8")
script.chmod(0o755)
return f"{bindir}{os.pathsep}{os.environ['PATH']}", log
def test_a_release_refuses_an_uncommitted_checkout_before_anything_runs(
tmp_path: Path, no_container: tuple
) -> None:
path, log = no_container
tree = make_tree(tmp_path / "tree", "0.18.5", template_version="0.18.5", template_tag="v0.18.5")
(tree / "scratch.txt").write_text("not committed\n", encoding="utf-8")
out = tmp_path / "out"
tmp = tmp_path / "work"
tmp.mkdir()
result = run_build(tree, "release", "v0.18.5", "--out-dir", str(out), path=path, tmp=tmp)
assert result.returncode == 1, result.stdout + result.stderr
assert "uncommitted or untracked changes" in result.stderr
assert not log.exists() and not out.exists()
def test_a_release_refuses_a_tree_that_is_not_a_git_checkout(
tmp_path: Path, no_container: tuple
) -> None:
path, log = no_container
tree = make_tree(tmp_path / "tree", "0.18.5", template_version="0.18.5", template_tag="v0.18.5")
shutil.rmtree(tree / ".git")
out = tmp_path / "out"
tmp = tmp_path / "work"
tmp.mkdir()
result = run_build(tree, "release", "v0.18.5", "--out-dir", str(out), path=path, tmp=tmp)
assert result.returncode == 1, result.stdout + result.stderr
assert "is not one" in result.stderr
assert not log.exists() and not out.exists()
def test_the_key_is_never_echoed_or_copied() -> None: def test_the_key_is_never_echoed_or_copied() -> None:
text = _code(BUILD_RELEASE) text = _code(BUILD_RELEASE)
assert "set -x" not in text assert "set -x" not in text
+2 -2
View File
@@ -92,7 +92,7 @@ def _gitea_workflow_text() -> str:
def test_the_xbps_job_lives_in_the_gitea_workflow_not_the_github_release() -> None: def test_the_xbps_job_lives_in_the_gitea_workflow_not_the_github_release() -> None:
"""One signing path: the tag-triggered Gitea workflow runs build-release.sh.""" """One signing path: the manual Gitea workflow runs build-release.sh, and no tag starts it."""
github = _release_workflow_text() github = _release_workflow_text()
for marker in ( for marker in (
"build-xbps", "build-xbps",
@@ -108,7 +108,7 @@ def test_the_xbps_job_lives_in_the_gitea_workflow_not_the_github_release() -> No
assert SIGNING_SECRET in gitea, "the Gitea workflow must read the XBPS_SIGNING_KEY secret" assert SIGNING_SECRET in gitea, "the Gitea workflow must read the XBPS_SIGNING_KEY secret"
assert "podman" in gitea, "the Gitea workflow must run the pipeline with podman" assert "podman" in gitea, "the Gitea workflow must run the pipeline with podman"
workflow = yaml.safe_load(gitea) workflow = yaml.safe_load(gitea)
assert workflow[True] == {"push": {"tags": ["v*"]}}, "tag pushes are the only trigger" assert set(workflow[True]) == {"workflow_dispatch"}, "manual dispatch is the only trigger"
#: Registration URL every client snippet names: the rolling ``latest`` #: Registration URL every client snippet names: the rolling ``latest``
+18 -6
View File
@@ -1,7 +1,8 @@
"""Offline guards for the signed xbps release workflow (tickets #28 and #29). """Offline guards for the signed xbps release workflow (tickets #28 and #29).
Releasing the Void repository is a tag push on the fork's Gitea host. The Releasing the Void repository is a local maintainer run on an x86_64 Void host;
workflow ``.gitea/workflows/xbps-release.yml`` gates the tag, stages the the workflow ``.gitea/workflows/xbps-release.yml`` is manual dispatch only and is
kept for a future x86_64 host-mode runner. It gates the tag, stages the
signing key on tmpfs, runs ``packaging/void/build-release.sh`` (the one release signing key on tmpfs, runs ``packaging/void/build-release.sh`` (the one release
path), and publishes exactly what that script staged to the tag's release and path), and publishes exactly what that script staged to the tag's release and
to the rolling ``latest`` release the client registration URL serves. to the rolling ``latest`` release the client registration URL serves.
@@ -135,10 +136,16 @@ def _run_step(
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def test_triggers_only_on_version_tag_pushes() -> None: def test_is_manual_dispatch_only_and_never_runs_on_a_tag_push() -> None:
"""A tag push is a human act: no schedule, no dispatch, no PR trigger.""" """A tag push must not start the job: releases are built and published locally.
The only runner with the bongbetic-ci label is ARM64 and cannot build the
glibc x86_64 release, so a tag-triggered run would fail (or worse, publish a
partial set). No schedule, no push, no pull request trigger.
"""
triggers = _workflow()[True] # YAML 1.1 reads the bare key `on` as True triggers = _workflow()[True] # YAML 1.1 reads the bare key `on` as True
assert triggers == {"push": {"tags": ["v*"]}}, triggers assert set(triggers) == {"workflow_dispatch"}, triggers
assert "push" not in triggers and "schedule" not in triggers
def test_grants_only_contents_write() -> None: def test_grants_only_contents_write() -> None:
@@ -462,7 +469,12 @@ def main(argv):
elif arg == "-X": elif arg == "-X":
method, i = argv[i + 1], i + 2 method, i = argv[i + 1], i + 2
elif arg == "-H": elif arg == "-H":
headers.append(argv[i + 1]) value = argv[i + 1]
if value.startswith("@"):
# curl reads a header from a file, which keeps the token off argv.
with open(value[1:], encoding="utf-8") as handle:
value = handle.read().strip()
headers.append(value)
i += 2 i += 2
elif arg == "-d": elif arg == "-d":
data, i = argv[i + 1], i + 2 data, i = argv[i + 1], i + 2
+2 -2
View File
@@ -1,12 +1,12 @@
{ {
"name": "voculinux-website", "name": "voculinux-website",
"version": "0.18.5", "version": "0.19.0",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "voculinux-website", "name": "voculinux-website",
"version": "0.18.5", "version": "0.19.0",
"dependencies": { "dependencies": {
"@types/node": "^20.14.10", "@types/node": "^20.14.10",
"@types/react": "^18.3.3", "@types/react": "^18.3.3",
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "voculinux-website", "name": "voculinux-website",
"version": "0.18.5", "version": "0.19.0",
"private": true, "private": true,
"type": "module", "type": "module",
"scripts": { "scripts": {
+1 -1
View File
@@ -29,7 +29,7 @@ const homeJsonLd = [
}, },
description: description:
"Offline voice dictation and speech-to-text for Linux with whisper.cpp, Faster Whisper, Whisper, VOSK, and Parakeet.", "Offline voice dictation and speech-to-text for Linux with whisper.cpp, Faster Whisper, Whisper, VOSK, and Parakeet.",
softwareVersion: "0.18.5", softwareVersion: "0.19.0",
author: { author: {
"@type": "Person", "@type": "Person",
name: "Xavier Karma", name: "Xavier Karma",