chore(dev): widen uv range and harden Fedora/AppImage builds (#952)

* chore(deps): allow uv versions below 0.14

* fix(ci): align uv version and generated exports

* fix(dev): check Fedora build headers and AppImage GI bundle

Fail early with distro-specific native dependency hints before dev sync. Verify the AppImage's PyGObject installation and Python path after linuxdeploy, and report isolated GI import failures with actionable path diagnostics. Guard the CI uv pin and cover both preflights with tests.
This commit is contained in:
Sergey Silaev
2026-10-23 22:16:16 -07:00
committed by GitHub
parent 498d876627
commit b59797f1af
16 changed files with 616 additions and 29 deletions
+4
View File
@@ -135,6 +135,8 @@ jobs:
- name: Set up uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
# Keep export formatting aligned with the documented generation version.
version: "0.13.3"
enable-cache: true
# --only-group lint installs the linters without the project itself, so
@@ -185,6 +187,7 @@ jobs:
- name: Set up uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
version: "0.13.3"
enable-cache: true
python-version: ${{ matrix.python-version }}
@@ -251,6 +254,7 @@ jobs:
- name: Set up uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
version: "0.13.3"
# Caches uv's download cache, not .venv/, so the checkout below stays
# the fresh clone this job exists to exercise.
enable-cache: true
+6 -4
View File
@@ -34,15 +34,15 @@ Rules:
| Piece | Current |
|---|---|
| Python | `>=3.11` (`requires-python` in `pyproject.toml`; CI: 3.11–3.14). Floor, classifiers, CI matrix and `install.sh` must agree — `tests/test_python_version_policy.py` |
| GTK | GTK 3 via distro `python3-gi` (PyGObject). Never pip-install it |
| uv | `>=0.12,<0.13` (`[tool.uv]` in `pyproject.toml`). `uv.lock` is the source of truth; CI runs `uv sync --locked` / `uv run --locked`, which fails on drift |
| GTK | Runtime `venv/` uses distro `python3-gi` (PyGObject), never pip-install it; uv builds the locked PyGObject in the separate dev `.venv/` on supported distros |
| uv | `>=0.12,<0.14` (`[tool.uv]` in `pyproject.toml`). `uv.lock` is the source of truth; CI runs `uv sync --locked` / `uv run --locked`, which fails on drift |
| just | https://just.systems or distro package `just` |
| Format / lint / types | Black + isort (line length 100), flake8 (`E9,F63,F7,F82` only), mypy `src/` (targets 3.11 and runs as a CI lint gate). The tools live in the `lint` dependency group, not the `dev` extra, so CI can install them without building the project |
| Website | Next.js / TypeScript in `web/` — see `web/AGENTS.md` |
Two virtualenvs, on purpose:
- **`.venv/`** — uv's, for dev tooling. `just deps` syncs it; every other Python `just` recipe uses `uv run --no-sync` so it does not prune extras that `just deps-all` installed. No activation needed.
- **`.venv/`** — uv's, for dev tooling. `just deps` checks native headers before syncing it (on Fedora 40+, install `glib2-devel`, `cairo-gobject-devel`, `gobject-introspection-devel`, `libffi-devel` and the PortAudio headers); every other Python `just` recipe uses `uv run --no-sync` so it does not prune extras that `just deps-all` installed. No activation needed.
- **`venv/`** — what `install.sh` builds for the app itself, from the *system* Python so distro `gi` is importable.
Never run `install.sh` from an activated `.venv`: it drops an inherited `VIRTUAL_ENV` from `PATH` and picks `$SYSTEM_PYTHON` (default `/usr/bin/python3`) precisely because a venv built from uv's interpreter cannot see distro PyGObject.
@@ -115,11 +115,13 @@ Website: `web/AGENTS.md`, `web/PRODUCT.md`, `web/DESIGN.md`. Do not duplicate si
## Dependencies (uv)
Use **uv 0.13.3** for `just lock` and `just export-check`, matching CI. Other versions in the supported contributor range can order equivalent platform markers differently; export verification compares files byte-for-byte. Update the CI pins and regenerate exports together when changing the generation version.
`uv.lock` is authoritative. `just lock` regenerates it, the hash-pinned `requirements/*.txt` exports, and `packaging/flatpak/python3-dependencies.yaml`. The AppImage build installs from `runtime`/`vad` plus `appimage.txt`; `install.sh` consumes the runtime and selected extras with `--require-hashes --no-deps --no-build-isolation`, then installs the project with `--no-deps --no-build-isolation`. **Do not edit `requirements/*.txt` by hand**, nor the url/sha256 pairs in `packaging/flatpak/python3-dependencies.yaml`. Change `pyproject.toml` (or `requirements/whisper.in` for the Whisper engine, `requirements/appimage*.in` for the AppImage), run `just lock`, and commit the lock plus the exports with the manifest change.
| Constraint | Rule |
|---|---|
| PyGObject | Distro `python3-gi` only, via `--system-site-packages`. Pip install fails on Ubuntu 24.04 (`girepository-2.0`). uv-managed interpreters do not see distro `gi` unless the venv is created that way |
| PyGObject | The runtime installer uses distro `python3-gi` via `--system-site-packages`; do not pip-install it into `venv/`. The dev `.venv/` instead builds the locked version with uv when glib 2.80+ and native headers are available. uv-managed interpreters do not see distro `gi` unless the venv is created with `--system-site-packages` |
| `[vosk]` extra | Wheel-only on PyPI (no sdist). Never part of a source-buildable lock. `install.sh --engine=vosk` installs it |
| Whisper CPU torch | `requirements/whisper.txt` is compiled from `requirements/whisper.in`. Pin `torch` to a `+cpu` local version — PyPI CUDA wheels win resolution regardless of index order. OpenAI Whisper does not use `torchaudio`, and the CPU index can lag torch |
| pywhispercpp | Pinned in `install.sh` as `PYWHISPERCPP_VERSION` (keep in sync with `uv.lock`) |
+31 -10
View File
@@ -23,9 +23,8 @@ cd vocalinux
./install.sh --dev
```
This installs system dependencies, creates a venv from the system Python, installs the package in editable mode with dev tools, and runs the test suite.
`install.sh` always builds `venv/` from the system Python (`/usr/bin/python3`, or `$SYSTEM_PYTHON`), because distro PyGObject is only importable from that interpreter. It ignores an activated virtualenv, so you can run it from a shell that still has uv's `.venv` active. See [The two environments](#the-two-environments).
This installs system dependencies, creates `venv/` with an editable install and dev
tools, and runs the test suite. See [The two environments](#the-two-environments).
### Manual setup
@@ -33,20 +32,20 @@ This installs system dependencies, creates a venv from the system Python, instal
2. Install system dependencies (examples):
**Ubuntu 24.04+** (`just deps` pip-builds PyGObject 3.56 from the lock):
**Ubuntu 24.04+:**
```bash
sudo apt update
sudo apt install -y python3-pip python3-gi python3-gi-cairo \
gir1.2-gtk-3.0 gir1.2-ibus-1.0 libgirepository-2.0-dev libgirepository1.0-dev \
build-essential pkg-config libcairo2-dev \
build-essential pkg-config libcairo2-dev libffi-dev \
python3-dev portaudio19-dev python3-venv xdotool
```
**Debian 12** cannot pip-build PyGObject 3.56 (glib 2.74). Use `./install.sh --dev` for tests and running from source, then `venv/bin/pytest` / `venv/bin/python -m vocalinux.main --debug`.
**Debian 12 (installer environment only):**
```bash
sudo apt install -y python3-pip python3-gi python3-gi-cairo \
gir1.2-gtk-3.0 gir1.2-ibus-1.0 libgirepository1.0-dev libcairo2-dev \
build-essential pkg-config \
build-essential pkg-config libffi-dev \
python3-dev portaudio19-dev python3-venv xdotool
```
@@ -54,10 +53,18 @@ This installs system dependencies, creates a venv from the system Python, instal
```bash
sudo apt install -y python3-pip python3-gi python3-gi-cairo \
gir1.2-gtk-3.0 gir1.2-ibus-1.0 libgirepository-2.0-dev libcairo2-dev \
build-essential pkg-config \
build-essential pkg-config libffi-dev \
python3-dev portaudio19-dev python3-venv xdotool
```
**Fedora 40+:**
```bash
sudo dnf install -y python3-pip python3-gobject gtk3 ibus-devel \
gobject-introspection-devel glib2-devel python3-devel portaudio-devel \
cairo-devel cairo-gobject-devel libffi-devel python3-virtualenv pkg-config cmake \
gcc-c++ make xdotool
```
AppIndicator (system tray):
```bash
# Older Ubuntu:
@@ -66,6 +73,10 @@ This installs system dependencies, creates a venv from the system Python, instal
sudo apt install -y gir1.2-ayatanaappindicator3-0.1
```
**macOS (tooling only):** `brew install portaudio pkg-config`; use
`xcode-select --install` if a C compiler is missing. PyGObject/pycairo and their
header checks apply only on Linux, which is required to run the desktop app.
3. Create the environment and install:
```bash
@@ -73,7 +84,8 @@ This installs system dependencies, creates a venv from the system Python, instal
just deps
```
`just deps` compiles PyGObject from the lock into `.venv`. That needs `libgirepository-2.0-dev` (Ubuntu 24.04+). Debian 12 cannot build it; use Option 1 and the installer `venv/` instead.
Missing native build headers are reported with install commands before syncing.
For older Linux releases, use the installer environment described below.
4. Run:
@@ -98,7 +110,16 @@ The repository uses two virtual environments on purpose. Do not merge them:
| `.venv/` | `just deps` (uv) | pinned in `.python-version` | dev tooling: pytest, black, mypy, `just` recipes |
| `venv/` | `./install.sh` | the system Python | running the installed app, which needs distro PyGObject |
`gi` (PyGObject) is the reason. `just deps` / `uv sync` build it from source into `.venv/`, which works wherever glib 2.80+ and `libgirepository-2.0-dev` are present (Arch, Fedora, Ubuntu 24.04 with that package, CI). Debian 12 cannot build PyGObject 3.56; use `./install.sh --dev` and `venv/bin/pytest`. `install.sh` never reuses `.venv/`: it builds `venv/` from the system Python with `--system-site-packages`, and rebuilds it if another interpreter created it. Set `SYSTEM_PYTHON=/usr/bin/python3.12 ./install.sh` on systems that ship several system interpreters.
- On Linux, `.venv/` builds the locked PyGObject, requiring glib 2.80+ and the
headers above. On Debian 12 or Fedora 39 and older, use `./install.sh --dev`,
then `venv/bin/pytest` or `venv/bin/python -m vocalinux.main --debug` instead.
- `venv/` uses distro PyGObject via `--system-site-packages`. The installer ignores
an activated venv and rebuilds one created by the wrong interpreter. It uses
`/usr/bin/python3` unless overridden, e.g. `SYSTEM_PYTHON=/usr/bin/python3.12 ./install.sh`.
`just appimage` bundles its own Python/PyGObject in the pinned Ubuntu Docker image.
For GI failures, check `Checking bundled PyGObject after linuxdeploy` and the smoke
test's path diagnostics; installing `python3-gobject` on the host won't fix the bundle.
## Making changes
+16 -5
View File
@@ -5,8 +5,9 @@
#
# Two environments, deliberately separate:
# .venv/ dev tooling, built by uv from .python-version (3.13). `just deps`
# pip-builds PyGObject from the lock and needs libgirepository-2.0-dev
# (Ubuntu 24.04+). Debian 12 cannot build 3.56; use `just install-dev`.
# builds PyGObject from the lock and needs libgirepository-2.0-dev
# (Ubuntu 24.04+) or glib2-devel (Fedora 40+), plus cairo-gobject
# headers on Fedora. Debian 12 cannot build 3.56; use `just install-dev`.
# venv/ what `just install` creates, always from the system Python so the
# distro PyGObject package is importable.
# install.sh ignores an activated .venv and rebuilds
@@ -27,13 +28,23 @@ DEV_EXTRAS := "--extra dev --extra vad --group lint"
default:
@just --list
# Fail before `uv sync` when the headers pyaudio, pycairo and PyGObject need are
# absent. Without this, the first error is a C compiler traceback from inside a
# dependency build, which reads as a uv problem rather than a distro package.
# PyGObject/pycairo checks apply only on Linux, matching the dependency marker.
# An existing .venv/ with the platform's native packages skips the header check.
[no-exit-message]
[private]
_check_build_deps:
@python3 scripts/check_build_deps.py
# Create .venv/ and keep it matching the lock. --inexact is what makes this safe
# to run ahead of every recipe: a plain `uv sync` removes extraneous packages, so
# it would uninstall whatever `just deps-all` installed — the very thing the
# --no-sync flags below exist to prevent. `version` skips this; it reads one
# string out of version.py and needs a bare interpreter, not a sync.
[private]
_tooling:
_tooling: _check_build_deps
uv sync --inexact {{DEV_EXTRAS}}
# Install Vocalinux
@@ -45,11 +56,11 @@ install-dev:
./install.sh --dev
# Install development dependencies into .venv/ (dev + vad extras)
deps:
deps: _check_build_deps
uv sync {{DEV_EXTRAS}}
# Install every optional extra — whisper/vosk engines and docs (CUDA torch, multi-GB)
deps-all:
deps-all: _check_build_deps
uv sync --all-extras --group lint
# Run test suite
+52 -1
View File
@@ -596,6 +596,32 @@ export DEPLOY_GTK_VERSION=3
-d "$APPDIR/usr/share/applications/vocalinux.desktop" \
-i "$APPDIR/usr/share/icons/hicolor/scalable/apps/vocalinux.svg"
# The plugin can change the bundle after PyGObject was installed. Check the
# package and the version-agnostic Python path that AppRun and the GI smoke use
# before entering a new mount/user namespace; a missing module there is not a
# missing python3-gi package on the builder's host.
echo "== Checking bundled PyGObject after linuxdeploy =="
GI_SITE="$APPDIR/usr/lib/python${PY_VER}/site-packages"
if [ ! -f "$GI_SITE/gi/__init__.py" ]; then
echo "PyGObject is missing from $GI_SITE after linuxdeploy." >&2
echo "Check the earlier uv install of requirements/appimage.txt." >&2
exit 1
fi
if [ -d "$APPDIR/usr/lib/python3" ] && [ ! -L "$APPDIR/usr/lib/python3" ]; then
echo "Expected usr/lib/python3 to be a symlink to python${PY_VER}, not a directory." >&2
exit 1
fi
# Restore this link if linuxdeploy replaced it; a broken alias hides gi even
# though its versioned site-packages directory is present in the bundle.
ln -sfn "python${PY_VER}" "$APPDIR/usr/lib/python3"
if ! PYTHONHOME="$APPDIR/usr" \
PYTHONPATH="$APPDIR/usr/lib/python3:$APPDIR/usr/lib/python3/site-packages" \
LD_LIBRARY_PATH="$APPDIR/usr/lib" \
"$APPDIR/usr/bin/python3" -c 'import gi; print(" gi:", gi.__file__)'; then
echo "The bundled Python cannot import gi before the isolated smoke test." >&2
exit 1
fi
# linuxdeploy copies the host's full typelib set; keep those extras (transitive
# GIR deps vary by distro) and re-assert our required seed on top.
echo "== Ensuring required typelibs are present (keeping linuxdeploy extras) =="
@@ -715,7 +741,32 @@ smoke_gi_imports() {
export GI_TYPELIB_PATH='$appdir/usr/lib/girepository-1.0'
export LD_LIBRARY_PATH='$appdir/usr/lib'
'$appdir/usr/bin/python3' - <<'PY'
import gi
try:
import gi
except ModuleNotFoundError:
import os
import pathlib
import stat
import sys
print('Bundled gi is not importable inside the isolated smoke test.', file=sys.stderr)
print(f' python: {sys.executable}, uid: {os.geteuid()}', file=sys.stderr)
print(' PYTHONHOME:', os.environ.get('PYTHONHOME'), file=sys.stderr)
print(' PYTHONPATH:', os.environ.get('PYTHONPATH'), file=sys.stderr)
print(f' sys.path: {sys.path}', file=sys.stderr)
lib = pathlib.Path(sys.prefix) / 'lib'
versioned = lib / f'python{sys.version_info.major}.{sys.version_info.minor}'
for candidate in (lib / 'python3', versioned / 'site-packages' / 'gi',
lib / 'python3' / 'site-packages' / 'gi'):
try:
details = candidate.stat()
except OSError as exc:
print(f' {candidate}: {exc}', file=sys.stderr)
else:
print(f' {candidate}: mode={stat.S_IMODE(details.st_mode):o}, '
f'uid={details.st_uid}, accessible={os.access(candidate, os.R_OK | os.X_OK)}',
file=sys.stderr)
raise
gi.require_version('Gtk', '3.0')
from gi.repository import Gtk # noqa: F401
try:
+3 -3
View File
@@ -130,9 +130,9 @@ Issues = "https://github.com/VocaHQ/vocalinux/issues"
Changelog = "https://github.com/VocaHQ/vocalinux/releases"
[tool.uv]
# Lock/exports are generated with uv; keep the minor pinned so lockfile
# upgrades are deliberate (see `just lock`).
required-version = ">=0.12,<0.13"
# Contributors may use uv 0.12 or 0.13. CI and lock/export generation use
# uv 0.13.3 for reproducible export formatting (see `just lock`).
required-version = ">=0.12,<0.14"
[tool.setuptools]
# Resource dirs carry no .py files, but setuptools still treats them as packages.
+1 -1
View File
@@ -356,7 +356,7 @@ distlib==0.4.3 \
--hash=sha256:4b0ce306c966eb73bc3a7b6abad017c556dadd92c44701562cd528ac7fde4d5b \
--hash=sha256:f152097224a0ae24be5a0f6bae1b9359af82133bce63f98a95f86cae1aede9ed
# via virtualenv
evdev==1.9.3 ; 'linux' in sys_platform or sys_platform == 'linux' \
evdev==1.9.3 ; sys_platform == 'linux' or 'linux' in sys_platform \
--hash=sha256:2c140e01ac8437758fa23fe5c871397412461f42d421aa20241dc8fe8cfccbc9
# via
# pynput
+1 -1
View File
@@ -211,7 +211,7 @@ ctranslate2==4.8.2 \
--hash=sha256:f6f0b576c247984d3fc299a372ccc9319b668d6d25b0539f3c861beba15d0504 \
--hash=sha256:fedda669421a57f8164568ee86ea265727016988d310fcc2e24ca30e9ec457cd
# via faster-whisper
evdev==1.9.3 ; 'linux' in sys_platform or sys_platform == 'linux' \
evdev==1.9.3 ; sys_platform == 'linux' or 'linux' in sys_platform \
--hash=sha256:2c140e01ac8437758fa23fe5c871397412461f42d421aa20241dc8fe8cfccbc9
# via
# pynput
+1 -1
View File
@@ -150,7 +150,7 @@ colorama==0.4.6 ; sys_platform == 'win32' \
--hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \
--hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6
# via tqdm
evdev==1.9.3 ; 'linux' in sys_platform or sys_platform == 'linux' \
evdev==1.9.3 ; sys_platform == 'linux' or 'linux' in sys_platform \
--hash=sha256:2c140e01ac8437758fa23fe5c871397412461f42d421aa20241dc8fe8cfccbc9
# via
# pynput
+1 -1
View File
@@ -150,7 +150,7 @@ colorama==0.4.6 ; sys_platform == 'win32' \
--hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \
--hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6
# via tqdm
evdev==1.9.3 ; 'linux' in sys_platform or sys_platform == 'linux' \
evdev==1.9.3 ; sys_platform == 'linux' or 'linux' in sys_platform \
--hash=sha256:2c140e01ac8437758fa23fe5c871397412461f42d421aa20241dc8fe8cfccbc9
# via
# pynput
+1 -1
View File
@@ -150,7 +150,7 @@ colorama==0.4.6 ; sys_platform == 'win32' \
--hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \
--hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6
# via tqdm
evdev==1.9.3 ; 'linux' in sys_platform or sys_platform == 'linux' \
evdev==1.9.3 ; sys_platform == 'linux' or 'linux' in sys_platform \
--hash=sha256:2c140e01ac8437758fa23fe5c871397412461f42d421aa20241dc8fe8cfccbc9
# via
# pynput
+1 -1
View File
@@ -240,7 +240,7 @@ colorama==0.4.6 ; sys_platform == 'win32' \
--hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \
--hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6
# via tqdm
evdev==1.9.3 ; 'linux' in sys_platform or sys_platform == 'linux' \
evdev==1.9.3 ; sys_platform == 'linux' or 'linux' in sys_platform \
--hash=sha256:2c140e01ac8437758fa23fe5c871397412461f42d421aa20241dc8fe8cfccbc9
# via
# pynput
+212
View File
@@ -0,0 +1,212 @@
#!/usr/bin/env python3
"""Fail fast when the native headers `just deps` needs are missing.
`uv sync` builds pyaudio, pycairo and PyGObject from source on Linux. Without
their development headers it fails deep inside a C compiler invocation, which
reads as a uv or dependency problem even though the fix is a distro package.
This check runs before the sync and names what is missing.
"""
from __future__ import annotations
import shutil
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
VENV_PYTHON = ROOT / ".venv" / "bin" / "python"
# pkg-config modules the source builds query. Labels explain the module to a
# contributor who does not know which package owns it.
BUILD_MODULES = (("portaudio-2.0", "PortAudio headers (pyaudio)"),)
# PyGObject and its pycairo dependency are selected only on Linux by the
# sys_platform marker in pyproject.toml. They must not gate macOS tooling.
PYGOBJECT_BUILD_MODULES = (
("cairo", "cairo headers (pycairo)"),
("cairo-gobject", "cairo-gobject headers (PyGObject)"),
("girepository-2.0", "GObject Introspection headers (glib 2.80+)"),
("libffi", "libffi headers (PyGObject)"),
)
MIN_GIREPOSITORY_VERSION = "2.80"
APT_HINT = [
"Ubuntu 24.04+ / Debian 13+:",
" sudo apt install -y build-essential pkg-config libcairo2-dev portaudio19-dev \\",
" libgirepository-2.0-dev libffi-dev",
]
DEBIAN_12_NOTE = [
"",
"Debian 12 cannot build PyGObject 3.56 (glib is too old). Use ./install.sh --dev",
"there and run the suite with venv/bin/pytest.",
]
DNF_HINT = [
"Fedora 40+ / RHEL-based systems with glib 2.80+:",
" sudo dnf install -y gcc gcc-c++ make pkg-config cairo-devel \\",
" cairo-gobject-devel portaudio-devel glib2-devel \\",
" gobject-introspection-devel libffi-devel",
"If your system's glib is older than 2.80, use ./install.sh --dev instead.",
]
FEDORA_OLD_NOTE = [
"Fedora 39 and older cannot build PyGObject 3.56 (glib is too old).",
"Use ./install.sh --dev there and run the suite with venv/bin/pytest.",
]
PACMAN_HINT = [
"Arch / Manjaro:",
" sudo pacman -S --needed base-devel pkgconf cairo portaudio gobject-introspection",
]
ZYPPER_HINT = [
"openSUSE:",
" sudo zypper install -y gcc gcc-c++ make pkg-config cairo-devel \\",
" portaudio-devel gobject-introspection-devel libffi-devel",
]
GENERIC_HINT = [
"Install the development packages for PortAudio, cairo, GObject Introspection",
"(glib 2.80+) and libffi, plus a C compiler and pkg-config.",
]
MACOS_HINT = [
"macOS (Homebrew):",
" brew install portaudio pkg-config",
"Install the Xcode Command Line Tools if a C compiler is missing:",
" xcode-select --install",
]
def read_os_release(path: Path = Path("/etc/os-release")) -> dict[str, str]:
"""Parse os-release without executing it as shell."""
values: dict[str, str] = {}
try:
lines = path.read_text(encoding="utf-8").splitlines()
except OSError:
return values
for line in lines:
key, separator, value = line.partition("=")
if not separator or not key or key.startswith("#"):
continue
values[key] = value.strip().strip('"').strip("'")
return values
def install_hint(release: dict[str, str]) -> list[str]:
"""Return distro-specific install commands for the missing build headers."""
distro = release.get("ID", "").lower()
like = release.get("ID_LIKE", "").lower()
if distro == "ubuntu" or "ubuntu" in like:
return APT_HINT
if distro == "debian" or "debian" in like:
return [*APT_HINT, *DEBIAN_12_NOTE]
if distro in {"fedora", "rhel", "centos", "rocky", "almalinux"} or any(
family in like for family in ("fedora", "rhel")
):
if distro == "fedora" and release.get("VERSION_ID", "").isdigit():
if int(release["VERSION_ID"]) < 40:
return FEDORA_OLD_NOTE
return DNF_HINT
if distro in {"arch", "manjaro", "endeavouros"} or "arch" in like:
return PACMAN_HINT
if distro == "opensuse" or distro.startswith("opensuse-") or "suse" in like:
return ZYPPER_HINT
return GENERIC_HINT
def venv_has_compiled_packages() -> bool:
"""True when `.venv` has the native packages selected for this platform."""
if not VENV_PYTHON.is_file():
return False
packages = ("pyaudio",)
if sys.platform == "linux":
packages += ("pycairo", "pygobject")
check = "import importlib.metadata as metadata; " + "; ".join(
f"metadata.version({package!r})" for package in packages
)
result = subprocess.run(
[str(VENV_PYTHON), "-c", check],
capture_output=True,
check=False,
)
return result.returncode == 0
def missing_build_dependencies() -> list[str]:
"""Return the compiler or pkg-config modules uv cannot find, if any."""
missing: list[str] = []
if shutil.which("pkg-config") is None:
missing.append("pkg-config")
if not any(shutil.which(compiler) for compiler in ("cc", "gcc", "clang")):
missing.append("a C compiler")
# Module probes need pkg-config; report the missing tool instead of a list
# of modules the user cannot check yet.
if missing:
return missing
modules = BUILD_MODULES
if sys.platform == "linux":
modules += PYGOBJECT_BUILD_MODULES
for module, description in modules:
# PyGObject 3.56 requires girepository-2.0 >= 2.80; finding an older
# .pc file is not enough to build it successfully.
args = ["pkg-config", "--exists", module]
if module == "girepository-2.0":
args = ["pkg-config", f"--atleast-version={MIN_GIREPOSITORY_VERSION}", module]
result = subprocess.run(
args,
capture_output=True,
check=False,
)
if result.returncode != 0:
missing.append(description)
return missing
def main() -> int:
"""Print an actionable dependency report when the build cannot succeed."""
if venv_has_compiled_packages():
return 0
missing = missing_build_dependencies()
if not missing:
return 0
print("error: missing native build dependencies for .venv/", file=sys.stderr)
print(file=sys.stderr)
print(
"`just deps` builds native Python packages from source, so uv needs their",
file=sys.stderr,
)
print("development headers before it can create the environment.", file=sys.stderr)
print(file=sys.stderr)
print("Missing:", file=sys.stderr)
for item in missing:
print(f" - {item}", file=sys.stderr)
print(file=sys.stderr)
print("Install them for your system:", file=sys.stderr)
hints = MACOS_HINT if sys.platform == "darwin" else install_hint(read_os_release())
for line in hints:
print(line, file=sys.stderr)
print(file=sys.stderr)
print(
"See the Manual setup section in CONTRIBUTING.md for the full package list.",
file=sys.stderr,
)
return 1
if __name__ == "__main__":
raise SystemExit(main())
+15
View File
@@ -142,6 +142,21 @@ def test_appimage_build_smokes_gi_without_host_typelibs():
assert "unshare --user --mount" in text
def test_appimage_build_checks_gi_after_linuxdeploy() -> None:
"""The GTK plugin must not silently remove gi or the path AppRun imports it from."""
text = BUILD_SH.read_text(encoding="utf-8")
deploy = text.index('"$TOOLDIR/linuxdeploy" --appdir')
check = text.index("== Checking bundled PyGObject after linuxdeploy ==", deploy)
smoke = text.index('smoke_gi_imports "$APPDIR"', check)
between = text[check:smoke]
assert 'GI_SITE="$APPDIR/usr/lib/python${PY_VER}/site-packages"' in between
assert '"$GI_SITE/gi/__init__.py"' in between
assert 'ln -sfn "python${PY_VER}" "$APPDIR/usr/lib/python3"' in between
assert "'import gi; print(" in between
assert "sys.path:" in text, "isolated GI failures should report where Python looked"
def test_appimage_build_rebuilds_pywhispercpp_with_vulkan():
text = BUILD_SH.read_text()
assert "GGML_VULKAN=1" in text
+250
View File
@@ -0,0 +1,250 @@
"""Guards for the `just deps` native build-dependency preflight."""
import importlib.metadata
import importlib.util
import subprocess
from pathlib import Path
from typing import Any
import pytest
SCRIPT = Path(__file__).resolve().parents[1] / "scripts" / "check_build_deps.py"
SPEC = importlib.util.spec_from_file_location("check_build_deps", SCRIPT)
assert SPEC is not None and SPEC.loader is not None
PREFLIGHT: Any = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(PREFLIGHT)
@pytest.fixture(autouse=True)
def linux_platform(monkeypatch: pytest.MonkeyPatch) -> None:
"""Keep Linux checks covered even when contributors run the suite on macOS."""
monkeypatch.setattr(PREFLIGHT.sys, "platform", "linux")
def test_os_release_parser_strips_quotes_and_ignores_comments(tmp_path: Path) -> None:
"""os-release is parsed, not sourced; quoted values lose their quotes."""
release = tmp_path / "os-release"
release.write_text(
'ID=fedora\nID_LIKE="rhel centos"\n# comment\nVERSION_ID="45"\n',
encoding="utf-8",
)
assert PREFLIGHT.read_os_release(release) == {
"ID": "fedora",
"ID_LIKE": "rhel centos",
"VERSION_ID": "45",
}
@pytest.mark.parametrize(
"distro,like,expected",
[
("ubuntu", "debian", "sudo apt install"),
("fedora", "rhel", "sudo dnf install"),
("rocky", "rhel fedora", "sudo dnf install"),
("arch", "", "sudo pacman -S"),
("opensuse-tumbleweed", "suse", "sudo zypper install"),
],
)
def test_install_hint_matches_the_distro_family(distro: str, like: str, expected: str) -> None:
"""The failure names the package manager the contributor actually has."""
hint = "\n".join(PREFLIGHT.install_hint({"ID": distro, "ID_LIKE": like}))
assert expected in hint
def test_debian_hint_points_older_releases_at_the_installer() -> None:
"""Debian 12 cannot build PyGObject 3.56, so the hint says what works."""
hint = "\n".join(PREFLIGHT.install_hint({"ID": "debian", "ID_LIKE": ""}))
assert "sudo apt install" in hint
assert "./install.sh --dev" in hint
def test_fedora_hint_keeps_the_split_cairo_gobject_package() -> None:
"""Fedora splits cairo-gobject.pc and girepository-2.0.pc across packages."""
hint = "\n".join(PREFLIGHT.install_hint({"ID": "fedora", "ID_LIKE": ""}))
assert "cairo-gobject-devel" in hint
assert "glib2-devel" in hint
def test_old_fedora_hint_points_at_the_installer() -> None:
"""Installing more headers cannot make Fedora 39's glib new enough."""
hint = "\n".join(PREFLIGHT.install_hint({"ID": "fedora", "VERSION_ID": "39"}))
assert "./install.sh --dev" in hint
assert "sudo dnf install" not in hint
def test_unknown_distro_gets_a_generic_hint() -> None:
"""No false package-manager command when the family is unknown."""
hint = "\n".join(PREFLIGHT.install_hint({"ID": "plan9", "ID_LIKE": ""}))
assert "development packages" in hint
assert "sudo " not in hint
def test_missing_pkg_config_is_reported_without_probing_modules(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The tool itself missing is the first actionable failure."""
monkeypatch.setattr(
PREFLIGHT.shutil,
"which",
lambda name: "/usr/bin/cc" if name == "cc" else None,
)
def fail_run(*args: Any, **kwargs: Any) -> Any:
raise AssertionError("pkg-config cannot be queried when it is absent")
monkeypatch.setattr(PREFLIGHT.subprocess, "run", fail_run)
assert PREFLIGHT.missing_build_dependencies() == ["pkg-config"]
def test_missing_modules_keep_their_explanations(monkeypatch: pytest.MonkeyPatch) -> None:
"""Every absent pkg-config module is reported with the package it comes from."""
monkeypatch.setattr(PREFLIGHT.shutil, "which", lambda name: f"/usr/bin/{name}")
class Result:
def __init__(self, returncode: int) -> None:
self.returncode = returncode
def fake_run(args: list[str], **kwargs: Any) -> Result:
return Result(0 if args[-1] == "portaudio-2.0" else 1)
monkeypatch.setattr(PREFLIGHT.subprocess, "run", fake_run)
missing = PREFLIGHT.missing_build_dependencies()
assert any("cairo headers" in item for item in missing)
assert any("libffi" in item for item in missing)
assert not any("portaudio" in item.lower() for item in missing)
def test_girepository_requires_glib_280(monkeypatch: pytest.MonkeyPatch) -> None:
"""A too-old girepository-2.0.pc must not pass the preflight."""
monkeypatch.setattr(PREFLIGHT.shutil, "which", lambda name: f"/usr/bin/{name}")
checked: list[list[str]] = []
class Result:
def __init__(self, returncode: int) -> None:
self.returncode = returncode
def fake_run(args: list[str], **kwargs: Any) -> Result:
checked.append(args)
return Result(1 if args[-1] == "girepository-2.0" else 0)
monkeypatch.setattr(PREFLIGHT.subprocess, "run", fake_run)
assert PREFLIGHT.missing_build_dependencies() == ["GObject Introspection headers (glib 2.80+)"]
assert ["pkg-config", "--atleast-version=2.80", "girepository-2.0"] in checked
def test_fresh_macos_environment_does_not_need_gi_headers(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""PyAudio's build prerequisites suffice when the lock excludes PyGObject."""
monkeypatch.setattr(PREFLIGHT.sys, "platform", "darwin")
monkeypatch.setattr(PREFLIGHT, "VENV_PYTHON", tmp_path / "absent-python")
monkeypatch.setattr(PREFLIGHT.shutil, "which", lambda name: f"/usr/bin/{name}")
checked: list[str] = []
def fake_run(args: list[str], **kwargs: Any) -> subprocess.CompletedProcess[str]:
checked.append(args[-1])
return subprocess.CompletedProcess(args, 0 if args[-1] == "portaudio-2.0" else 1)
monkeypatch.setattr(PREFLIGHT.subprocess, "run", fake_run)
assert PREFLIGHT.main() == 0
assert checked == ["portaudio-2.0"]
@pytest.mark.parametrize(
"platform,installed,expected",
[
("darwin", {"pyaudio"}, True),
("darwin", set(), False),
("linux", {"pyaudio"}, False),
("linux", {"pyaudio", "pygobject"}, False),
("linux", {"pyaudio", "pycairo"}, False),
("linux", {"pyaudio", "pycairo", "pygobject"}, True),
],
)
def test_venv_probe_matches_platform_dependencies(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
platform: str,
installed: set[str],
expected: bool,
) -> None:
"""Run the metadata probe against complete and partial per-platform environments."""
monkeypatch.setattr(PREFLIGHT.sys, "platform", platform)
python = tmp_path / "python"
python.touch()
monkeypatch.setattr(PREFLIGHT, "VENV_PYTHON", python)
def version(package: str) -> str:
if package not in installed:
raise importlib.metadata.PackageNotFoundError(package)
return "1.0"
def fake_run(args: list[str], **kwargs: Any) -> subprocess.CompletedProcess[str]:
assert args[:2] == [str(python), "-c"]
try:
exec(args[2], {})
except importlib.metadata.PackageNotFoundError:
return subprocess.CompletedProcess(args, 1)
return subprocess.CompletedProcess(args, 0)
monkeypatch.setattr(importlib.metadata, "version", version)
monkeypatch.setattr(PREFLIGHT.subprocess, "run", fake_run)
assert PREFLIGHT.venv_has_compiled_packages() is expected
def test_macos_missing_portaudio_has_no_gi_install_hint(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""Failure guidance should request only packages macOS tooling actually needs."""
monkeypatch.setattr(PREFLIGHT.sys, "platform", "darwin")
monkeypatch.setattr(PREFLIGHT, "VENV_PYTHON", tmp_path / "absent-python")
monkeypatch.setattr(PREFLIGHT.shutil, "which", lambda name: f"/usr/bin/{name}")
monkeypatch.setattr(
PREFLIGHT.subprocess, "run", lambda args, **kwargs: subprocess.CompletedProcess(args, 1)
)
assert PREFLIGHT.main() == 1
error = capsys.readouterr().err
assert "PortAudio headers" in error
assert "brew install portaudio pkg-config" in error
assert "xcode-select --install" in error
assert "PyGObject" not in error
assert "cairo" not in error
assert "libffi" not in error
def test_existing_environment_short_circuits_the_check(monkeypatch: pytest.MonkeyPatch) -> None:
"""An environment that already holds its native packages needs no header check."""
monkeypatch.setattr(PREFLIGHT, "venv_has_compiled_packages", lambda: True)
monkeypatch.setattr(
PREFLIGHT,
"missing_build_dependencies",
lambda: pytest.fail("the preflight should not inspect headers"),
)
assert PREFLIGHT.main() == 0
def test_missing_dependencies_fail_with_install_commands(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""The default failure is the actionable report, not a compiler traceback."""
monkeypatch.setattr(PREFLIGHT, "venv_has_compiled_packages", lambda: False)
monkeypatch.setattr(
PREFLIGHT, "missing_build_dependencies", lambda: ["PortAudio headers (pyaudio)"]
)
monkeypatch.setattr(PREFLIGHT, "read_os_release", lambda: {"ID": "fedora"})
assert PREFLIGHT.main() == 1
error = capsys.readouterr().err
assert "missing native build dependencies" in error
assert "PortAudio headers (pyaudio)" in error
assert "sudo dnf install" in error
assert "CONTRIBUTING.md" in error
+21
View File
@@ -39,6 +39,27 @@ def _pyproject() -> dict[str, Any]:
return tomllib.load(handle)
def test_ci_uv_pin_is_supported_and_consistent() -> None:
"""All CI jobs must use one supported uv to reproduce the committed exports."""
required = _pyproject()["tool"]["uv"]["required-version"]
bounds = re.fullmatch(r">=(\d+\.\d+),<(\d+\.\d+)", required)
assert bounds, f"unexpected uv required-version: {required!r}"
workflow = WORKFLOW.read_text(encoding="utf-8")
steps = re.findall(r"- name: Set up uv\n(.*?)(?=\n\s+- name:|\Z)", workflow, re.DOTALL)
assert steps and len(steps) == workflow.count("uses: astral-sh/setup-uv@")
pins = []
for step in steps:
pin = re.search(r'^\s+version: "(\d+\.\d+\.\d+)"$', step, re.MULTILINE)
assert pin, "setup-uv in CI must explicitly pin the export generation version"
pins.append(pin.group(1))
assert len(set(pins)) == 1, "uv versions in CI disagree; exports are byte-for-byte checked"
minimum, maximum = (tuple(map(int, bound.split("."))) for bound in bounds.groups())
minor = tuple(map(int, pins[0].split(".")[:2]))
assert minimum <= minor < maximum
def _requirement_names(specs: list[str]) -> list[str]:
"""Distribution names out of PEP 508 specifiers, lowercased."""
return [re.split(r"[<>=!~\[; ]", spec, maxsplit=1)[0].strip().lower() for spec in specs]