fix(void): report pywhispercpp 1.5.0 and load CPU variants at runtime
CI / security (pull_request) Successful in 1m11s
CI / lint (pull_request) Successful in 25s
CI / ai-review (pull_request) Successful in 8s

python3-pywhispercpp was installed as 0.0.0. setup.py passes no version to
setup() and takes it from setuptools-scm, which the Void build did not
install; the template's PYWHISPERCPP_VERSION export is never read. The app
read 0.0.0 as an old binding and skipped GPU device selection. Install
setuptools-scm, pin the version with SETUPTOOLS_SCM_PRETEND_VERSION, and
make the app treat a 0.0.0 distribution as unknown so the runtime feature
probe decides.

The CPU backend was built with every SIMD option off, so the real-time
factor on an AVX2 host was about 1.7. Build with GGML_BACKEND_DL and
GGML_CPU_ALL_VARIANTS instead: one CPU library per instruction-set level,
with the x86-64 baseline always shipped. The binding loads the variants
from its own directory (patches/), and pre_install places them beside the
extension. The app's library preloader skips those variants: they export
identical symbols, so preloading them RTLD_GLOBAL would let one variant
serve every other. The install smoke asserts that the installed package
reports 1.5.0, that the baseline variant holds no AVX code, that the AVX2
variant does, and that the CPU and Vulkan backends load from the package.

Fixes #43
Fixes #44
This commit is contained in:
2026-11-07 00:03:22 +05:30
parent a4a831c3c6
commit 97f93e9492
11 changed files with 376 additions and 66 deletions
+27 -10
View File
@@ -88,19 +88,35 @@ for the sdist; `tests/test_void_pywhispercpp_template.py` fails otherwise.
There is no CUDA build: Void ships no CUDA toolkit package. Rebuilding
locally with a hand-installed toolkit (`GGML_CUDA=1`) is a documented,
unsupported escape hatch.
- **Portable CPU flags**: every ggml SIMD option is pinned off
(`-DGGML_NATIVE=OFF -DGGML_SSE42=OFF ...`), the plain x86-64 baseline
already used by the Flatpak (`#938`, `#957`). The install smoke scans the
shipped `libggml-cpu` (`.GCC.command.line`) for `-march=native`,
`-mcpu=native` and `-mtune=native` and fails on a match. The scan is a
tripwire, not a proof: the proof is the pinned configuration.
- **CPU variants (`#44`)**: `GGML_BACKEND_DL=ON` and
`GGML_CPU_ALL_VARIANTS=ON` build one CPU library per instruction-set
level (`libggml-cpu-x64.so` for the plain x86-64 baseline, then
`haswell`, `skylakex`, `zen4`, ...). ggml loads the best one the host
supports, so a host with AVX2 runs AVX2 code, and the baseline variant
ships everywhere. `GGML_NATIVE` stays `OFF`: ggml refuses it with
`GGML_BACKEND_DL`, and it keeps `-march=native` out of every variant
(`#938`). `patches/load-ggml-backends-beside-module.patch` makes the
extension load the variants from its own directory: ggml's default search
covers the interpreter's directory and the working directory, and neither
holds them, so without the patch no CPU backend is registered. The install
smoke checks that `libggml-cpu-x64.so` holds no AVX (`ymm`/`zmm`) code,
that `libggml-cpu-haswell.so` does, that no variant carries host-native
tuning (`.GCC.command.line`), and that the CPU and Vulkan backends load
from the installed package.
- **Version (`#43`)**: `setup.py` passes no version to `setup()`. The
version comes from setuptools-scm, which the sdist declares as a build
requirement, so `hostmakedepends` provides `python3-setuptools_scm` and
`SETUPTOOLS_SCM_PRETEND_VERSION` pins `1.5.0` without git metadata. Without
setuptools-scm, setuptools writes `0.0.0`, which the app reads as an
unknown binding version and then skips device selection. The install smoke
asserts the installed package reports `1.5.0`.
- **Shared libraries**: `BUILD_SHARED_LIBS=ON` with an `$ORIGIN` rpath
(as `install.sh` does), so the extension resolves the whisper.cpp
libraries installed beside it with no `LD_LIBRARY_PATH`.
- **Offline build**: the sdist vendors whisper.cpp, pybind11, and the
ggml sources; the build fetches nothing beyond the declared distfiles.
`PYWHISPERCPP_VERSION` is injected because the sdist carries no
`version.txt` (its version otherwise comes from git metadata).
The sdist carries no git metadata, which is why the version is pinned
through setuptools-scm (see Version above).
## voculinux
@@ -202,8 +218,9 @@ host Void install and nothing is published. Stages, in order, each fails the run
`python3-pysocks` are installed, then `voculinux` installs and upgrades both
to the carried versions (see [Version floors](#version-floors)),
`voculinux --version` shows the version, the payload checks pass (desktop
entry, icon, marker, no models, no pre-rename names, the Vulkan backend, no
host-native CPU tuning), and the imports load (`pynput` through its dummy
entry, icon, marker, no models, no pre-rename names, the Vulkan backend, the
CPU variants with an AVX-free x86-64 baseline, no host-native CPU tuning),
the installed pywhispercpp reports `1.5.0`, and the imports load (`pynput` through its dummy
backend, `evdev`, `gi` with Gtk 3, `pywhispercpp.model`, `pyaudio`, `socks`,
and the app's own modules, which keep their retained internal name).
+76 -9
View File
@@ -106,19 +106,46 @@ payload_voculinux() {
echo "payload ok: desktop entry, icon and marker present; no models, wheels or pre-rename names"
}
# whisper.cpp must ship its Vulkan backend and no host-native CPU tuning.
# whisper.cpp ships its Vulkan backend and its CPU backend as runtime-loaded
# variants beside the extension (GGML_BACKEND_DL, #44). Every CPU variant
# carries its own instruction set, but the x86-64 baseline variant must hold no
# AVX code (ymm/zmm), the AVX2 variant must hold some (so the variants are real,
# not one baseline build), and every other shipped native library (the ggml
# base, the registry, whisper, the extension, Vulkan) must stay AVX-free. No
# library may carry host-native tuning.
payload_pywhispercpp() {
PKGFILE=$1
tar -tf "$PKGFILE" | grep -q "libggml-vulkan" || fail "the Vulkan backend is missing from $PKGFILE"
LIBMEMBER=$(tar -tf "$PKGFILE" | grep -E "libggml-cpu\.so" | head -n1)
[ -n "$LIBMEMBER" ] || fail "libggml-cpu is missing from $PKGFILE"
mkdir -p /flagscan
tar -xf "$PKGFILE" -C /flagscan "$LIBMEMBER"
LIBSCAN="/flagscan/${LIBMEMBER#./}"
if readelf -p .GCC.command.line "$LIBSCAN" 2>/dev/null | grep -E "march=native|mcpu=native|mtune=native"; then
fail "host-native CPU tuning in the shipped libggml-cpu"
for VARIANT in x64 haswell; do
tar -tf "$PKGFILE" | grep -q "libggml-cpu-$VARIANT\.so" ||
fail "CPU variant libggml-cpu-$VARIANT.so is missing from $PKGFILE"
done
rm -rf /flagscan && mkdir -p /flagscan
tar -xf "$PKGFILE" -C /flagscan --wildcards '*.so*'
while IFS= read -r LIBSCAN; do
NAME=${LIBSCAN##*/}
if readelf -p .GCC.command.line "$LIBSCAN" 2>/dev/null | grep -E "march=native|mcpu=native|mtune=native"; then
fail "host-native CPU tuning in $NAME"
fi
case "$NAME" in
libggml-cpu-*) ;; # each variant is checked below
*)
if objdump -d "$LIBSCAN" | grep -Eq '%[yz]mm[0-9]'; then
fail "$NAME contains AVX code; only the CPU variants may"
fi
;;
esac
done <<LIBS
$(find /flagscan -type f \( -name '*.so' -o -name '*.so.*' \))
LIBS
BASELINE=$(find /flagscan -type f -name 'libggml-cpu-x64.so')
AVX2=$(find /flagscan -type f -name 'libggml-cpu-haswell.so')
[ -n "$BASELINE" ] && [ -n "$AVX2" ] || fail "the CPU variants were not extracted from $PKGFILE"
if objdump -d "$BASELINE" | grep -Eq '%[yz]mm[0-9]'; then
fail "the x86-64 baseline CPU variant contains AVX code; it would not run on every x86-64 CPU"
fi
echo "payload ok: Vulkan backend present; no host-native tuning in libggml-cpu"
objdump -d "$AVX2" | grep -Eq '%ymm[0-9]' || fail "the AVX2 CPU variant has no AVX2 code"
echo "payload ok: Vulkan backend; CPU variants present; baseline and other libraries free of AVX; no host-native tuning"
}
phase_install() {
@@ -179,6 +206,46 @@ phase_install() {
python3 -c "import evdev; print('evdev ok')"
python3 -c "import gi; gi.require_version('Gtk', '3.0'); from gi.repository import Gtk; print('gi ok: Gtk', Gtk.get_major_version(), Gtk.get_minor_version())"
python3 -c "from pywhispercpp.model import Model; print('pywhispercpp.model ok')"
echo "== pywhispercpp reports the version it is pinned to (#43)"
PWC_VERSION=$(python3 -c "import importlib.metadata as m; print(m.version('pywhispercpp'))") ||
fail "pywhispercpp has no installed version"
[ "$PWC_VERSION" = "1.5.0" ] || fail "pywhispercpp reports $PWC_VERSION, not 1.5.0 (#43)"
echo "pywhispercpp version: $PWC_VERSION"
echo "== the extension loads its CPU and Vulkan backends from the package (#44)"
python3 - <<'PY' || fail "the dynamic ggml backends did not load from the package"
import ctypes
import os
import _pywhispercpp
import pywhispercpp.model # noqa: F401 (importing the extension loads the backends)
libdir = os.path.dirname(_pywhispercpp.__file__)
base = ctypes.CDLL(os.path.join(libdir, "libggml.so.0"))
base.ggml_backend_reg_count.restype = ctypes.c_size_t
base.ggml_backend_reg_get.argtypes = [ctypes.c_size_t]
base.ggml_backend_reg_get.restype = ctypes.c_void_p
base.ggml_backend_reg_name.argtypes = [ctypes.c_void_p]
base.ggml_backend_reg_name.restype = ctypes.c_char_p
names = [
base.ggml_backend_reg_name(base.ggml_backend_reg_get(i)).decode()
for i in range(base.ggml_backend_reg_count())
]
print("ggml backends registered:", ", ".join(names))
assert "CPU" in names, names
loaded = set()
for line in open("/proc/self/maps"):
parts = line.split()
if len(parts) >= 6 and "libggml-" in parts[-1]:
loaded.add(os.path.basename(parts[-1]))
cpu = sorted(name for name in loaded if name.startswith("libggml-cpu-"))
assert cpu, "no CPU variant was loaded"
print("CPU variant loaded:", ", ".join(cpu))
# The Vulkan module registers only when a GPU driver answers, and this image
# has none, so prove what it needs instead: its NEEDED libraries (libvulkan,
# the ggml base) resolve from the installed layout.
ctypes.CDLL(os.path.join(libdir, "libggml-vulkan.so"))
print("Vulkan backend library loads from the package")
PY
python3 -c "import pyaudio; print('pyaudio ok')"
python3 -c "import socks; print('pysocks ok')"
python3 -c "import vocalinux.main, vocalinux.speech_recognition.recognition_manager; print('vocalinux imports ok')"
@@ -0,0 +1,39 @@
--- a/src/main.cpp
+++ b/src/main.cpp
@@ -16,6 +16,10 @@
#include <pybind11/numpy.h>
#include "whisper.h"
+#include "ggml-backend.h"
+
+#include <dlfcn.h>
+#include <string>
#define STRINGIFY(x) #x
@@ -850,7 +854,25 @@
////////////
+// With GGML_BACKEND_DL the CPU and GPU backends are libraries next to this
+// extension, and whisper.cpp cannot create a context until a CPU device is
+// registered. ggml searches the executable's directory (the interpreter's, for
+// Python) and the working directory, so load them from this module's directory.
+static void load_ggml_backends_beside_module() {
+ Dl_info info;
+ if (dladdr(reinterpret_cast<void *>(&load_ggml_backends_beside_module), &info) == 0 ||
+ info.dli_fname == nullptr) {
+ ggml_backend_load_all();
+ return;
+ }
+ const std::string path(info.dli_fname);
+ const std::size_t slash = path.find_last_of('/');
+ const std::string dir = slash == std::string::npos ? std::string(".") : path.substr(0, slash);
+ ggml_backend_load_all_from_path(dir.c_str());
+}
+
PYBIND11_MODULE(_pywhispercpp, m) {
+ load_ggml_backends_beside_module();
m.doc() = R"pbdoc(
Pywhispercpp: Python binding to whisper.cpp
-----------------------
@@ -3,7 +3,7 @@ pkgname=python3-pywhispercpp
version=1.5.0
revision=1
build_style=python3-module
hostmakedepends="cmake pkg-config python3-setuptools python3-wheel $(vopt_if vulkan shaderc)"
hostmakedepends="cmake pkg-config python3-setuptools python3-setuptools_scm python3-wheel $(vopt_if vulkan shaderc)"
makedepends="python3-devel $(vopt_if vulkan vulkan-loader-devel)"
depends="python3-numpy python3-requests python3-tqdm python3-platformdirs $(vopt_if vulkan vulkan-loader)"
short_desc="Python bindings for whisper.cpp speech recognition"
@@ -20,21 +20,23 @@ build_options="vulkan"
build_options_default="vulkan"
pre_build() {
# The sdist derives its version from git metadata, which is absent in
# the xbps build root, so inject the pinned version explicitly. This is
# the same version install.sh (PYWHISPERCPP_VERSION) and uv.lock pin.
export PYWHISPERCPP_VERSION=${version}
# Portable x86-64 baseline: GGML_NATIVE is ggml's default and compiles
# libggml-cpu with -march=native, so the package would inherit the
# builder's instruction set and SIGILL on older CPUs (#938).
# GGML_NATIVE=OFF alone is not enough: while the native default is on,
# ggml turns every GGML_* SIMD option on and still compiles -mavx2
# -mfma (#957). Each option is pinned off, mirroring the Flatpak build.
# Shared libraries with an $ORIGIN rpath let the extension resolve the
# whisper.cpp libraries installed beside it (as install.sh does), with
# no LD_LIBRARY_PATH at runtime. (setup.py install never builds a
# wheel, so NO_REPAIR is meaningless here and deliberately unset.)
export CMAKE_ARGS='-DCMAKE_INSTALL_RPATH=$ORIGIN -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF -DGGML_SSE42=OFF -DGGML_AVX=OFF -DGGML_AVX2=OFF -DGGML_BMI2=OFF -DGGML_FMA=OFF -DGGML_F16C=OFF'
# Version (#43). setup.py passes no version to setup(); the version comes
# from setuptools-scm, which the sdist declares as a build requirement and
# which hostmakedepends provides. The xbps build root has no git metadata,
# so pin the version through setuptools-scm. Without it setuptools falls
# back to 0.0.0. The sdist's PYWHISPERCPP_VERSION hook is never called.
export SETUPTOOLS_SCM_PRETEND_VERSION=${version}
# CPU (#44). GGML_BACKEND_DL builds each CPU variant (x64 baseline,
# sandybridge, haswell, icelake, zen4, ...) as its own library. The
# extension, through patches/, loads the variants that sit beside it, and
# ggml picks the best one the host CPU supports. The x64 baseline always
# ships, so the package still runs on any x86-64 CPU, and hosts with AVX2
# use it. GGML_NATIVE must stay OFF: ggml refuses it with GGML_BACKEND_DL,
# and it keeps -march=native out of every variant (#938).
# Shared libraries with an $ORIGIN rpath let each variant resolve
# libggml-base beside it, with no LD_LIBRARY_PATH at runtime. (setup.py
# install never builds a wheel, so NO_REPAIR is meaningless here.)
export CMAKE_ARGS='-DCMAKE_INSTALL_RPATH=$ORIGIN -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON'
if [ -n "$build_option_vulkan" ]; then
# Vulkan acceleration for every GPU vendor. There is no CUDA
# build: Void ships no CUDA toolkit package.
@@ -42,6 +44,20 @@ pre_build() {
fi
}
pre_install() {
# The backend modules (libggml-cpu-*.so, libggml-vulkan.so) are written to
# build/bin, a CMake directory whose whisper.cpp setup fixes, and setup.py
# installs only the build/lib tree that holds the extension. Copy them
# beside the extension, where patches/ loads them from.
local backend found=""
for backend in build/temp.*/_pywhispercpp/bin/libggml-*.so; do
[ -f "$backend" ] || continue
cp "$backend" build/lib.*/
found=1
done
[ -n "$found" ] || msg_error "no ggml backend modules were built; the CPU variants are missing\n"
}
post_install() {
vlicense LICENSE
}
@@ -195,7 +195,11 @@ def _pywhispercpp_distribution_version() -> Optional[tuple[int, ...]]:
parts.append(int(token))
else:
break
return tuple(parts) if parts else None
# setuptools writes 0.0.0 when a build had no version (#43). That means
# unknown, not old, so the feature probe in the caller decides instead.
if not parts or tuple(parts) == (0, 0, 0):
return None
return tuple(parts)
def _select_ggml_gpu_device(
+15 -1
View File
@@ -83,6 +83,18 @@ def find_shared_library_dirs() -> list[str]:
return library_dirs
def _is_runtime_cpu_variant(path: Path) -> bool:
"""True for a ggml CPU variant that ggml's own loader selects at run time.
GGML_CPU_ALL_VARIANTS builds one ``libggml-cpu-<level>.so`` per instruction
set level. The variants export the same symbols, so preloading them with
RTLD_GLOBAL would let the first one loaded interpose the others. ggml
loads the one the host needs. A non-variant build has a single
``libggml-cpu.so``, which the extension still needs preloaded.
"""
return path.name.startswith("libggml-cpu-")
def preload_shared_libraries() -> None:
"""Preload bundled pywhispercpp shared libraries for source-built installs."""
if _PRELOADED_LIBS:
@@ -91,7 +103,9 @@ def preload_shared_libraries() -> None:
libraries: list[Path] = []
for library_dir in find_shared_library_dirs():
root = Path(library_dir)
libraries.extend(sorted(root.glob("libggml*.so*")))
libraries.extend(
path for path in sorted(root.glob("libggml*.so*")) if not _is_runtime_cpu_variant(path)
)
libraries.extend(sorted(root.glob("libwhisper*.so*")))
if not libraries:
+3 -1
View File
@@ -361,7 +361,9 @@ def detect_ggml_gpu_devices() -> Optional[list[dict[str, Any]]]:
if version_match is None:
return None
installed_version = tuple(int(part or 0) for part in version_match.groups())
if installed_version < (1, 5, 0):
# 0.0.0 is what setuptools writes when a build had no version (#43). That means
# unknown, not old, so the runtime probe below decides whether devices are listed.
if installed_version != (0, 0, 0) and installed_version < (1, 5, 0):
return None
try:
+50
View File
@@ -0,0 +1,50 @@
"""Preloading must leave ggml's runtime-selected CPU variants to ggml (#44)."""
from pathlib import Path
from unittest.mock import patch
from vocalinux.utils import pywhispercpp_loader
def _preload_names(tmp_path: Path, *names: str) -> list[str]:
"""Run the preloader over ``names`` in one directory and return what it dlopens."""
for name in names:
(tmp_path / name).write_bytes(b"")
loaded: list[str] = []
def record(path: str, mode: int = 0) -> object:
loaded.append(Path(path).name)
return object()
with (
patch.object(pywhispercpp_loader, "find_shared_library_dirs", return_value=[str(tmp_path)]),
patch.object(pywhispercpp_loader, "_PRELOADED_LIBS", []),
patch.object(pywhispercpp_loader.ctypes, "CDLL", side_effect=record),
):
pywhispercpp_loader.preload_shared_libraries()
return loaded
def test_preload_leaves_runtime_cpu_variants_to_ggml(tmp_path: Path) -> None:
"""Variants share one symbol table; preloading them RTLD_GLOBAL would interpose them."""
loaded = _preload_names(
tmp_path,
"libggml.so.0",
"libggml-base.so.0",
"libggml-cpu-x64.so",
"libggml-cpu-haswell.so",
"libggml-vulkan.so",
"libwhisper.so.1",
)
assert not [name for name in loaded if name.startswith("libggml-cpu-")]
assert {"libggml.so.0", "libggml-base.so.0", "libggml-vulkan.so", "libwhisper.so.1"} <= set(
loaded
)
def test_preload_keeps_the_single_cpu_library_of_a_non_variant_build(tmp_path: Path) -> None:
"""Without GGML_CPU_ALL_VARIANTS the CPU backend is a dependency and must be preloaded."""
loaded = _preload_names(tmp_path, "libggml.so.0", "libggml-cpu.so")
assert "libggml-cpu.so" in loaded
@@ -1868,6 +1868,40 @@ class TestWhispercppGpuDeviceSelection(unittest.TestCase):
assert len(self.ContextParamsModel.calls) == 1
assert "context_params" not in self.ContextParamsModel.calls[0][1]
def test_placeholder_distribution_version_is_unknown_not_old(self):
"""setuptools writes 0.0.0 when a build had no version; that is not 'too old' (#43)."""
from vocalinux.speech_recognition import recognition_manager
with patch("importlib.metadata.version", return_value="0.0.0"):
assert recognition_manager._pywhispercpp_distribution_version() is None
with patch("importlib.metadata.version", return_value="1.5.0"):
assert recognition_manager._pywhispercpp_distribution_version() == (1, 5, 0)
def test_gpu_device_kept_when_distribution_version_is_placeholder(self):
"""A 0.0.0 build still has context_params, so device selection must stay on (#43)."""
manager = _make_manager(engine="whisper_cpp", whispercpp_gpu_device=1)
manager.model_size = "tiny"
mock_pywhispercpp = MagicMock()
self.ContextParamsModel.calls = []
mock_pywhispercpp.Model = self.ContextParamsModel
with patch.dict(
"sys.modules",
{
"pywhispercpp": mock_pywhispercpp,
"pywhispercpp.model": mock_pywhispercpp,
},
):
with patch("importlib.metadata.version", return_value="0.0.0"):
result = manager._load_model_with_compatible_params(
"/fake/model.bin", {}, gpu_device=1
)
assert isinstance(result, self.ContextParamsModel)
assert len(self.ContextParamsModel.calls) == 1
assert self.ContextParamsModel.calls[0][1].get("context_params") == {"gpu_device": 1}
def test_host_cuda_label_runtime_vulkan_selects_discrete(self):
"""Device selection follows bundled Vulkan libs, not a host CUDA label (#604)."""
manager = _make_manager(engine="whisper_cpp", whispercpp_gpu_device=None)
+87 -28
View File
@@ -3,7 +3,8 @@
Spec #16 packages four carried templates in void-packages layout under
``packaging/void``; this ticket owns the hard one, ``python3-pywhispercpp``,
built from the pinned PyPI source distribution with the Vulkan backend,
shared libraries, and portable CPU flags. Ticket #28 builds the container
shared libraries, runtime-selected CPU variants (#44), and a version pinned
through setuptools-scm (#43). Ticket #28 builds the container
smoke gate on top; everything asserted here runs offline from sources
already in the repo.
@@ -24,21 +25,27 @@ BUILD_SH = VOID_DIR / "build-release.sh"
INSTALL_SH = REPO_ROOT / "install.sh"
UV_LOCK = REPO_ROOT / "uv.lock"
#: ggml SIMD options pinned off for the plain x86-64 baseline (#957).
#: GGML_NATIVE=OFF alone is not portable: while ggml's native default is on,
#: every GGML_* SIMD option stays on and still compiles -mavx2 -mfma.
#: Mirrors PORTABLE_GGML_OPTIONS in tests/test_flatpak_packaging.py and the
#: CMAKE_ARGS line in packaging/flatpak/python3-dependencies.yaml.
PORTABLE_GGML_OPTIONS = (
"GGML_NATIVE",
"GGML_SSE42",
"GGML_AVX",
"GGML_AVX2",
"GGML_BMI2",
"GGML_FMA",
"GGML_F16C",
#: CMake arguments of the CPU backend (#44): one library per CPU level, loaded
#: at run time, with the x86-64 baseline always shipped. GGML_NATIVE must stay
#: OFF: ggml refuses it with GGML_BACKEND_DL, and it would compile the builder's
#: instruction set into every variant (#938).
REQUIRED_CMAKE_ARGS = (
"-DGGML_NATIVE=OFF",
"-DGGML_BACKEND_DL=ON",
"-DGGML_CPU_ALL_VARIANTS=ON",
"-DBUILD_SHARED_LIBS=ON",
)
#: The binding patch that loads the backend modules from the extension's directory.
BINDING_PATCH = (
VOID_DIR
/ "srcpkgs"
/ "python3-pywhispercpp"
/ "patches"
/ "load-ggml-backends-beside-module.patch"
)
INSTALL_SMOKE = VOID_DIR / "container-release.sh"
#: install_requires of pywhispercpp's setup.py, mapped to the Void binary
#: package names. Each name was verified to exist as srcpkgs/<name>/template
#: in current void-packages (2026-11-05); the container build re-verifies via
@@ -126,21 +133,57 @@ def test_build_style_is_python3_module() -> None:
assert re.search(r"^build_style=python3-module$", _template(), re.M)
def test_portable_cpu_flags_pin_every_ggml_simd_option_off() -> None:
"""No host-native tuning: the plain x86-64 baseline, like the Flatpak."""
text = _template()
for flag in PORTABLE_GGML_OPTIONS:
assert f"-D{flag}=OFF" in text, (
f"template does not pin {flag} off; the package inherits the "
"builder's instruction set and SIGILLs on older CPUs (#938, #957)"
)
def _code_lines() -> str:
"""Template with comment lines stripped; flags live in code, not prose."""
return "\n".join(line for line in _template().splitlines() if not line.lstrip().startswith("#"))
def test_cpu_backend_is_built_as_runtime_selected_variants() -> None:
"""#44: every CPU level is its own library, and the x86-64 baseline ships."""
text = _code_lines()
for arg in REQUIRED_CMAKE_ARGS:
assert arg in text, f"template does not pass {arg}"
def test_no_simd_option_is_forced_on_for_the_whole_build() -> None:
"""A global SIMD switch would give every variant one instruction set (#938, #957)."""
text = _code_lines()
for feature in ("NATIVE", "SSE42", "AVX", "AVX2", "BMI2", "FMA", "F16C", "AVX512"):
assert f"-DGGML_{feature}=ON" not in text, f"GGML_{feature} forced on for every variant"
def test_backends_are_copied_beside_the_extension_before_install() -> None:
"""#44: whisper.cpp writes the modules to build/bin, which setup.py does not install."""
text = _template()
assert "pre_install()" in text
assert "build/temp.*/_pywhispercpp/bin/libggml-*.so" in text
assert "build/lib.*/" in text
assert "msg_error" in text, "the build must fail loudly when no backend module was built"
def test_binding_loads_the_backends_beside_the_extension() -> None:
"""#44: ggml's default search misses the package directory, so the binding loads it."""
assert BINDING_PATCH.is_file(), f"missing binding patch: {BINDING_PATCH}"
text = BINDING_PATCH.read_text(encoding="utf-8")
assert "--- a/src/main.cpp" in text and "+++ b/src/main.cpp" in text
assert " PYBIND11_MODULE(_pywhispercpp, m) {" in text
assert "+ load_ggml_backends_beside_module();" in text
assert "+ ggml_backend_load_all_from_path(dir.c_str());" in text
def test_install_smoke_checks_version_and_cpu_variants() -> None:
"""#43 and #44: the installed binding reports the pinned version, and its variants load."""
text = INSTALL_SMOKE.read_text(encoding="utf-8")
version = re.search(r"^version=(\S+)", _template(), re.M)
assert version, "template declares no version"
assert (
f'[ "$PWC_VERSION" = "{version.group(1)}" ]' in text
), "the install smoke must assert the installed pywhispercpp reports the template version"
assert "libggml-cpu-x64.so" in text and "libggml-cpu-haswell.so" in text
assert "objdump -d" in text, "the baseline variant must be checked for AVX code"
assert "libggml.so.0" in text, "the backend registry lives in libggml"
def test_no_native_tuning_or_cuda_anywhere() -> None:
"""Void has no CUDA toolkit package; portable means portable."""
text = _code_lines()
@@ -181,15 +224,31 @@ def test_build_time_deps_cover_setup_py() -> None:
text = _template()
host = re.search(r"^hostmakedepends=(.*)$", text, re.M)
assert host, "template declares no hostmakedepends"
for dep in ("cmake", "pkg-config", "python3-setuptools", "python3-wheel"):
for dep in (
"cmake",
"pkg-config",
"python3-setuptools",
"python3-setuptools_scm",
"python3-wheel",
):
assert dep in host.group(1), f"{dep} missing from hostmakedepends"
make = re.search(r"^makedepends=(.*)$", text, re.M)
assert make and "python3-devel" in make.group(1), "compiled extension needs python3-devel"
def test_version_is_injected_for_the_offline_build() -> None:
"""The sdist carries no version.txt; setup.py reads PYWHISPERCPP_VERSION."""
assert "PYWHISPERCPP_VERSION" in _template()
def test_version_is_pinned_through_setuptools_scm() -> None:
"""#43: setup() gets no version; setuptools-scm supplies it, so the build must provide it."""
host = re.search(r"^hostmakedepends=(.*)$", _template(), re.M)
assert host and "python3-setuptools_scm" in host.group(
1
), "without setuptools-scm the build reports version 0.0.0"
assert (
"SETUPTOOLS_SCM_PRETEND_VERSION=${version}" in _code_lines()
), "the xbps build root has no git metadata; pin the template version"
installer = (REPO_ROOT / "requirements" / "installer-build.txt").read_text(encoding="utf-8")
assert re.search(
r"^setuptools-scm==", installer, re.M
), "the installer build tools pin setuptools-scm as well"
def test_build_fetches_nothing_beyond_the_sdist() -> None:
+8
View File
@@ -580,6 +580,14 @@ class TestDetectGgmlGpuDevices(unittest.TestCase):
assert devices is not None
self.assertEqual(len(devices), 1)
def test_probe_trusts_runtime_when_version_is_placeholder(
self: "TestDetectGgmlGpuDevices",
) -> None:
"""setuptools' 0.0.0 means the build had no version (#43): probe the runtime."""
devices = _run_ggml_probe([("Vulkan0", "GPU", 1)], version_string="0.0.0")
assert devices is not None
self.assertEqual(len(devices), 1)
class TestPreferDiscreteVulkanDeviceMore(unittest.TestCase):
"""Exercise additional Vulkan selection edge cases."""