diff --git a/packaging/void/README.md b/packaging/void/README.md index 30d825a..5f3d52d 100644 --- a/packaging/void/README.md +++ b/packaging/void/README.md @@ -88,19 +88,35 @@ for the sdist; `tests/test_void_pywhispercpp_template.py` fails otherwise. There is no CUDA build: Void ships no CUDA toolkit package. Rebuilding locally with a hand-installed toolkit (`GGML_CUDA=1`) is a documented, unsupported escape hatch. -- **Portable CPU flags**: every ggml SIMD option is pinned off - (`-DGGML_NATIVE=OFF -DGGML_SSE42=OFF ...`), the plain x86-64 baseline - already used by the Flatpak (`#938`, `#957`). The install smoke scans the - shipped `libggml-cpu` (`.GCC.command.line`) for `-march=native`, - `-mcpu=native` and `-mtune=native` and fails on a match. The scan is a - tripwire, not a proof: the proof is the pinned configuration. +- **CPU variants (`#44`)**: `GGML_BACKEND_DL=ON` and + `GGML_CPU_ALL_VARIANTS=ON` build one CPU library per instruction-set + level (`libggml-cpu-x64.so` for the plain x86-64 baseline, then + `haswell`, `skylakex`, `zen4`, ...). ggml loads the best one the host + supports, so a host with AVX2 runs AVX2 code, and the baseline variant + ships everywhere. `GGML_NATIVE` stays `OFF`: ggml refuses it with + `GGML_BACKEND_DL`, and it keeps `-march=native` out of every variant + (`#938`). `patches/load-ggml-backends-beside-module.patch` makes the + extension load the variants from its own directory: ggml's default search + covers the interpreter's directory and the working directory, and neither + holds them, so without the patch no CPU backend is registered. The install + smoke checks that `libggml-cpu-x64.so` holds no AVX (`ymm`/`zmm`) code, + that `libggml-cpu-haswell.so` does, that no variant carries host-native + tuning (`.GCC.command.line`), and that the CPU and Vulkan backends load + from the installed package. +- **Version (`#43`)**: `setup.py` passes no version to `setup()`. The + version comes from setuptools-scm, which the sdist declares as a build + requirement, so `hostmakedepends` provides `python3-setuptools_scm` and + `SETUPTOOLS_SCM_PRETEND_VERSION` pins `1.5.0` without git metadata. Without + setuptools-scm, setuptools writes `0.0.0`, which the app reads as an + unknown binding version and then skips device selection. The install smoke + asserts the installed package reports `1.5.0`. - **Shared libraries**: `BUILD_SHARED_LIBS=ON` with an `$ORIGIN` rpath (as `install.sh` does), so the extension resolves the whisper.cpp libraries installed beside it with no `LD_LIBRARY_PATH`. - **Offline build**: the sdist vendors whisper.cpp, pybind11, and the ggml sources; the build fetches nothing beyond the declared distfiles. - `PYWHISPERCPP_VERSION` is injected because the sdist carries no - `version.txt` (its version otherwise comes from git metadata). + The sdist carries no git metadata, which is why the version is pinned + through setuptools-scm (see Version above). ## voculinux @@ -202,8 +218,9 @@ host Void install and nothing is published. Stages, in order, each fails the run `python3-pysocks` are installed, then `voculinux` installs and upgrades both to the carried versions (see [Version floors](#version-floors)), `voculinux --version` shows the version, the payload checks pass (desktop - entry, icon, marker, no models, no pre-rename names, the Vulkan backend, no - host-native CPU tuning), and the imports load (`pynput` through its dummy + entry, icon, marker, no models, no pre-rename names, the Vulkan backend, the + CPU variants with an AVX-free x86-64 baseline, no host-native CPU tuning), + the installed pywhispercpp reports `1.5.0`, and the imports load (`pynput` through its dummy backend, `evdev`, `gi` with Gtk 3, `pywhispercpp.model`, `pyaudio`, `socks`, and the app's own modules, which keep their retained internal name). diff --git a/packaging/void/container-release.sh b/packaging/void/container-release.sh index c0556d6..a3e6885 100755 --- a/packaging/void/container-release.sh +++ b/packaging/void/container-release.sh @@ -106,19 +106,46 @@ payload_voculinux() { echo "payload ok: desktop entry, icon and marker present; no models, wheels or pre-rename names" } -# whisper.cpp must ship its Vulkan backend and no host-native CPU tuning. +# whisper.cpp ships its Vulkan backend and its CPU backend as runtime-loaded +# variants beside the extension (GGML_BACKEND_DL, #44). Every CPU variant +# carries its own instruction set, but the x86-64 baseline variant must hold no +# AVX code (ymm/zmm), the AVX2 variant must hold some (so the variants are real, +# not one baseline build), and every other shipped native library (the ggml +# base, the registry, whisper, the extension, Vulkan) must stay AVX-free. No +# library may carry host-native tuning. payload_pywhispercpp() { PKGFILE=$1 tar -tf "$PKGFILE" | grep -q "libggml-vulkan" || fail "the Vulkan backend is missing from $PKGFILE" - LIBMEMBER=$(tar -tf "$PKGFILE" | grep -E "libggml-cpu\.so" | head -n1) - [ -n "$LIBMEMBER" ] || fail "libggml-cpu is missing from $PKGFILE" - mkdir -p /flagscan - tar -xf "$PKGFILE" -C /flagscan "$LIBMEMBER" - LIBSCAN="/flagscan/${LIBMEMBER#./}" - if readelf -p .GCC.command.line "$LIBSCAN" 2>/dev/null | grep -E "march=native|mcpu=native|mtune=native"; then - fail "host-native CPU tuning in the shipped libggml-cpu" + for VARIANT in x64 haswell; do + tar -tf "$PKGFILE" | grep -q "libggml-cpu-$VARIANT\.so" || + fail "CPU variant libggml-cpu-$VARIANT.so is missing from $PKGFILE" + done + rm -rf /flagscan && mkdir -p /flagscan + tar -xf "$PKGFILE" -C /flagscan --wildcards '*.so*' + while IFS= read -r LIBSCAN; do + NAME=${LIBSCAN##*/} + if readelf -p .GCC.command.line "$LIBSCAN" 2>/dev/null | grep -E "march=native|mcpu=native|mtune=native"; then + fail "host-native CPU tuning in $NAME" + fi + case "$NAME" in + libggml-cpu-*) ;; # each variant is checked below + *) + if objdump -d "$LIBSCAN" | grep -Eq '%[yz]mm[0-9]'; then + fail "$NAME contains AVX code; only the CPU variants may" + fi + ;; + esac + done <= 6 and "libggml-" in parts[-1]: + loaded.add(os.path.basename(parts[-1])) +cpu = sorted(name for name in loaded if name.startswith("libggml-cpu-")) +assert cpu, "no CPU variant was loaded" +print("CPU variant loaded:", ", ".join(cpu)) +# The Vulkan module registers only when a GPU driver answers, and this image +# has none, so prove what it needs instead: its NEEDED libraries (libvulkan, +# the ggml base) resolve from the installed layout. +ctypes.CDLL(os.path.join(libdir, "libggml-vulkan.so")) +print("Vulkan backend library loads from the package") +PY python3 -c "import pyaudio; print('pyaudio ok')" python3 -c "import socks; print('pysocks ok')" python3 -c "import vocalinux.main, vocalinux.speech_recognition.recognition_manager; print('vocalinux imports ok')" diff --git a/packaging/void/srcpkgs/python3-pywhispercpp/patches/load-ggml-backends-beside-module.patch b/packaging/void/srcpkgs/python3-pywhispercpp/patches/load-ggml-backends-beside-module.patch new file mode 100644 index 0000000..5ff6a43 --- /dev/null +++ b/packaging/void/srcpkgs/python3-pywhispercpp/patches/load-ggml-backends-beside-module.patch @@ -0,0 +1,39 @@ +--- a/src/main.cpp ++++ b/src/main.cpp +@@ -16,6 +16,10 @@ + #include + + #include "whisper.h" ++#include "ggml-backend.h" ++ ++#include ++#include + + + #define STRINGIFY(x) #x +@@ -850,7 +854,25 @@ + + //////////// + ++// With GGML_BACKEND_DL the CPU and GPU backends are libraries next to this ++// extension, and whisper.cpp cannot create a context until a CPU device is ++// registered. ggml searches the executable's directory (the interpreter's, for ++// Python) and the working directory, so load them from this module's directory. ++static void load_ggml_backends_beside_module() { ++ Dl_info info; ++ if (dladdr(reinterpret_cast(&load_ggml_backends_beside_module), &info) == 0 || ++ info.dli_fname == nullptr) { ++ ggml_backend_load_all(); ++ return; ++ } ++ const std::string path(info.dli_fname); ++ const std::size_t slash = path.find_last_of('/'); ++ const std::string dir = slash == std::string::npos ? std::string(".") : path.substr(0, slash); ++ ggml_backend_load_all_from_path(dir.c_str()); ++} ++ + PYBIND11_MODULE(_pywhispercpp, m) { ++ load_ggml_backends_beside_module(); + m.doc() = R"pbdoc( + Pywhispercpp: Python binding to whisper.cpp + ----------------------- diff --git a/packaging/void/srcpkgs/python3-pywhispercpp/template b/packaging/void/srcpkgs/python3-pywhispercpp/template index 602ca6f..b78721b 100644 --- a/packaging/void/srcpkgs/python3-pywhispercpp/template +++ b/packaging/void/srcpkgs/python3-pywhispercpp/template @@ -3,7 +3,7 @@ pkgname=python3-pywhispercpp version=1.5.0 revision=1 build_style=python3-module -hostmakedepends="cmake pkg-config python3-setuptools python3-wheel $(vopt_if vulkan shaderc)" +hostmakedepends="cmake pkg-config python3-setuptools python3-setuptools_scm python3-wheel $(vopt_if vulkan shaderc)" makedepends="python3-devel $(vopt_if vulkan vulkan-loader-devel)" depends="python3-numpy python3-requests python3-tqdm python3-platformdirs $(vopt_if vulkan vulkan-loader)" short_desc="Python bindings for whisper.cpp speech recognition" @@ -20,21 +20,23 @@ build_options="vulkan" build_options_default="vulkan" pre_build() { - # The sdist derives its version from git metadata, which is absent in - # the xbps build root, so inject the pinned version explicitly. This is - # the same version install.sh (PYWHISPERCPP_VERSION) and uv.lock pin. - export PYWHISPERCPP_VERSION=${version} - # Portable x86-64 baseline: GGML_NATIVE is ggml's default and compiles - # libggml-cpu with -march=native, so the package would inherit the - # builder's instruction set and SIGILL on older CPUs (#938). - # GGML_NATIVE=OFF alone is not enough: while the native default is on, - # ggml turns every GGML_* SIMD option on and still compiles -mavx2 - # -mfma (#957). Each option is pinned off, mirroring the Flatpak build. - # Shared libraries with an $ORIGIN rpath let the extension resolve the - # whisper.cpp libraries installed beside it (as install.sh does), with - # no LD_LIBRARY_PATH at runtime. (setup.py install never builds a - # wheel, so NO_REPAIR is meaningless here and deliberately unset.) - export CMAKE_ARGS='-DCMAKE_INSTALL_RPATH=$ORIGIN -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF -DGGML_SSE42=OFF -DGGML_AVX=OFF -DGGML_AVX2=OFF -DGGML_BMI2=OFF -DGGML_FMA=OFF -DGGML_F16C=OFF' + # Version (#43). setup.py passes no version to setup(); the version comes + # from setuptools-scm, which the sdist declares as a build requirement and + # which hostmakedepends provides. The xbps build root has no git metadata, + # so pin the version through setuptools-scm. Without it setuptools falls + # back to 0.0.0. The sdist's PYWHISPERCPP_VERSION hook is never called. + export SETUPTOOLS_SCM_PRETEND_VERSION=${version} + # CPU (#44). GGML_BACKEND_DL builds each CPU variant (x64 baseline, + # sandybridge, haswell, icelake, zen4, ...) as its own library. The + # extension, through patches/, loads the variants that sit beside it, and + # ggml picks the best one the host CPU supports. The x64 baseline always + # ships, so the package still runs on any x86-64 CPU, and hosts with AVX2 + # use it. GGML_NATIVE must stay OFF: ggml refuses it with GGML_BACKEND_DL, + # and it keeps -march=native out of every variant (#938). + # Shared libraries with an $ORIGIN rpath let each variant resolve + # libggml-base beside it, with no LD_LIBRARY_PATH at runtime. (setup.py + # install never builds a wheel, so NO_REPAIR is meaningless here.) + export CMAKE_ARGS='-DCMAKE_INSTALL_RPATH=$ORIGIN -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -DBUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON' if [ -n "$build_option_vulkan" ]; then # Vulkan acceleration for every GPU vendor. There is no CUDA # build: Void ships no CUDA toolkit package. @@ -42,6 +44,20 @@ pre_build() { fi } +pre_install() { + # The backend modules (libggml-cpu-*.so, libggml-vulkan.so) are written to + # build/bin, a CMake directory whose whisper.cpp setup fixes, and setup.py + # installs only the build/lib tree that holds the extension. Copy them + # beside the extension, where patches/ loads them from. + local backend found="" + for backend in build/temp.*/_pywhispercpp/bin/libggml-*.so; do + [ -f "$backend" ] || continue + cp "$backend" build/lib.*/ + found=1 + done + [ -n "$found" ] || msg_error "no ggml backend modules were built; the CPU variants are missing\n" +} + post_install() { vlicense LICENSE } diff --git a/src/vocalinux/speech_recognition/recognition_manager.py b/src/vocalinux/speech_recognition/recognition_manager.py index 7b45704..69ff1b8 100644 --- a/src/vocalinux/speech_recognition/recognition_manager.py +++ b/src/vocalinux/speech_recognition/recognition_manager.py @@ -195,7 +195,11 @@ def _pywhispercpp_distribution_version() -> Optional[tuple[int, ...]]: parts.append(int(token)) else: break - return tuple(parts) if parts else None + # setuptools writes 0.0.0 when a build had no version (#43). That means + # unknown, not old, so the feature probe in the caller decides instead. + if not parts or tuple(parts) == (0, 0, 0): + return None + return tuple(parts) def _select_ggml_gpu_device( diff --git a/src/vocalinux/utils/pywhispercpp_loader.py b/src/vocalinux/utils/pywhispercpp_loader.py index 843c4f6..4cf023e 100644 --- a/src/vocalinux/utils/pywhispercpp_loader.py +++ b/src/vocalinux/utils/pywhispercpp_loader.py @@ -83,6 +83,18 @@ def find_shared_library_dirs() -> list[str]: return library_dirs +def _is_runtime_cpu_variant(path: Path) -> bool: + """True for a ggml CPU variant that ggml's own loader selects at run time. + + GGML_CPU_ALL_VARIANTS builds one ``libggml-cpu-.so`` per instruction + set level. The variants export the same symbols, so preloading them with + RTLD_GLOBAL would let the first one loaded interpose the others. ggml + loads the one the host needs. A non-variant build has a single + ``libggml-cpu.so``, which the extension still needs preloaded. + """ + return path.name.startswith("libggml-cpu-") + + def preload_shared_libraries() -> None: """Preload bundled pywhispercpp shared libraries for source-built installs.""" if _PRELOADED_LIBS: @@ -91,7 +103,9 @@ def preload_shared_libraries() -> None: libraries: list[Path] = [] for library_dir in find_shared_library_dirs(): root = Path(library_dir) - libraries.extend(sorted(root.glob("libggml*.so*"))) + libraries.extend( + path for path in sorted(root.glob("libggml*.so*")) if not _is_runtime_cpu_variant(path) + ) libraries.extend(sorted(root.glob("libwhisper*.so*"))) if not libraries: diff --git a/src/vocalinux/utils/whispercpp_model_info.py b/src/vocalinux/utils/whispercpp_model_info.py index 777eeb9..252628d 100644 --- a/src/vocalinux/utils/whispercpp_model_info.py +++ b/src/vocalinux/utils/whispercpp_model_info.py @@ -361,7 +361,9 @@ def detect_ggml_gpu_devices() -> Optional[list[dict[str, Any]]]: if version_match is None: return None installed_version = tuple(int(part or 0) for part in version_match.groups()) - if installed_version < (1, 5, 0): + # 0.0.0 is what setuptools writes when a build had no version (#43). That means + # unknown, not old, so the runtime probe below decides whether devices are listed. + if installed_version != (0, 0, 0) and installed_version < (1, 5, 0): return None try: diff --git a/tests/test_pywhispercpp_loader.py b/tests/test_pywhispercpp_loader.py new file mode 100644 index 0000000..c49379e --- /dev/null +++ b/tests/test_pywhispercpp_loader.py @@ -0,0 +1,50 @@ +"""Preloading must leave ggml's runtime-selected CPU variants to ggml (#44).""" + +from pathlib import Path +from unittest.mock import patch + +from vocalinux.utils import pywhispercpp_loader + + +def _preload_names(tmp_path: Path, *names: str) -> list[str]: + """Run the preloader over ``names`` in one directory and return what it dlopens.""" + for name in names: + (tmp_path / name).write_bytes(b"") + loaded: list[str] = [] + + def record(path: str, mode: int = 0) -> object: + loaded.append(Path(path).name) + return object() + + with ( + patch.object(pywhispercpp_loader, "find_shared_library_dirs", return_value=[str(tmp_path)]), + patch.object(pywhispercpp_loader, "_PRELOADED_LIBS", []), + patch.object(pywhispercpp_loader.ctypes, "CDLL", side_effect=record), + ): + pywhispercpp_loader.preload_shared_libraries() + return loaded + + +def test_preload_leaves_runtime_cpu_variants_to_ggml(tmp_path: Path) -> None: + """Variants share one symbol table; preloading them RTLD_GLOBAL would interpose them.""" + loaded = _preload_names( + tmp_path, + "libggml.so.0", + "libggml-base.so.0", + "libggml-cpu-x64.so", + "libggml-cpu-haswell.so", + "libggml-vulkan.so", + "libwhisper.so.1", + ) + + assert not [name for name in loaded if name.startswith("libggml-cpu-")] + assert {"libggml.so.0", "libggml-base.so.0", "libggml-vulkan.so", "libwhisper.so.1"} <= set( + loaded + ) + + +def test_preload_keeps_the_single_cpu_library_of_a_non_variant_build(tmp_path: Path) -> None: + """Without GGML_CPU_ALL_VARIANTS the CPU backend is a dependency and must be preloaded.""" + loaded = _preload_names(tmp_path, "libggml.so.0", "libggml-cpu.so") + + assert "libggml-cpu.so" in loaded diff --git a/tests/test_recognition_manager_device_config.py b/tests/test_recognition_manager_device_config.py index f570697..07aca81 100644 --- a/tests/test_recognition_manager_device_config.py +++ b/tests/test_recognition_manager_device_config.py @@ -1868,6 +1868,40 @@ class TestWhispercppGpuDeviceSelection(unittest.TestCase): assert len(self.ContextParamsModel.calls) == 1 assert "context_params" not in self.ContextParamsModel.calls[0][1] + def test_placeholder_distribution_version_is_unknown_not_old(self): + """setuptools writes 0.0.0 when a build had no version; that is not 'too old' (#43).""" + from vocalinux.speech_recognition import recognition_manager + + with patch("importlib.metadata.version", return_value="0.0.0"): + assert recognition_manager._pywhispercpp_distribution_version() is None + with patch("importlib.metadata.version", return_value="1.5.0"): + assert recognition_manager._pywhispercpp_distribution_version() == (1, 5, 0) + + def test_gpu_device_kept_when_distribution_version_is_placeholder(self): + """A 0.0.0 build still has context_params, so device selection must stay on (#43).""" + manager = _make_manager(engine="whisper_cpp", whispercpp_gpu_device=1) + manager.model_size = "tiny" + + mock_pywhispercpp = MagicMock() + self.ContextParamsModel.calls = [] + mock_pywhispercpp.Model = self.ContextParamsModel + + with patch.dict( + "sys.modules", + { + "pywhispercpp": mock_pywhispercpp, + "pywhispercpp.model": mock_pywhispercpp, + }, + ): + with patch("importlib.metadata.version", return_value="0.0.0"): + result = manager._load_model_with_compatible_params( + "/fake/model.bin", {}, gpu_device=1 + ) + + assert isinstance(result, self.ContextParamsModel) + assert len(self.ContextParamsModel.calls) == 1 + assert self.ContextParamsModel.calls[0][1].get("context_params") == {"gpu_device": 1} + def test_host_cuda_label_runtime_vulkan_selects_discrete(self): """Device selection follows bundled Vulkan libs, not a host CUDA label (#604).""" manager = _make_manager(engine="whisper_cpp", whispercpp_gpu_device=None) diff --git a/tests/test_void_pywhispercpp_template.py b/tests/test_void_pywhispercpp_template.py index 7a4f40e..9896b3a 100644 --- a/tests/test_void_pywhispercpp_template.py +++ b/tests/test_void_pywhispercpp_template.py @@ -3,7 +3,8 @@ Spec #16 packages four carried templates in void-packages layout under ``packaging/void``; this ticket owns the hard one, ``python3-pywhispercpp``, built from the pinned PyPI source distribution with the Vulkan backend, -shared libraries, and portable CPU flags. Ticket #28 builds the container +shared libraries, runtime-selected CPU variants (#44), and a version pinned +through setuptools-scm (#43). Ticket #28 builds the container smoke gate on top; everything asserted here runs offline from sources already in the repo. @@ -24,21 +25,27 @@ BUILD_SH = VOID_DIR / "build-release.sh" INSTALL_SH = REPO_ROOT / "install.sh" UV_LOCK = REPO_ROOT / "uv.lock" -#: ggml SIMD options pinned off for the plain x86-64 baseline (#957). -#: GGML_NATIVE=OFF alone is not portable: while ggml's native default is on, -#: every GGML_* SIMD option stays on and still compiles -mavx2 -mfma. -#: Mirrors PORTABLE_GGML_OPTIONS in tests/test_flatpak_packaging.py and the -#: CMAKE_ARGS line in packaging/flatpak/python3-dependencies.yaml. -PORTABLE_GGML_OPTIONS = ( - "GGML_NATIVE", - "GGML_SSE42", - "GGML_AVX", - "GGML_AVX2", - "GGML_BMI2", - "GGML_FMA", - "GGML_F16C", +#: CMake arguments of the CPU backend (#44): one library per CPU level, loaded +#: at run time, with the x86-64 baseline always shipped. GGML_NATIVE must stay +#: OFF: ggml refuses it with GGML_BACKEND_DL, and it would compile the builder's +#: instruction set into every variant (#938). +REQUIRED_CMAKE_ARGS = ( + "-DGGML_NATIVE=OFF", + "-DGGML_BACKEND_DL=ON", + "-DGGML_CPU_ALL_VARIANTS=ON", + "-DBUILD_SHARED_LIBS=ON", ) +#: The binding patch that loads the backend modules from the extension's directory. +BINDING_PATCH = ( + VOID_DIR + / "srcpkgs" + / "python3-pywhispercpp" + / "patches" + / "load-ggml-backends-beside-module.patch" +) +INSTALL_SMOKE = VOID_DIR / "container-release.sh" + #: install_requires of pywhispercpp's setup.py, mapped to the Void binary #: package names. Each name was verified to exist as srcpkgs//template #: in current void-packages (2026-11-05); the container build re-verifies via @@ -126,21 +133,57 @@ def test_build_style_is_python3_module() -> None: assert re.search(r"^build_style=python3-module$", _template(), re.M) -def test_portable_cpu_flags_pin_every_ggml_simd_option_off() -> None: - """No host-native tuning: the plain x86-64 baseline, like the Flatpak.""" - text = _template() - for flag in PORTABLE_GGML_OPTIONS: - assert f"-D{flag}=OFF" in text, ( - f"template does not pin {flag} off; the package inherits the " - "builder's instruction set and SIGILLs on older CPUs (#938, #957)" - ) - - def _code_lines() -> str: """Template with comment lines stripped; flags live in code, not prose.""" return "\n".join(line for line in _template().splitlines() if not line.lstrip().startswith("#")) +def test_cpu_backend_is_built_as_runtime_selected_variants() -> None: + """#44: every CPU level is its own library, and the x86-64 baseline ships.""" + text = _code_lines() + for arg in REQUIRED_CMAKE_ARGS: + assert arg in text, f"template does not pass {arg}" + + +def test_no_simd_option_is_forced_on_for_the_whole_build() -> None: + """A global SIMD switch would give every variant one instruction set (#938, #957).""" + text = _code_lines() + for feature in ("NATIVE", "SSE42", "AVX", "AVX2", "BMI2", "FMA", "F16C", "AVX512"): + assert f"-DGGML_{feature}=ON" not in text, f"GGML_{feature} forced on for every variant" + + +def test_backends_are_copied_beside_the_extension_before_install() -> None: + """#44: whisper.cpp writes the modules to build/bin, which setup.py does not install.""" + text = _template() + assert "pre_install()" in text + assert "build/temp.*/_pywhispercpp/bin/libggml-*.so" in text + assert "build/lib.*/" in text + assert "msg_error" in text, "the build must fail loudly when no backend module was built" + + +def test_binding_loads_the_backends_beside_the_extension() -> None: + """#44: ggml's default search misses the package directory, so the binding loads it.""" + assert BINDING_PATCH.is_file(), f"missing binding patch: {BINDING_PATCH}" + text = BINDING_PATCH.read_text(encoding="utf-8") + assert "--- a/src/main.cpp" in text and "+++ b/src/main.cpp" in text + assert " PYBIND11_MODULE(_pywhispercpp, m) {" in text + assert "+ load_ggml_backends_beside_module();" in text + assert "+ ggml_backend_load_all_from_path(dir.c_str());" in text + + +def test_install_smoke_checks_version_and_cpu_variants() -> None: + """#43 and #44: the installed binding reports the pinned version, and its variants load.""" + text = INSTALL_SMOKE.read_text(encoding="utf-8") + version = re.search(r"^version=(\S+)", _template(), re.M) + assert version, "template declares no version" + assert ( + f'[ "$PWC_VERSION" = "{version.group(1)}" ]' in text + ), "the install smoke must assert the installed pywhispercpp reports the template version" + assert "libggml-cpu-x64.so" in text and "libggml-cpu-haswell.so" in text + assert "objdump -d" in text, "the baseline variant must be checked for AVX code" + assert "libggml.so.0" in text, "the backend registry lives in libggml" + + def test_no_native_tuning_or_cuda_anywhere() -> None: """Void has no CUDA toolkit package; portable means portable.""" text = _code_lines() @@ -181,15 +224,31 @@ def test_build_time_deps_cover_setup_py() -> None: text = _template() host = re.search(r"^hostmakedepends=(.*)$", text, re.M) assert host, "template declares no hostmakedepends" - for dep in ("cmake", "pkg-config", "python3-setuptools", "python3-wheel"): + for dep in ( + "cmake", + "pkg-config", + "python3-setuptools", + "python3-setuptools_scm", + "python3-wheel", + ): assert dep in host.group(1), f"{dep} missing from hostmakedepends" make = re.search(r"^makedepends=(.*)$", text, re.M) assert make and "python3-devel" in make.group(1), "compiled extension needs python3-devel" -def test_version_is_injected_for_the_offline_build() -> None: - """The sdist carries no version.txt; setup.py reads PYWHISPERCPP_VERSION.""" - assert "PYWHISPERCPP_VERSION" in _template() +def test_version_is_pinned_through_setuptools_scm() -> None: + """#43: setup() gets no version; setuptools-scm supplies it, so the build must provide it.""" + host = re.search(r"^hostmakedepends=(.*)$", _template(), re.M) + assert host and "python3-setuptools_scm" in host.group( + 1 + ), "without setuptools-scm the build reports version 0.0.0" + assert ( + "SETUPTOOLS_SCM_PRETEND_VERSION=${version}" in _code_lines() + ), "the xbps build root has no git metadata; pin the template version" + installer = (REPO_ROOT / "requirements" / "installer-build.txt").read_text(encoding="utf-8") + assert re.search( + r"^setuptools-scm==", installer, re.M + ), "the installer build tools pin setuptools-scm as well" def test_build_fetches_nothing_beyond_the_sdist() -> None: diff --git a/tests/test_whispercpp_model_info_ext.py b/tests/test_whispercpp_model_info_ext.py index d4db8c9..a1c35f9 100644 --- a/tests/test_whispercpp_model_info_ext.py +++ b/tests/test_whispercpp_model_info_ext.py @@ -580,6 +580,14 @@ class TestDetectGgmlGpuDevices(unittest.TestCase): assert devices is not None self.assertEqual(len(devices), 1) + def test_probe_trusts_runtime_when_version_is_placeholder( + self: "TestDetectGgmlGpuDevices", + ) -> None: + """setuptools' 0.0.0 means the build had no version (#43): probe the runtime.""" + devices = _run_ggml_probe([("Vulkan0", "GPU", 1)], version_string="0.0.0") + assert devices is not None + self.assertEqual(len(devices), 1) + class TestPreferDiscreteVulkanDeviceMore(unittest.TestCase): """Exercise additional Vulkan selection edge cases."""