feat(void): add signed xbps repository pipeline

Build every template in smoke-gate order, index the full repository and
sign packages plus metadata with the RSA4096 key, in the pinned Void
container via podman. Fail-closed without a key: no key file, no repo.

Ticket #29 (needs #27, #28).
This commit is contained in:
soubarnak
2026-11-06 06:29:22 +05:30
parent 1ae453e7d9
commit 9bc04936b6
6 changed files with 489 additions and 0 deletions
+3
View File
@@ -178,3 +178,6 @@ opentree.toml
# Nix build output symlinks # Nix build output symlinks
result result
result-* result-*
# Signed xbps repository staging (ticket #29, local build output)
/publish/
+9
View File
@@ -134,6 +134,15 @@ native-smoke distro="debian:12":
void-smoke: void-smoke:
bash packaging/void/smoke.sh bash packaging/void/smoke.sh
# Build every Void template, index the full repository and sign packages
# plus metadata with the RSA4096 repository key, in the pinned Void
# container via podman (never Docker), as ticket #29 pipelines. Fails
# closed without a key: set XBPS_SIGNING_KEY_FILE (CI writes the
# XBPS_SIGNING_KEY secret to a file first). Needs no host Void install;
# run `just void-smoke` first — the repo assembles tested packages.
void-repo:
bash packaging/void/repo.sh
# Build the canonical sdist, prepare an explicitly unattested local PKGBUILD, # Build the canonical sdist, prepare an explicitly unattested local PKGBUILD,
# then build/install it unchanged on current Arch (needs uv and docker). # then build/install it unchanged on current Arch (needs uv and docker).
# Keep distributions and diagnostics away from other packaging commands' outputs. # Keep distributions and diagnostics away from other packaging commands' outputs.
+51
View File
@@ -12,6 +12,8 @@ packaging/void/
├── README.md this file ├── README.md this file
├── build.sh podman harness: lint + xbps-src build + import smoke ├── build.sh podman harness: lint + xbps-src build + import smoke
├── smoke.sh ticket #28 gate: every template, one command ├── smoke.sh ticket #28 gate: every template, one command
├── repo.sh ticket #29 pipeline: build all, index, sign
├── container-repo.sh ticket #29 container half (index + sign + verify)
├── tool_checksums.txt pinned base image and void-packages commit ├── tool_checksums.txt pinned base image and void-packages commit
└── srcpkgs/ └── srcpkgs/
├── python3-pywhispercpp/ ticket #26: whisper.cpp Python binding ├── python3-pywhispercpp/ ticket #26: whisper.cpp Python binding
@@ -130,3 +132,52 @@ prints `FAIL: <pkgname>` naming which package broke, above it the
harness echoes which stage broke. `tests/test_void_smoke_gate.py` harness echoes which stage broke. `tests/test_void_smoke_gate.py`
asserts offline that the gate stays wired to every `srcpkgs` template, asserts offline that the gate stays wired to every `srcpkgs` template,
so a renamed or added template cannot silently drop out. so a renamed or added template cannot silently drop out.
## Signed repository
`packaging/void/repo.sh` builds every template in smoke-gate order, indexes
the full repository (`x86_64-repodata`), and signs every binary package
(`*.sig2`) plus the repository metadata with the RSA4096 repository key —
one command, same pinned container via podman, no host Void install:
```sh
# Run the smoke gate first: the repo assembles tested packages.
packaging/void/smoke.sh
# or: just void-smoke
printf '%s' "$XBPS_SIGNING_KEY" > /tmp/xbps-sign-key.pem
XBPS_SIGNING_KEY_FILE=/tmp/xbps-sign-key.pem packaging/void/repo.sh
# or: XBPS_SIGNING_KEY_FILE=/tmp/xbps-sign-key.pem just void-repo
```
The pipeline stages `publish/xbps/repo/` (packages, signatures, repodata)
plus `publish/xbps/keys/` (the client `.plist` named by key fingerprint,
and a `FINGERPRINT` file carrying the signer identity and fingerprint for
release notes). `publish/` is local build output and stays uncommitted.
Fail-closed: a missing, unreadable, or non-RSA4096 key aborts before any
container runs, so an unsigned repository can never publish. CI holds the
PEM-encoded key in the `XBPS_SIGNING_KEY` secret and writes it to a file
first (a file keeps newlines intact and the key out of process listings).
**Maintainer key setup (one time):**
```sh
openssl genrsa -out xbps-sign-key.pem 4096
# back the key up somewhere safe, then store it as XBPS_SIGNING_KEY
```
**Client setup:**
```sh
# as root on the Void system (the release notes carry the repo URL):
cp <fingerprint>.plist /var/db/xbps/keys/
echo 'repository=<repo-url-from-release-notes>' > /etc/xbps.d/voculinux.conf
xbps-install -Sy voculinux
```
Answer `Y` when xbps shows the fingerprint prompt on first sync, and check
it matches the published `FINGERPRINT` file. `tests/test_void_signed_repo.py`
asserts offline that the pipeline stays wired to every `srcpkgs` template,
stays podman-only on the pinned inputs, and keeps the fail-closed key
contract.
+129
View File
@@ -0,0 +1,129 @@
#!/bin/sh
# Repository half of packaging/void/repo.sh (ticket #29). Runs inside the
# pinned Void container as root: system setup, the void-packages checkout
# at the pinned commit, then every template built in smoke-gate order as
# the unprivileged builder user (xbps-src refuses root), then index, sign,
# verify and stage as root again.
#
# Not executed directly: repo.sh mounts this file and runs it. VOID_COMMIT,
# PKGS and SIGNEDBY arrive via the environment; the RSA4096 private key is
# mounted read-only at /run/xbps-sign-key.pem (repo.sh validated it before
# any container ran).
#
# Keeping this a real file (instead of an inline `sh -c` string) means
# normal quoting, heredocs, and shellcheck apply.
set -eu
export PATH=/usr/bin:/bin:/usr/sbin:/sbin
KEY=/run/xbps-sign-key.pem
OUT=/mnt/out
REPO="$OUT/repo"
KEYS="$OUT/keys"
[ -r "$KEY" ] || {
echo "container-repo.sh: signing key not mounted at $KEY" >&2
exit 1
}
xbps-install -Suy xbps >/dev/null
xbps-install -y git base-devel xtools binutils zstd openssl >/dev/null
git clone --quiet --depth 1 https://github.com/void-linux/void-packages /void-packages
git -C /void-packages fetch --quiet --depth 1 origin "$VOID_COMMIT"
git -C /void-packages checkout --quiet "$VOID_COMMIT"
cp -r /overlay/. /void-packages/srcpkgs/
id builder >/dev/null 2>&1 || useradd -m builder
chown builder:builder /void-packages
echo "== build every template in order: $PKGS"
cd /void-packages
# shellcheck disable=SC2086
for PKG in $PKGS; do
echo "== xlint $PKG"
su -s /bin/sh builder -c "cd /void-packages && xlint 'srcpkgs/$PKG/template'"
echo "== xbps-src pkg $PKG"
su -s /bin/sh builder -c "cd /void-packages && ./xbps-src pkg '$PKG'"
done
echo "== collect built packages"
mkdir -p "$REPO" "$KEYS"
# shellcheck disable=SC2086
for PKG in $PKGS; do
PKGFILE=$(ls /void-packages/hostdir/binpkgs/"$PKG"-*.x86_64.xbps 2>/dev/null | head -n1)
[ -n "$PKGFILE" ] || {
echo "FAIL: $PKG (built package missing from hostdir/binpkgs)" >&2
exit 1
}
echo "collected: $(basename "$PKGFILE")"
cp "$PKGFILE" "$REPO/"
done
echo "== index the full repository"
# shellcheck disable=SC2086
xbps-rindex -a "$REPO"/*.x86_64.xbps
echo "== sign every binary package"
xbps-rindex -S --privkey "$KEY" --signedby "$SIGNEDBY" "$REPO"/*.x86_64.xbps
echo "== sign the repository metadata"
xbps-rindex -s --privkey "$KEY" --signedby "$SIGNEDBY" "$REPO"
echo "== prove an unsigned install is refused"
set +e
PROBE=$(xbps-install -r /tmp/keyprobe --repository="$REPO" -Sy voculinux </dev/null 2>&1)
PROBE_RC=$?
set -e
if [ "$PROBE_RC" -eq 0 ] || ! printf '%s' "$PROBE" | grep -q "RSA signed"; then
echo "FAIL: unsigned install was not refused — signatures are not enforced" >&2
printf '%s\n' "$PROBE" >&2
exit 1
fi
echo "unsigned install refused as required"
FP=$(printf '%s' "$PROBE" | grep -i Fingerprint | awk '{print $2}' | head -n1)
[ -n "$FP" ] || {
echo "FAIL: could not read the key fingerprint from the refusal" >&2
exit 1
}
echo "key fingerprint: $FP"
echo "== stage the client key"
openssl rsa -in "$KEY" -pubout -out /tmp/repo-pub.pem 2>/dev/null
PUB64=$(base64 -w0 /tmp/repo-pub.pem)
# XML-escape the signer identity: xbps writes &lt;/&gt; itself on import
# and rejects a plist carrying raw angle brackets.
ESCAPED_SIGNEDBY=$(printf '%s' "$SIGNEDBY" | sed -e 's/&/\&amp;/g' -e 's/</\&lt;/g' -e 's/>/\&gt;/g')
{
printf '%s\n' '<?xml version="1.0" encoding="UTF-8"?>'
printf '%s\n' '<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">'
printf '%s\n' '<plist version="1.0">' '<dict>' ' <key>public-key</key>'
printf ' <data>%s</data>\n' "$PUB64"
printf '%s\n' ' <key>public-key-size</key>' ' <integer>4096</integer>' ' <key>signature-by</key>'
printf ' <string>%s</string>\n' "$ESCAPED_SIGNEDBY"
printf '%s\n' '</dict>' '</plist>'
} > "$KEYS/$FP.plist"
{
printf 'signed-by: %s\n' "$SIGNEDBY"
printf 'fingerprint: %s\n' "$FP"
} > "$KEYS/FINGERPRINT"
echo "staged: keys/$FP.plist"
echo "== verify: install the app from the signed repo in a clean rootdir"
VERIFY=/tmp/verify-install
mkdir -p "$VERIFY/var/db/xbps/keys"
# The app's full dependency tree resolves from the official remote, so the
# verify rootdir trusts the official keys too; our repo stays enforced
# because its packages verify only against the staged key below.
cp /var/db/xbps/keys/*.plist "$VERIFY/var/db/xbps/keys/"
cp "$KEYS/$FP.plist" "$VERIFY/var/db/xbps/keys/"
xbps-install -y -r "$VERIFY" \
--repository="$REPO" \
--repository=https://repo-default.voidlinux.org/current \
-Sy voculinux
xbps-query -r "$VERIFY" -l | grep -q "^ii voculinux-" || {
echo "FAIL: voculinux not installed in the verify rootdir" >&2
exit 1
}
echo "verified: voculinux installs from the signed repository"
ls "$REPO"
echo "SIGNED REPO OK: fingerprint $FP"
+97
View File
@@ -0,0 +1,97 @@
#!/bin/sh
# Signed xbps repository pipeline (ticket #29): build every template in
# smoke-gate order, index the full repository, and sign every binary
# package plus the repository metadata with the RSA4096 repository key.
# Runs in the pinned Void container via podman, never Docker; no host
# Void install needed.
#
# Usage: XBPS_SIGNING_KEY_FILE=/path/to/key.pem packaging/void/repo.sh
# or: XBPS_SIGNING_KEY_FILE=/path/to/key.pem just void-repo
#
# Options (all optional):
# --key-file PATH repository private key (default: $XBPS_SIGNING_KEY_FILE)
# --out-dir DIR stage the signed repo here (default: publish/xbps)
# --signed-by "N <m>" signer identity (default: Voculinux <xbps@voculinux.com>)
#
# The private key arrives as a file so CI can write the XBPS_SIGNING_KEY
# secret to disk with newlines intact:
#
# printf '%s' "$XBPS_SIGNING_KEY" > /tmp/xbps-sign-key.pem
# XBPS_SIGNING_KEY_FILE=/tmp/xbps-sign-key.pem packaging/void/repo.sh
#
# Fail-closed: a missing, unreadable, or non-RSA4096 key aborts before
# any container runs, so an unsigned repository can never publish.
set -eu
CDPATH=""
HERE=$(dirname -- "$0")
HERE=$(cd -- "$HERE" && pwd)
REPO_ROOT=$(cd -- "$HERE/../.." && pwd)
KEY_FILE=${XBPS_SIGNING_KEY_FILE:-}
OUT_DIR="$REPO_ROOT/publish/xbps"
SIGNEDBY="Voculinux <xbps@voculinux.com>"
while [ $# -gt 0 ]; do
case "$1" in
--key-file) KEY_FILE=${2:?usage: repo.sh --key-file PATH} ; shift 2 ;;
--out-dir) OUT_DIR=${2:?usage: repo.sh --out-dir DIR} ; shift 2 ;;
--signed-by) SIGNEDBY=${2:?usage: repo.sh --signed-by "Name <mail>"} ; shift 2 ;;
-*) echo "repo.sh: unknown option: $1" >&2 ; exit 1 ;;
*) echo "repo.sh: unexpected argument: $1" >&2 ; exit 1 ;;
esac
done
# Fail closed before any container runs: no key, no repo, no publish.
[ -n "$KEY_FILE" ] || {
echo "repo.sh: no signing key: pass --key-file or set XBPS_SIGNING_KEY_FILE" >&2
echo "repo.sh: CI writes the XBPS_SIGNING_KEY secret to a file first:" >&2
echo "repo.sh: printf '%s' \"\$XBPS_SIGNING_KEY\" > /tmp/xbps-sign-key.pem" >&2
exit 1
}
[ -r "$KEY_FILE" ] && [ -s "$KEY_FILE" ] || {
echo "repo.sh: signing key missing or empty: $KEY_FILE" >&2
echo "repo.sh: refusing to publish an unsigned repository" >&2
exit 1
}
openssl rsa -in "$KEY_FILE" -check -noout >/dev/null 2>&1 || {
echo "repo.sh: not a valid RSA private key: $KEY_FILE" >&2
echo "repo.sh: refusing to publish an unsigned repository" >&2
exit 1
}
openssl rsa -in "$KEY_FILE" -text -noout 2>/dev/null | head -1 | grep -q "4096 bit" || {
echo "repo.sh: signing key is not RSA4096: $KEY_FILE" >&2
echo "repo.sh: generate one with: openssl genrsa -out xbps-sign-key.pem 4096" >&2
exit 1
}
BASE_IMAGE=$(awk '$1 == "base-image" {print $3}' "$HERE/tool_checksums.txt")
VOID_COMMIT=$(awk '$1 == "void-packages" {print $3}' "$HERE/tool_checksums.txt")
[ -n "$BASE_IMAGE" ] && [ -n "$VOID_COMMIT" ] || {
echo "repo.sh: pins missing from tool_checksums.txt" >&2
exit 1
}
# One dependency order for the whole track: the signed repo builds the
# same set in the same order as the smoke gate (the app last), so a
# renamed or added template cannot silently drop out of either.
PKGS=$(awk -F'"' '/^PKGS=/{print $2; exit}' "$HERE/smoke.sh")
[ -n "$PKGS" ] || {
echo "repo.sh: could not read the package order from smoke.sh" >&2
exit 1
}
mkdir -p "$OUT_DIR"
echo "Pulling pinned base image: $BASE_IMAGE"
podman pull "$BASE_IMAGE"
podman run --rm \
-v "$REPO_ROOT/packaging/void/srcpkgs:/overlay:ro" \
-v "$HERE/container-repo.sh:/mnt/container-repo.sh:ro" \
-v "$KEY_FILE:/run/xbps-sign-key.pem:ro" \
-v "$OUT_DIR:/mnt/out" \
-e "VOID_COMMIT=$VOID_COMMIT" \
-e "PKGS=$PKGS" \
-e "SIGNEDBY=$SIGNEDBY" \
"$BASE_IMAGE" /bin/sh /mnt/container-repo.sh
echo "OK: signed repository staged in $OUT_DIR"
+200
View File
@@ -0,0 +1,200 @@
"""Offline policy guards for the signed Void xbps repository (ticket #29).
Spec #16 serves the Void templates from a signed personal xbps
repository: every template builds in dependency order in the pinned Void
container via podman, the repository index carries full repo data, and
packages plus metadata are signed with the RSA4096 repository key. The
pipeline fails closed — a missing, unreadable, or non-RSA4096 key aborts
before any container runs, so an unsigned repository can never publish.
Seams under test (public interfaces, not internals):
- ``packaging/void/repo.sh`` — the host entry: key handling plus the
podman delegation. Exit status and stderr on a missing key are the
fail-closed contract.
- ``packaging/void/container-repo.sh`` — the container half: build-all,
index, sign, verify, stage. Its xbps-rindex invocations are the
signing contract.
- The ``void-repo`` just recipe — how release work and CI call the
pipeline, same shape as the other packaging gates.
- ``packaging/void/README.md`` — the documented maintainer and client
flow, including the CI secret name.
Independent sources of truth: ``packaging/void/srcpkgs/*/template``
owns the template set (a renamed or added template must not silently
drop out), ``packaging/void/tool_checksums.txt`` owns the pins,
``packaging/void/smoke.sh`` owns the dependency order the signed repo
must agree with, and ``openssl`` plus ``xbps-rindex`` own the RSA4096
signing mechanics the container half must invoke. The container run
itself is the proof; these tests are the wiring guard.
"""
import re
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
VOID = REPO_ROOT / "packaging" / "void"
SRC = VOID / "srcpkgs"
REPO_SH = VOID / "repo.sh"
CONTAINER_REPO = VOID / "container-repo.sh"
SMOKE_SH = VOID / "smoke.sh"
PINS = VOID / "tool_checksums.txt"
JUSTFILE = REPO_ROOT / "justfile"
README = VOID / "README.md"
#: The CI secret carrying the PEM-encoded RSA4096 repository private key.
SIGNING_SECRET = "XBPS_SIGNING_KEY"
#: The host-side path variable pointing at the key file. A file (not the
#: secret content on the command line) keeps newlines intact and the key
#: out of process listings.
KEY_FILE_VAR = "XBPS_SIGNING_KEY_FILE"
def _srcpkg_templates() -> list:
"""Every template directory under srcpkgs, in dependency build order."""
return sorted(p.name for p in SRC.iterdir() if (p / "template").is_file())
def _repo_text() -> str:
assert REPO_SH.is_file(), f"missing signed-repo pipeline: {REPO_SH}"
return REPO_SH.read_text(encoding="utf-8")
def _container_text() -> str:
assert (
CONTAINER_REPO.is_file()
), f"missing container half of the signed-repo pipeline: {CONTAINER_REPO}"
return CONTAINER_REPO.read_text(encoding="utf-8")
def _smoke_packages() -> list:
"""The ordered package list the smoke gate builds, from smoke.sh."""
match = re.search(r'^PKGS="([^"]+)"', SMOKE_SH.read_text(encoding="utf-8"), re.M)
assert match, "smoke.sh must declare its ordered PKGS list"
return match.group(1).split()
def test_pipeline_builds_every_template_and_nothing_else() -> None:
"""A renamed or added template cannot silently drop out of the repo."""
text = _repo_text()
assert "smoke.sh" in text or "PKGS" in text, (
"repo.sh must take its package set from the smoke gate order, "
"not a second hand-maintained list"
)
assert set(_smoke_packages()) == set(_srcpkg_templates()), (
f"smoke order covers {_smoke_packages()}, " f"srcpkgs holds {_srcpkg_templates()}"
)
def test_pipeline_builds_the_app_last() -> None:
"""voculinux resolves the other templates from the signed repo."""
assert (
_smoke_packages()[-1] == "voculinux"
), f"the app must build last, order is {_smoke_packages()}"
def test_pipeline_stays_podman_only_with_pinned_inputs() -> None:
"""Podman, never Docker; pins live in tool_checksums.txt, not the flow."""
for path, text in (
(REPO_SH, _repo_text()),
(CONTAINER_REPO, _container_text()),
):
assert "podman" in text or path == CONTAINER_REPO, f"{path.name} must use podman"
assert "docker" not in text.lower().replace(
"never docker", ""
), f"{path.name} must never invoke Docker"
assert "ghcr.io" not in _repo_text(), "the base image pin lives in tool_checksums.txt"
assert PINS.is_file(), f"missing pins file: {PINS}"
def test_pipeline_fails_closed_without_a_key() -> None:
"""No key file means no container, no repo, no publish — never unsigned."""
text = _repo_text()
assert KEY_FILE_VAR in text, f"repo.sh must read the key path from {KEY_FILE_VAR}"
assert (
SIGNING_SECRET in text or "SIGNING_KEY" in text
), "repo.sh must name the CI secret carrying the private key"
for marker in ("exit 1",):
assert marker in text, "the missing-key path must abort non-zero"
assert re.search(
r"[Nn]o .*key|missing .*key|key .*missing", text
), "the missing-key abort must say the key is missing"
def test_pipeline_requires_rsa4096_before_any_container_runs() -> None:
"""The contract is RSA4096: any other key aborts before podman starts."""
text = _repo_text()
assert "4096" in text, "repo.sh must pin the 4096-bit key size"
assert "openssl" in text, "repo.sh must validate the key with openssl"
key_check = text.index("openssl")
first_run = min(i for marker in ("podman pull", "podman run") if (i := text.find(marker)) >= 0)
assert key_check < first_run, "key validation must run before the first podman invocation"
def test_container_signs_packages_and_metadata_with_the_key() -> None:
"""xbps-rindex signs every binary package and the repository metadata."""
text = _container_text()
assert "xbps-rindex -a" in text, "the repo index (full repodata) is missing"
assert "-S" in text or "--sign-pkg" in text, "per-package signing (--sign-pkg) is missing"
assert (
re.search(r"(^|\s)-s(\s|$)", text) or "--sign " in text
), "repository metadata signing (-s/--sign) is missing"
assert "--privkey" in text, "signing must use the provisioned private key"
assert "--signedby" in text, "signing must carry the signer identity"
def test_container_lints_every_template_before_building() -> None:
"""One bad template fails the repo, same as the per-package harness."""
assert (
"xlint" in _container_text()
), "container-repo.sh must xlint each template before building it"
def test_container_names_the_key_fingerprint_for_clients() -> None:
"""Clients pin the key by fingerprint; the pipeline must report it."""
assert (
"ingerprint" in _container_text()
), "container-repo.sh must capture the key fingerprint for release notes"
def test_container_ships_an_importable_client_key() -> None:
"""The staged plist must parse: xbps rejects raw <> in signature-by."""
text = _container_text()
assert ".plist" in text, "container-repo.sh must stage the client .plist"
assert "&lt;" in text and "&gt;" in text, (
"the plist writer must XML-escape the signer identity "
"(xbps shows Fingerprint and refuses the import otherwise)"
)
def test_container_proves_signatures_with_a_clean_rootdir_install() -> None:
"""The pipeline verifies what it signed: a fresh rootdir install."""
text = _container_text()
assert (
"xbps-install" in text and "-r " in text
), "container-repo.sh must prove the signed repo with a rootdir install"
assert "voculinux" in text, "the proof install must be the app, resolving deps from the repo"
def test_just_recipe_calls_the_pipeline_like_other_packaging_gates() -> None:
"""`just void-repo` runs the pipeline script with no shared-output step."""
recipe = re.search(
r"^void-repo:([^\n]*)\n((?:[ \t].*\n|\n)+)",
JUSTFILE.read_text(encoding="utf-8"),
re.MULTILINE,
)
assert recipe, "justfile must define a void-repo recipe"
assert not recipe.group(1).strip(), "void-repo must not take a shared-output prerequisite"
assert "packaging/void/repo.sh" in recipe.group(2), "void-repo must run packaging/void/repo.sh"
def test_readme_documents_the_signed_repo_flow() -> None:
"""Maintainer key setup, CI secret, fail-closed rule, and client setup."""
text = README.read_text(encoding="utf-8")
assert "void-repo" in text or "repo.sh" in text, "README must document the signed-repo command"
assert "4096" in text, "README must document RSA4096 key generation"
assert SIGNING_SECRET in text, "README must name the CI secret"
assert (
"fail" in text.lower() and "closed" in text.lower()
), "README must state the fail-closed rule: no key, no repo"
assert ".plist" in text, "README must document the client key install"