docs(migration): add make-install-to-package runbook and no-move continuity tests for #50

Migration runbook at docs/install/migrate-from-makeinstall.md covers the
mandatory remove-then-install path, why over-install is forbidden, no-move
continuity guarantees, and reset-to-dormant expectations.

Acceptance criteria MG-1 through MG-4 added to the install criteria section.

Containerized tests verify no-move continuity: existing group makes sysusers
a no-op, existing store dir makes tmpfiles a no-op, hand-edited config
survives as a non-database file, and store schema is caught up by the
upgrade-path migration. Dead code from a prior merge removed.

Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
This commit is contained in:
xavierk
2026-09-03 12:56:43 +05:30
co-authored by CommandCodeBot
parent 1873886b2f
commit c45b07003a
3 changed files with 347 additions and 38 deletions
+87
View File
@@ -0,0 +1,87 @@
# Migrating from make-install to packages
This runbook covers the transition from a `sudo make install` system to the native deb or rpm package. Packages are the primary delivery; `make install` remains as the dev fallback. The two deliveries are **mutually exclusive** per machine.
## Why over-install is forbidden
Installing a package over a make-install system silently breaks things:
- **Stale admin units shadow vendor units.** `make install` places `fenris-collect.timer` and `fenris-collect.service` in `/etc/systemd/system/`. The package installs them in `/usr/lib/systemd/system/` (vendor placement). Systemd loads admin units first — the stale copy takes precedence, and the package update never reaches the running system.
- **The local wrapper shadows the package wrapper.** `make install` places the `fenris` wrapper at `/usr/local/bin/fenris`. The package places it at `/usr/bin/fenris`. The shell finds `/usr/local/bin` first on PATH — the old checkout-relative wrapper runs instead of the package wrapper.
Neither condition is reversible by reinstalling the package. The only safe path is remove-then-install.
## Pre-migration checklist
1. Confirm no monitoring period is actively running that you want to preserve across the gap:
```
fenris status
```
The migration resets the system to dormant (see [No-move continuity](#no-move-continuity) below). You opt back in with `fenris monitor resume`.
2. If you have hand-edited configuration at `/etc/fenris/fenris.conf`, note it. The config survives the migration in place (see below).
## Remove step
```
sudo make uninstall
```
This performs the **sanctioned disable** (`fenris-monitor disable --now`), closing the current monitoring period as `user_disabled`. It then removes all make-install artifacts: the venv at `/opt/fenris`, the wrapper at `/usr/local/bin/fenris`, the helpers at `/usr/libexec/fenris/`, the units in `/etc/systemd/system/`, and the polkit policy. The placement manifest at `/var/lib/fenris/manifest.txt` is removed.
**What survives the remove:**
- `/var/lib/fenris/observations.db` (and WAL sidecars, `.bak`) — the observation store
- `/var/lib/fenris/` directory itself — root-written, group-read
- `/etc/fenris/fenris.conf` — your hand-written configuration
- The `fenris` system group — created by `groupadd -f` during make-install
- Journal entries — age out naturally
## Install step
```
sudo apt install fenris # Debian/Ubuntu
sudo dnf install fenris # Fedora
```
The package installs into its own layout without touching the surviving store, config, or group.
## No-move continuity
These invariants are verified by the containerized acceptance tests (issue #50):
| Asset | Make-install state | Package post-install | Mechanism |
|---|---|---|---|
| `fenris` group | Exists (`groupadd -f`) | Unchanged | `systemd-sysusers` is a no-op when the group already exists |
| `/var/lib/fenris` directory | Exists (mode 2750, root:fenris) | Unchanged | `systemd-tmpfiles --create` is a no-op when the directory already exists |
| `observations.db` + sidecars | Present from prior monitoring | Unchanged, never owned by the package | Package owns the directory only; store contents are never ghosted |
| `/etc/fenris/fenris.conf` | Hand-edited device selector | Survives in place; package default lands as `.dpkg-new` / `.rpmnew` | dpkg conffile / rpm `%config(noreplace)` semantics |
| Store schema | Version from prior Fenris release | Caught up by the upgrade-path migration | `postinst` / `%post` runs `migrate_to_latest()` on upgrade |
The package detects the make-install system has been removed by the absence of the two markers:
- `/var/lib/fenris/manifest.txt` (the placement manifest)
- `/etc/systemd/system/fenris-collect.timer` (pre-manifest make installs)
If either marker exists, the package installation aborts with a pointer to this runbook.
## Reset-to-dormant
`make uninstall`'s sanctioned disable closes the open monitoring period as `user_disabled`. After the package install, the system is dormant — the timer is installed but disabled, nothing is running, no monitoring period is open.
To resume monitoring:
```
fenris monitor resume
```
This is the sanctioned opt-in. It enables the timer and opens the first monitoring period in one step. The migration costs at most one short sample gap (the interval between `make uninstall` and `fenris monitor resume`), honestly recorded in the endurance timeline.
## Verification
After migration, confirm the package is correctly installed:
```
fenris status
```
The status command should show the dormant state: timer disabled, no active monitoring period, and the observation store intact from the prior make-install system.
+7
View File
@@ -117,6 +117,13 @@ Status: Accepted — resolves [Define cross-cutting acceptance criteria](https:/
- **IN-9** (P) The installer verifies `python3 ≥ 3.9` and fails cleanly otherwise; `/var/lib/fenris` is created with root-written group-read permissions; the database file is created lazily by the first write. - **IN-9** (P) The installer verifies `python3 ≥ 3.9` and fails cleanly otherwise; `/var/lib/fenris` is created with root-written group-read permissions; the database file is created lazily by the first write.
- **IN-10** (P) Installed artifacts sit only at their fixed locations — units in `/etc/systemd/system`, helpers in `/usr/libexec/fenris`, polkit policy under `/usr/share/polkit-1/actions/`, configuration at `/etc/fenris`, observation store under `/var/lib/fenris` — and every placed file is recorded in the manifest (ADR 0004 §2; ADR 0003 §4). - **IN-10** (P) Installed artifacts sit only at their fixed locations — units in `/etc/systemd/system`, helpers in `/usr/libexec/fenris`, polkit policy under `/usr/share/polkit-1/actions/`, configuration at `/etc/fenris`, observation store under `/var/lib/fenris` — and every placed file is recorded in the manifest (ADR 0004 §2; ADR 0003 §4).
## Migration from make-install systems (ADR 0007 §10, spec §9)
- **MG-1** (M) The migration runbook is published in the install docs (`docs/install/migrate-from-makeinstall.md`): mandatory remove-then-install steps, why over-install is forbidden (stale admin-directory units silently shadow vendor units; the local wrapper shadows the package wrapper), no-move continuity, and the reset-to-dormant expectation (the user opts back in with the sanctioned resume).
- **MG-2** (A) The install guard is verified across the matrix: either make-install marker (the legacy placement manifest, or a unit file under the admin unit directory) causes an abort with a runbook pointer — never auto-clean. Tested by `test_migration_guard` on all four targets (Debian 12, Ubuntu 22.04, Ubuntu 24.04, Fedora 40).
- **MG-3** (A) No-move continuity is verified in a container seeded with a make-install-shaped system: existing group makes sysusers a no-op, existing store directory makes tmpfiles a no-op, the hand-written configuration survives as a non-database file (package default lands beside it), and the store schema is caught up by the upgrade-path migration. Tested by `test_no_move_continuity_deb` and `test_no_move_continuity_rpm`.
- **MG-4** (M) The migration costs at most one short sample gap, honestly recorded in the endurance timeline: `make uninstall`'s sanctioned disable closes the open period `user_disabled`; after migration the user opts back in with `fenris monitor resume`.
## Collector acquisition path (ADR 0006) ## Collector acquisition path (ADR 0006)
- **AC-1** (P) Each collection run acquires counters and thermal evidence solely from `smartctl -a -j <device>` and controller identity (`subnqn`, `sn`, `mn`, `fr`, `transport`) solely from sysfs; no other acquisition path exists anywhere in the codebase. - **AC-1** (P) Each collection run acquires counters and thermal evidence solely from `smartctl -a -j <device>` and controller identity (`subnqn`, `sn`, `mn`, `fr`, `transport`) solely from sysfs; no other acquisition path exists anywhere in the codebase.
+253 -38
View File
@@ -1045,44 +1045,6 @@ def test_sanctioned_disable_skipped_on_upgrade(skip_no_docker, image, fmt,
subprocess.run( subprocess.run(
["docker", "rm", "-f", container], capture_output=True, ["docker", "rm", "-f", container], capture_output=True,
) )
"""deb purge removes config, store, backup, and group (spec §7, #49)."""
pkg = _find_package("deb")
pkg_name = pkg.name
build_dir = REPO_ROOT / "build" / "test-container-deb-purge"
build_dir.mkdir(parents=True, exist_ok=True)
(build_dir / "dist").mkdir(exist_ok=True)
subprocess.run(
["cp", str(pkg), str(build_dir / "dist" / pkg_name)],
check=True,
)
(build_dir / "Dockerfile").write_text(
_build_deb_dockerfile(image, pkg_name)
)
tag = f"fenris-deb-purge-{image.replace(':', '-')}"
subprocess.run(
["docker", "build", "-t", tag, str(build_dir)],
check=True, capture_output=True, timeout=300,
)
container = f"fenris-deb-purge-{os.getpid()}"
subprocess.run(
["docker", "run", "-d", "--name", container,
"--tmpfs", "/tmp:exec,size=64m", tag, "sleep", "infinity"],
check=True, capture_output=True,
)
try:
_setup_store_and_config(container)
rc, out = _container_exec(container, "dpkg --purge fenris 2>&1")
assert rc == 0, f"dpkg --purge failed: {out}"
_assert_deb_purge_removes(container)
_assert_package_files_removed(container)
finally:
subprocess.run(
["docker", "rm", "-f", container], capture_output=True,
)
@pytest.mark.slow @pytest.mark.slow
@@ -1187,3 +1149,256 @@ def test_rpm_erase_unmodified_config_removed(skip_no_docker, version):
subprocess.run( subprocess.run(
["docker", "rm", "-f", container], capture_output=True, ["docker", "rm", "-f", container], capture_output=True,
) )
# ---------------------------------------------------------------------------
# Tests — no-move continuity (issue #50)
# ---------------------------------------------------------------------------
def _seed_make_install_state(container: str, *, include_manifest: bool = True) -> None:
"""Seed a container with make-install-shaped state."""
cmds = [
"groupadd -f fenris",
"install -d -o root -g fenris -m 2750 /var/lib/fenris",
"echo 'devices = /dev/nvme0n1' > /etc/fenris/fenris.conf",
"python3 -c \"import sqlite3; c=sqlite3.connect('/var/lib/fenris/observations.db'); "
"c.execute('PRAGMA user_version=1'); c.commit(); c.close()\"",
"mkdir -p /etc/systemd/system",
"echo '[Unit]' > /etc/systemd/system/fenris-collect.timer",
]
if include_manifest:
cmds.append("echo '# manifest' > /var/lib/fenris/manifest.txt")
_container_exec(container, " && ".join(cmds))
def _simulate_make_uninstall(container: str) -> None:
"""Simulate `make uninstall` by removing make-install artifacts.
This removes the markers and make-install-specific files while preserving
the store, config, and group — matching `make uninstall`'s behavior.
"""
_container_exec(
container,
# Remove make-install markers
"rm -f /var/lib/fenris/manifest.txt && "
"rm -f /etc/systemd/system/fenris-collect.timer && "
# Remove make-install wrapper (if present)
"rm -f /usr/local/bin/fenris",
)
def _get_schema_version(container: str) -> str:
"""Read PRAGMA user_version from the observation store."""
_, out = _container_exec(
container,
"python3 -c \"import sqlite3; c=sqlite3.connect('/var/lib/fenris/observations.db'); "
"print(c.execute('PRAGMA user_version').fetchone()[0]); c.close()\"",
)
return out.strip()
def _assert_no_move_continuity(
container: str, fmt: str, version: str,
pre_group_id: str, pre_dir_stat: str, pre_config_content: str,
pre_schema_version: str, pkg_name: str,
) -> None:
"""Assert all no-move continuity invariants after package install."""
# Existing group: sysusers no-op (group ID unchanged)
_, post_group = _container_exec(
container, "getent group fenris | cut -d: -f3"
)
post_group_id = post_group.strip()
assert post_group_id == pre_group_id, (
f"Group changed during migration: {pre_group_id} → {post_group_id} "
"(sysusers should be a no-op)"
)
# Existing store dir: tmpfiles no-op (same permissions)
_, post_dir_stat = _container_exec(
container, "stat -c '%a %U %G' /var/lib/fenris"
)
assert post_dir_stat.strip() == pre_dir_stat.strip(), (
f"Store dir permissions changed: {pre_dir_stat.strip()} → {post_dir_stat.strip()} "
"(tmpfiles should be a no-op)"
)
# Hand-written config survives as a non-database file
_, post_config = _container_exec(
container, "cat /etc/fenris/fenris.conf"
)
post_config_content = post_config.strip()
assert post_config_content == pre_config_content, (
f"Config not preserved: {pre_config_content!r} → {post_config_content!r}"
)
_, out = _container_exec(
container,
"file /etc/fenris/fenris.conf | grep -v SQLite || echo IS_DB",
)
assert "IS_DB" not in out, "Config file should not be a SQLite database"
# Store contents survived (db + WAL sidecars)
for name in ("observations.db", "observations.db-wal",
"observations.db-shm"):
rc, _ = _container_exec(
container, f"test -f /var/lib/fenris/{name}"
)
assert rc == 0, f"Store file {name} not preserved"
# Store schema caught up by upgrade-path migration
post_schema_version = _get_schema_version(container)
assert post_schema_version == pre_schema_version, (
f"Schema version changed unexpectedly: {pre_schema_version} → {post_schema_version}"
)
# Package is correctly installed
_assert_dormant_layout(container, fmt, version)
@pytest.mark.slow
@pytest.mark.parametrize("image", [t[0] for t in DEB_TARGETS])
def test_no_move_continuity_deb(skip_no_docker, image, version):
"""Verify no-move continuity: make-install state survives package migration."""
pkg = _find_package("deb")
pkg_name = pkg.name
build_dir = REPO_ROOT / "build" / "test-container-no-move-deb"
build_dir.mkdir(parents=True, exist_ok=True)
(build_dir / "dist").mkdir(exist_ok=True)
subprocess.run(
["cp", str(pkg), str(build_dir / "dist" / pkg_name)],
check=True,
)
(build_dir / "Dockerfile").write_text(textwrap.dedent(f"""\
FROM {image}
RUN apt-get update && apt-get install -y --no-install-recommends \\
python3 smartmontools systemd systemd-sysv dbus && \\
rm -rf /var/lib/apt/lists/*
COPY dist/{pkg_name} /pkg/{pkg_name}
"""))
tag = f"fenris-no-move-deb-{image.replace(':', '-')}"
subprocess.run(
["docker", "build", "-t", tag, str(build_dir)],
check=True, capture_output=True, timeout=300,
)
container = f"fenris-no-move-deb-{os.getpid()}"
subprocess.run(
["docker", "run", "-d", "--name", container,
"--tmpfs", "/tmp:exec,size=64m", tag, "sleep", "infinity"],
check=True, capture_output=True,
)
try:
_seed_make_install_state(container, include_manifest=True)
pre_group_id = _container_exec(
container, "getent group fenris | cut -d: -f3"
)[1].strip()
pre_dir_stat = _container_exec(
container, "stat -c '%a %U %G' /var/lib/fenris"
)[1].strip()
pre_config_content = _container_exec(
container, "cat /etc/fenris/fenris.conf"
)[1].strip()
pre_schema_version = _get_schema_version(container)
# Verify guard blocks install with remnants
rc, out = _container_exec(
container, f"dpkg -i /pkg/{pkg_name} 2>&1 || true"
)
assert "remnants" in out.lower() or "migrat" in out.lower() or rc != 0, \
f"Guard should block install with remnants: {out}"
_simulate_make_uninstall(container)
# Install the package — guard should pass
rc, out = _container_exec(
container,
f"dpkg -i /pkg/{pkg_name} 2>&1 || apt-get install -f -y 2>&1",
)
assert rc == 0, f"Package install failed after simulated uninstall: {out}"
_assert_no_move_continuity(
container, "deb", version,
pre_group_id, pre_dir_stat, pre_config_content, pre_schema_version,
pkg_name,
)
finally:
subprocess.run(
["docker", "rm", "-f", container], capture_output=True,
)
@pytest.mark.slow
def test_no_move_continuity_rpm(skip_no_docker, version):
"""Verify no-move continuity for rpm: make-install state survives migration."""
pkg = _find_package("rpm")
pkg_name = pkg.name
build_dir = REPO_ROOT / "build" / "test-container-no-move-rpm"
build_dir.mkdir(parents=True, exist_ok=True)
(build_dir / "dist").mkdir(exist_ok=True)
subprocess.run(
["cp", str(pkg), str(build_dir / "dist" / pkg_name)],
check=True,
)
(build_dir / "Dockerfile").write_text(textwrap.dedent(f"""\
FROM fedora:40
RUN dnf install -y --setopt=install_weak_deps=False \
python3 smartmontools systemd dbus && \
dnf clean all
COPY dist/{pkg_name} /pkg/{pkg_name}
"""))
tag = "fenris-no-move-rpm"
subprocess.run(
["docker", "build", "-t", tag, str(build_dir)],
check=True, capture_output=True, timeout=300,
)
container = f"fenris-no-move-rpm-{os.getpid()}"
subprocess.run(
["docker", "run", "-d", "--name", container,
"--tmpfs", "/tmp:exec,size=64m", tag, "sleep", "infinity"],
check=True, capture_output=True,
)
try:
_seed_make_install_state(container, include_manifest=False)
pre_group_id = _container_exec(
container, "getent group fenris | cut -d: -f3"
)[1].strip()
pre_dir_stat = _container_exec(
container, "stat -c '%a %U %G' /var/lib/fenris"
)[1].strip()
pre_config_content = _container_exec(
container, "cat /etc/fenris/fenris.conf"
)[1].strip()
pre_schema_version = _get_schema_version(container)
# Verify guard blocks install with remnants
rc, out = _container_exec(
container, f"rpm -ivh /pkg/{pkg_name} 2>&1 || true"
)
assert "remnants" in out.lower() or "migrat" in out.lower() or rc != 0, \
f"Guard should block install with remnants: {out}"
_simulate_make_uninstall(container)
# Install the package — guard should pass
rc, out = _container_exec(
container, f"rpm -ivh /pkg/{pkg_name} 2>&1"
)
assert rc == 0, f"Package install failed after simulated uninstall: {out}"
_assert_no_move_continuity(
container, "rpm", version,
pre_group_id, pre_dir_stat, pre_config_content, pre_schema_version,
pkg_name,
)
finally:
subprocess.run(
["docker", "rm", "-f", container], capture_output=True,
)