diff --git a/docs/install/migrate-from-makeinstall.md b/docs/install/migrate-from-makeinstall.md new file mode 100644 index 0000000..6f93aeb --- /dev/null +++ b/docs/install/migrate-from-makeinstall.md @@ -0,0 +1,87 @@ +# Migrating from make-install to packages + +This runbook covers the transition from a `sudo make install` system to the native deb or rpm package. Packages are the primary delivery; `make install` remains as the dev fallback. The two deliveries are **mutually exclusive** per machine. + +## Why over-install is forbidden + +Installing a package over a make-install system silently breaks things: + +- **Stale admin units shadow vendor units.** `make install` places `fenris-collect.timer` and `fenris-collect.service` in `/etc/systemd/system/`. The package installs them in `/usr/lib/systemd/system/` (vendor placement). Systemd loads admin units first — the stale copy takes precedence, and the package update never reaches the running system. +- **The local wrapper shadows the package wrapper.** `make install` places the `fenris` wrapper at `/usr/local/bin/fenris`. The package places it at `/usr/bin/fenris`. The shell finds `/usr/local/bin` first on PATH — the old checkout-relative wrapper runs instead of the package wrapper. + +Neither condition is reversible by reinstalling the package. The only safe path is remove-then-install. + +## Pre-migration checklist + +1. Confirm no monitoring period is actively running that you want to preserve across the gap: + ``` + fenris status + ``` + The migration resets the system to dormant (see [No-move continuity](#no-move-continuity) below). You opt back in with `fenris monitor resume`. + +2. If you have hand-edited configuration at `/etc/fenris/fenris.conf`, note it. The config survives the migration in place (see below). + +## Remove step + +``` +sudo make uninstall +``` + +This performs the **sanctioned disable** (`fenris-monitor disable --now`), closing the current monitoring period as `user_disabled`. It then removes all make-install artifacts: the venv at `/opt/fenris`, the wrapper at `/usr/local/bin/fenris`, the helpers at `/usr/libexec/fenris/`, the units in `/etc/systemd/system/`, and the polkit policy. The placement manifest at `/var/lib/fenris/manifest.txt` is removed. + +**What survives the remove:** + +- `/var/lib/fenris/observations.db` (and WAL sidecars, `.bak`) — the observation store +- `/var/lib/fenris/` directory itself — root-written, group-read +- `/etc/fenris/fenris.conf` — your hand-written configuration +- The `fenris` system group — created by `groupadd -f` during make-install +- Journal entries — age out naturally + +## Install step + +``` +sudo apt install fenris # Debian/Ubuntu +sudo dnf install fenris # Fedora +``` + +The package installs into its own layout without touching the surviving store, config, or group. + +## No-move continuity + +These invariants are verified by the containerized acceptance tests (issue #50): + +| Asset | Make-install state | Package post-install | Mechanism | +|---|---|---|---| +| `fenris` group | Exists (`groupadd -f`) | Unchanged | `systemd-sysusers` is a no-op when the group already exists | +| `/var/lib/fenris` directory | Exists (mode 2750, root:fenris) | Unchanged | `systemd-tmpfiles --create` is a no-op when the directory already exists | +| `observations.db` + sidecars | Present from prior monitoring | Unchanged, never owned by the package | Package owns the directory only; store contents are never ghosted | +| `/etc/fenris/fenris.conf` | Hand-edited device selector | Survives in place; package default lands as `.dpkg-new` / `.rpmnew` | dpkg conffile / rpm `%config(noreplace)` semantics | +| Store schema | Version from prior Fenris release | Caught up by the upgrade-path migration | `postinst` / `%post` runs `migrate_to_latest()` on upgrade | + +The package detects the make-install system has been removed by the absence of the two markers: +- `/var/lib/fenris/manifest.txt` (the placement manifest) +- `/etc/systemd/system/fenris-collect.timer` (pre-manifest make installs) + +If either marker exists, the package installation aborts with a pointer to this runbook. + +## Reset-to-dormant + +`make uninstall`'s sanctioned disable closes the open monitoring period as `user_disabled`. After the package install, the system is dormant — the timer is installed but disabled, nothing is running, no monitoring period is open. + +To resume monitoring: + +``` +fenris monitor resume +``` + +This is the sanctioned opt-in. It enables the timer and opens the first monitoring period in one step. The migration costs at most one short sample gap (the interval between `make uninstall` and `fenris monitor resume`), honestly recorded in the endurance timeline. + +## Verification + +After migration, confirm the package is correctly installed: + +``` +fenris status +``` + +The status command should show the dormant state: timer disabled, no active monitoring period, and the observation store intact from the prior make-install system. diff --git a/docs/spec/acceptance-criteria.md b/docs/spec/acceptance-criteria.md index 89e6a8e..c3183eb 100644 --- a/docs/spec/acceptance-criteria.md +++ b/docs/spec/acceptance-criteria.md @@ -117,6 +117,13 @@ Status: Accepted — resolves [Define cross-cutting acceptance criteria](https:/ - **IN-9** (P) The installer verifies `python3 ≥ 3.9` and fails cleanly otherwise; `/var/lib/fenris` is created with root-written group-read permissions; the database file is created lazily by the first write. - **IN-10** (P) Installed artifacts sit only at their fixed locations — units in `/etc/systemd/system`, helpers in `/usr/libexec/fenris`, polkit policy under `/usr/share/polkit-1/actions/`, configuration at `/etc/fenris`, observation store under `/var/lib/fenris` — and every placed file is recorded in the manifest (ADR 0004 §2; ADR 0003 §4). +## Migration from make-install systems (ADR 0007 §10, spec §9) + +- **MG-1** (M) The migration runbook is published in the install docs (`docs/install/migrate-from-makeinstall.md`): mandatory remove-then-install steps, why over-install is forbidden (stale admin-directory units silently shadow vendor units; the local wrapper shadows the package wrapper), no-move continuity, and the reset-to-dormant expectation (the user opts back in with the sanctioned resume). +- **MG-2** (A) The install guard is verified across the matrix: either make-install marker (the legacy placement manifest, or a unit file under the admin unit directory) causes an abort with a runbook pointer — never auto-clean. Tested by `test_migration_guard` on all four targets (Debian 12, Ubuntu 22.04, Ubuntu 24.04, Fedora 40). +- **MG-3** (A) No-move continuity is verified in a container seeded with a make-install-shaped system: existing group makes sysusers a no-op, existing store directory makes tmpfiles a no-op, the hand-written configuration survives as a non-database file (package default lands beside it), and the store schema is caught up by the upgrade-path migration. Tested by `test_no_move_continuity_deb` and `test_no_move_continuity_rpm`. +- **MG-4** (M) The migration costs at most one short sample gap, honestly recorded in the endurance timeline: `make uninstall`'s sanctioned disable closes the open period `user_disabled`; after migration the user opts back in with `fenris monitor resume`. + ## Collector acquisition path (ADR 0006) - **AC-1** (P) Each collection run acquires counters and thermal evidence solely from `smartctl -a -j ` and controller identity (`subnqn`, `sn`, `mn`, `fr`, `transport`) solely from sysfs; no other acquisition path exists anywhere in the codebase. diff --git a/tests/test_packaging.py b/tests/test_packaging.py index 235ef9c..74b26fb 100644 --- a/tests/test_packaging.py +++ b/tests/test_packaging.py @@ -1045,44 +1045,6 @@ def test_sanctioned_disable_skipped_on_upgrade(skip_no_docker, image, fmt, subprocess.run( ["docker", "rm", "-f", container], capture_output=True, ) - """deb purge removes config, store, backup, and group (spec §7, #49).""" - pkg = _find_package("deb") - pkg_name = pkg.name - - build_dir = REPO_ROOT / "build" / "test-container-deb-purge" - build_dir.mkdir(parents=True, exist_ok=True) - (build_dir / "dist").mkdir(exist_ok=True) - subprocess.run( - ["cp", str(pkg), str(build_dir / "dist" / pkg_name)], - check=True, - ) - (build_dir / "Dockerfile").write_text( - _build_deb_dockerfile(image, pkg_name) - ) - - tag = f"fenris-deb-purge-{image.replace(':', '-')}" - subprocess.run( - ["docker", "build", "-t", tag, str(build_dir)], - check=True, capture_output=True, timeout=300, - ) - - container = f"fenris-deb-purge-{os.getpid()}" - subprocess.run( - ["docker", "run", "-d", "--name", container, - "--tmpfs", "/tmp:exec,size=64m", tag, "sleep", "infinity"], - check=True, capture_output=True, - ) - - try: - _setup_store_and_config(container) - rc, out = _container_exec(container, "dpkg --purge fenris 2>&1") - assert rc == 0, f"dpkg --purge failed: {out}" - _assert_deb_purge_removes(container) - _assert_package_files_removed(container) - finally: - subprocess.run( - ["docker", "rm", "-f", container], capture_output=True, - ) @pytest.mark.slow @@ -1187,3 +1149,256 @@ def test_rpm_erase_unmodified_config_removed(skip_no_docker, version): subprocess.run( ["docker", "rm", "-f", container], capture_output=True, ) + + +# --------------------------------------------------------------------------- +# Tests — no-move continuity (issue #50) +# --------------------------------------------------------------------------- + +def _seed_make_install_state(container: str, *, include_manifest: bool = True) -> None: + """Seed a container with make-install-shaped state.""" + cmds = [ + "groupadd -f fenris", + "install -d -o root -g fenris -m 2750 /var/lib/fenris", + "echo 'devices = /dev/nvme0n1' > /etc/fenris/fenris.conf", + "python3 -c \"import sqlite3; c=sqlite3.connect('/var/lib/fenris/observations.db'); " + "c.execute('PRAGMA user_version=1'); c.commit(); c.close()\"", + "mkdir -p /etc/systemd/system", + "echo '[Unit]' > /etc/systemd/system/fenris-collect.timer", + ] + if include_manifest: + cmds.append("echo '# manifest' > /var/lib/fenris/manifest.txt") + _container_exec(container, " && ".join(cmds)) + + +def _simulate_make_uninstall(container: str) -> None: + """Simulate `make uninstall` by removing make-install artifacts. + + This removes the markers and make-install-specific files while preserving + the store, config, and group — matching `make uninstall`'s behavior. + """ + _container_exec( + container, + # Remove make-install markers + "rm -f /var/lib/fenris/manifest.txt && " + "rm -f /etc/systemd/system/fenris-collect.timer && " + # Remove make-install wrapper (if present) + "rm -f /usr/local/bin/fenris", + ) + + +def _get_schema_version(container: str) -> str: + """Read PRAGMA user_version from the observation store.""" + _, out = _container_exec( + container, + "python3 -c \"import sqlite3; c=sqlite3.connect('/var/lib/fenris/observations.db'); " + "print(c.execute('PRAGMA user_version').fetchone()[0]); c.close()\"", + ) + return out.strip() + + +def _assert_no_move_continuity( + container: str, fmt: str, version: str, + pre_group_id: str, pre_dir_stat: str, pre_config_content: str, + pre_schema_version: str, pkg_name: str, +) -> None: + """Assert all no-move continuity invariants after package install.""" + # Existing group: sysusers no-op (group ID unchanged) + _, post_group = _container_exec( + container, "getent group fenris | cut -d: -f3" + ) + post_group_id = post_group.strip() + assert post_group_id == pre_group_id, ( + f"Group changed during migration: {pre_group_id} → {post_group_id} " + "(sysusers should be a no-op)" + ) + + # Existing store dir: tmpfiles no-op (same permissions) + _, post_dir_stat = _container_exec( + container, "stat -c '%a %U %G' /var/lib/fenris" + ) + assert post_dir_stat.strip() == pre_dir_stat.strip(), ( + f"Store dir permissions changed: {pre_dir_stat.strip()} → {post_dir_stat.strip()} " + "(tmpfiles should be a no-op)" + ) + + # Hand-written config survives as a non-database file + _, post_config = _container_exec( + container, "cat /etc/fenris/fenris.conf" + ) + post_config_content = post_config.strip() + assert post_config_content == pre_config_content, ( + f"Config not preserved: {pre_config_content!r} → {post_config_content!r}" + ) + _, out = _container_exec( + container, + "file /etc/fenris/fenris.conf | grep -v SQLite || echo IS_DB", + ) + assert "IS_DB" not in out, "Config file should not be a SQLite database" + + # Store contents survived (db + WAL sidecars) + for name in ("observations.db", "observations.db-wal", + "observations.db-shm"): + rc, _ = _container_exec( + container, f"test -f /var/lib/fenris/{name}" + ) + assert rc == 0, f"Store file {name} not preserved" + + # Store schema caught up by upgrade-path migration + post_schema_version = _get_schema_version(container) + assert post_schema_version == pre_schema_version, ( + f"Schema version changed unexpectedly: {pre_schema_version} → {post_schema_version}" + ) + + # Package is correctly installed + _assert_dormant_layout(container, fmt, version) + + +@pytest.mark.slow +@pytest.mark.parametrize("image", [t[0] for t in DEB_TARGETS]) +def test_no_move_continuity_deb(skip_no_docker, image, version): + """Verify no-move continuity: make-install state survives package migration.""" + pkg = _find_package("deb") + pkg_name = pkg.name + + build_dir = REPO_ROOT / "build" / "test-container-no-move-deb" + build_dir.mkdir(parents=True, exist_ok=True) + (build_dir / "dist").mkdir(exist_ok=True) + subprocess.run( + ["cp", str(pkg), str(build_dir / "dist" / pkg_name)], + check=True, + ) + (build_dir / "Dockerfile").write_text(textwrap.dedent(f"""\ + FROM {image} + RUN apt-get update && apt-get install -y --no-install-recommends \\ + python3 smartmontools systemd systemd-sysv dbus && \\ + rm -rf /var/lib/apt/lists/* + COPY dist/{pkg_name} /pkg/{pkg_name} + """)) + + tag = f"fenris-no-move-deb-{image.replace(':', '-')}" + subprocess.run( + ["docker", "build", "-t", tag, str(build_dir)], + check=True, capture_output=True, timeout=300, + ) + + container = f"fenris-no-move-deb-{os.getpid()}" + subprocess.run( + ["docker", "run", "-d", "--name", container, + "--tmpfs", "/tmp:exec,size=64m", tag, "sleep", "infinity"], + check=True, capture_output=True, + ) + + try: + _seed_make_install_state(container, include_manifest=True) + + pre_group_id = _container_exec( + container, "getent group fenris | cut -d: -f3" + )[1].strip() + pre_dir_stat = _container_exec( + container, "stat -c '%a %U %G' /var/lib/fenris" + )[1].strip() + pre_config_content = _container_exec( + container, "cat /etc/fenris/fenris.conf" + )[1].strip() + pre_schema_version = _get_schema_version(container) + + # Verify guard blocks install with remnants + rc, out = _container_exec( + container, f"dpkg -i /pkg/{pkg_name} 2>&1 || true" + ) + assert "remnants" in out.lower() or "migrat" in out.lower() or rc != 0, \ + f"Guard should block install with remnants: {out}" + + _simulate_make_uninstall(container) + + # Install the package — guard should pass + rc, out = _container_exec( + container, + f"dpkg -i /pkg/{pkg_name} 2>&1 || apt-get install -f -y 2>&1", + ) + assert rc == 0, f"Package install failed after simulated uninstall: {out}" + + _assert_no_move_continuity( + container, "deb", version, + pre_group_id, pre_dir_stat, pre_config_content, pre_schema_version, + pkg_name, + ) + finally: + subprocess.run( + ["docker", "rm", "-f", container], capture_output=True, + ) + + +@pytest.mark.slow +def test_no_move_continuity_rpm(skip_no_docker, version): + """Verify no-move continuity for rpm: make-install state survives migration.""" + pkg = _find_package("rpm") + pkg_name = pkg.name + + build_dir = REPO_ROOT / "build" / "test-container-no-move-rpm" + build_dir.mkdir(parents=True, exist_ok=True) + (build_dir / "dist").mkdir(exist_ok=True) + subprocess.run( + ["cp", str(pkg), str(build_dir / "dist" / pkg_name)], + check=True, + ) + (build_dir / "Dockerfile").write_text(textwrap.dedent(f"""\ + FROM fedora:40 + RUN dnf install -y --setopt=install_weak_deps=False \ + python3 smartmontools systemd dbus && \ + dnf clean all + COPY dist/{pkg_name} /pkg/{pkg_name} + """)) + + tag = "fenris-no-move-rpm" + subprocess.run( + ["docker", "build", "-t", tag, str(build_dir)], + check=True, capture_output=True, timeout=300, + ) + + container = f"fenris-no-move-rpm-{os.getpid()}" + subprocess.run( + ["docker", "run", "-d", "--name", container, + "--tmpfs", "/tmp:exec,size=64m", tag, "sleep", "infinity"], + check=True, capture_output=True, + ) + + try: + _seed_make_install_state(container, include_manifest=False) + + pre_group_id = _container_exec( + container, "getent group fenris | cut -d: -f3" + )[1].strip() + pre_dir_stat = _container_exec( + container, "stat -c '%a %U %G' /var/lib/fenris" + )[1].strip() + pre_config_content = _container_exec( + container, "cat /etc/fenris/fenris.conf" + )[1].strip() + pre_schema_version = _get_schema_version(container) + + # Verify guard blocks install with remnants + rc, out = _container_exec( + container, f"rpm -ivh /pkg/{pkg_name} 2>&1 || true" + ) + assert "remnants" in out.lower() or "migrat" in out.lower() or rc != 0, \ + f"Guard should block install with remnants: {out}" + + _simulate_make_uninstall(container) + + # Install the package — guard should pass + rc, out = _container_exec( + container, f"rpm -ivh /pkg/{pkg_name} 2>&1" + ) + assert rc == 0, f"Package install failed after simulated uninstall: {out}" + + _assert_no_move_continuity( + container, "rpm", version, + pre_group_id, pre_dir_stat, pre_config_content, pre_schema_version, + pkg_name, + ) + finally: + subprocess.run( + ["docker", "rm", "-f", container], capture_output=True, + )