# Installed-build performance baseline protocol (ticket #19)

Locked measurement protocol for the Voculinux port's deferred performance
numbers. The numbers themselves are recorded as evidence on the ticket,
never here: until measured they are protocol, not results.

## Machine class

Void Linux (glibc, x86_64), niri Wayland session, NVIDIA RTX 2050 4 GB,
no CUDA toolkit installed (Vulkan then CPU backend order).

## Prerequisites

- Installed build from `install.sh` (venv or system install), default
  whisper.cpp engine, debug logging available (`--debug`).
- Ladder models on disk with pinned checksums verified against
  `src/vocalinux/utils/model_checksums.txt`: `large-v3-turbo-q8_0`
  (default rung), `large-v3-turbo-q5_0` (first fallback), `small` and
  `small.en` (floor, by language).
- Isolated XDG directories so the pass never touches a live install:
  point `XDG_CONFIG_HOME`, `XDG_DATA_HOME`, `XDG_STATE_HOME` at scratch
  roots and link the verified models under `<data>/vocalinux/models/`.
- Seed `config.json` from the app's own `DEFAULT_CONFIG`: engine
  `whisper_cpp`, the rung under test, keep-alive enabled with the
  minimum 60 s timeout, `first_run` false, update checks off.

## The three metrics

All commands are `scripts/measure_perf_baseline.py` subcommands run
with the installed build's interpreter.

1. **Idle tray memory, both states.** Start with `--debug
   --start-minimized`, sample RSS with `sample-rss --pid`, and read
   VRAM with `nvidia-smi`. Loaded state is right after startup; the
   unloaded state follows the keep-alive timeout (watch the log for
   `reason=idle_keepalive`). RSS does not reflect the unload (freed
   native pages stay mapped); VRAM-used is the state signal.
2. **Wake latency, toggle to capture start.** Drive starts over D-Bus
   (`busctl --user call com.vocalinux.Vocalinux
   /com/vocalinux/Vocalinux com.vocalinux.Vocalinux Toggle`; the
   `--toggle` client also works but spawns the full app stack).
   Parse repeated runs with `parse-log --log`: each `Toggle requested
   in state IDLE` pairs with the next `Opened capture stream`. Report
   steady-state spread separately from the first-capture cost (model
   reload after an idle unload lands on the first toggle).
3. **Transcription real-time factor, fixed sample.** Regenerate the
   sample with `make-sample --out scripts/perf_baseline_sample.wav
   --seed 19` (30 s speech-like babble, 16 kHz mono 16-bit, committed
   next to the script so reruns are bit-identical), then `transcribe
   --model-file … --backend vulkan|cpu --wav … --runs 3`. Report the
   first (cold: shader/pipeline build, one-time) run separately from
   warm runs. CUDA is measured only where a complete toolkit exists;
   otherwise its absence is recorded, not worked around.

## Same-pass extras

- **Per-rung load footprint:** `probe-load --model-file … --backend …`
  reports load wall time, child RSS, and VRAM used while the model is
  held. Free versus total memory is distinguished via `nvidia-smi`
  before/during/after. CPU loads touch no VRAM; their RSS is the
  footprint.
- **Injection latency:** `inject-probe` records the RemoteDesktop
  version-probe result and timing, then a session-establishment attempt
  (fast failure versus hang matters more than the version answer), plus
  the resolved first rung and a no-emission `wtype ""` spawn cost as a
  lower bound. End-to-end key-to-screen latency is deliberately not
  taken here: emitting keystrokes into a live desktop from a background
  run would disturb the session. It belongs to the hardware checklist
  with a dedicated target window.
- **Thresholds for the model ladder:** derive free-memory cutoffs from
  the maximum observed per-rung footprint (transcribe peak, not just
  load) plus explicit headroom for the compositor and other GPU
  clients, and record the derivation next to the numbers.

## Report

`render_markdown_report` (and the JSON the subcommands print) carries
the protocol version, full host facts, and observed numbers only.
`tests/test_perf_baseline_protocol.py` covers the parsing, statistics,
threshold-derivation, and sample-determinism helpers offline.