Fenris/docs/research/controller-identity.md

Controller identity for observation-history segmentation

Research for Verify the controller identity that segments observation history.

Decision

Record the subsystem NQN as exposed by the Linux kernel — normalized, trailing-space stripped — as the controller-segment identity key:

identity_key = strip(subnqn)                # /sys/class/nvme-subsystem/…/subsysnqn,
                                            # identical to /sys/class/nvme/nvmeX/subsysnqn
fallback:    "nqn.2014.08.org.nvmexpress:" + hex4(vid) + hex4(ssvid)
             + raw20(sn) + raw40(mn)        # byte-for-byte the kernel's synthesized NQN
last resort: strip(mn) + "|" + strip(sn)    # when only smartctl-style fields exist

Store the raw subnqn, sn, mn strings (normalized) plus fr (firmware revision) as segment metadata, never fr inside the key: firmware revision is the one mandatory field that legitimately changes on the same drive (Base Spec 2.0e §5.17.2.1: FR is the currently active firmware revision). Namespace identifiers (NGUID, EUI-64, UUID) are excluded from the key: they are namespace-scoped while the SMART counters being segmented are controller-scoped, and each may be absent or reused. When the key is blank, record an empty key with a degraded marker and rely on the DUW-monotonic rule; a same-model same-serial replacement is unobservable by any identifier and is caught — as a boundary, not an identity change — by the DUW-decrease rule of ADR 0001.

This key satisfies the ticket's asymmetry: a drive replacement changes subnqn (real NQNs are unique per subsystem; kernel-generated ones embed serial+model), while firmware quirks and counter resets on the same drive leave it unchanged — counter discontinuities are already ADR 0001's second segmentation axis.

What each candidate is

Candidate	Spec definition	Scope	Mandatory	Verdict for the key
SN + MN	ASCII strings assigned by the vendor in Identify Controller, bytes 23:04 and 63:24; §4.3 shows them left-justified and space-padded (2.0e §5.17.2.1, Identify Controller data structure, §4.3 Identifier Format and Layout)	NVM subsystem	Mandatory for I/O and Admin controllers	Core of the fallback; uniqueness explicitly not guaranteed by the spec
FR	Currently active firmware revision, ASCII, bytes 71:64 (2.0e §5.17.2.1)	Domain (subsystem)	Mandatory	Never in the key — it is meant to change on the same drive
SUBNQN	NVM Subsystem NQN, UTF-8 null-terminated, bytes 1023:768; mandatory if the controller is ≥ 1.2.1, otherwise may be all zero (2.0e §5.17.2.1)	NVM subsystem (shared by all its controllers)	Mandatory ≥ 1.2.1, optional below	Chosen key; the spec says hosts should use it as the subsystem's unique identifier
NGUID / EUI-64	IEEE-based identifiers in Identify Namespace, bytes 119:104 / 127:120 (§4.3.4, §4.3.5)	Namespace	Optional; may both be zero	Rejected — wrong scope, may be missing, may be reused
CNTLID	Controller ID, unique only within a subsystem (§4.5.1)	Controller	Mandatory	Rejected — not unique across subsystems (this host's drive reports 0)

A note on the PDF: figure numbers in the table of contents of the 2.0e revision are offset from the body captions (e.g. the SN/MN figure is "Figure 128" in the TOC but "Figure 130" in the body), so this document cites section numbers, which are stable.

Scope: the counters being segmented are controller-scoped

SMART / Health Information (LID 02h) is scope Controller (mandatory) with an optional namespace view (2.0e §5.16.1, Get Log Page – Log Page Identifiers). Section 5.16.1.3: "The information provided is over the life of the controller and is retained across power cycles"; hosts request the controller log page with NSID FFFFFFFFh/0h, the per-namespace view is optional (LPA bit 0), and "the controller log page and namespaces specific log page contain identical information" in 2.0e (§5.16.1.3). Data Units Written therefore accumulates per controller/subsystem, not per namespace — the identity key must be subsystem-scoped, and SN/MN/SUBNQN are all defined as NVM-subsystem fields (§5.17.2.1: SN/MN are "the serial number/model number for the NVM subsystem"; the Persistent Event Log repeats that its SN/MN/SUBNQN copies are the same subsystem values).

NGUID and EUI-64 live in Identify Namespace, not the controller structure (§4.3.4, §4.3.5; libnvme documents them on struct nvme_id_ns, while sn/mn/fr live on struct nvme_id_ctrl and subnqn on the same structure (types.h)). Segmentation keyed on a namespace identifier would split or merge history whenever namespaces are attached, detached, formatted, or recreated, while the DUW counter — the thing being differenced — sails on unchanged. The spec's own namespace-identity guidance (§3.2.1.6: NSIDs "may change across power off conditions"; to detect the same namespace use UUID, NGUID, or EUI-64) addresses a different problem from ours.

Stability verdicts (reboots, firmware, replacement)

1. Reboots, same drive — SN, MN, SUBNQN are stable: they are vendor-assigned subsystem fields, and an NQN "is permanent for the lifetime of the host or NVM subsystem" (§4.5). SMART data "is retained across power cycles" (§5.16.1.3).
2. Firmware update, same drive — FR changes by definition (it reports the active revision). The spec guarantees persistence of SMART data across power cycles, and says nothing about firmware commits resetting counters — vendor behavior, which is precisely why ADR 0001 keeps the DUW-monotonic rule as an independent boundary. Identity (SN/MN/SUBNQN) is not specified to change with firmware; keeping FR out of the key means a firmware update never quarantines history as a "new drive", and a firmware-induced counter reset is caught by the DUW rule instead.
3. Drive replacement, different model — every candidate changes.
4. Drive replacement, identical model — MN unchanged; SN changes if vendor serials are unique; SUBNQN changes because both real NQNs (empirically this host's Micron embeds the serial: nqn.2016-08.com.micron:nvme:nvm-subsystem-sn-233542F44436) and kernel-generated NQNs (which concatenate SN and MN, drivers/nvme/host/core.c nvme_init_subnqn) derive from the serial.
5. NGUID/EUI-64 under namespace churn — not stable in the needed sense: if the UIDREUSE bit is 0 "a controller may reuse a non-zero NGUID/EUI64 value for a new namespace after the original namespace using the value has been deleted" (§4.5.1); libnvme's own field docs say the values hold only "throughout the life of the namespace", "preserved across namespace and controller operations" (types.h nguid/eui64).

Availability and known pathologies

• SN/MN: mandatory for I/O and Admin controllers (§5.17.2.1) and exposed by the kernel since 4.5 (sysfs-nvme: /sys/class/nvme/nvmeX/{model,serial,firmware_rev}). But uniqueness is disclaimed: "The mechanism used by the vendor to assign Serial Number and Model Number values to ensure uniqueness is outside the scope of this specification" (§4.5.1). Duplicate serials across units are therefore spec-legal.
• SUBNQN: zero on pre-1.2.1 subsystems (§5.17.2.1; the Persistent Event Log likewise defines the not-supported case as all bytes cleared to 0h), and some real devices report garbage: the kernel carries NVME_QUIRK_IGNORE_DEV_SUBNQN for, among others, Intel P4500/P4600, Intel 760p/Pro 7600p, and a Silicon Motion device (pci.c quirk table, nvme.h flag definition). This is why Fenris should read the kernel-exposed value rather than the raw Identify bytes: when the device NQN is missing, invalid, or quirk-ignored, nvme_init_subnqn synthesizes nqn.2014.08.org.nvmexpress:{vid}{ssvid}{sn}{mn} — mirroring the spec's own construction for pre-1.2.1 subsystems (§4.5.1, "NQN Construction for Older NVM Subsystems", which composes the NQN starting string, VID, SSVID, SN, MN) — so /sys/.../subsysnqn is populated on every kernel ≥ 4.8 for every controller (sysfs-nvme subsysnqn entry, added 4.8). The kernel also uses the NQN as the subsystem key when building multipath heads (core.c: subsystems are matched by subsys->subnqn).
• Virtual controllers / blank serials: the kernel's own NVMe target (nvmet, the loop transport) sets model number to the literal "Linux" and generates a random serial per subsystem "as our controllers are ephemeral" (target/core.c nvmet_subsys_alloc, nvmet.h NVMET_DEFAULT_CTRL_MODEL); Identify then reports those values verbatim (target/admin-cmd.c). For such devices model|serial is unstable across target re-creation, while the NQN is the configured subsystem name. No identifier can make an ephemeral virtual drive look like stable hardware; the degraded marker covers it.
• NGUID/EUI-64: optional — the kernel sysfs attributes are documented as "Hidden if all zeros" (sysfs-nvme nguid/eui entries), and the spec requires only that at least one of EUI64/NGUID/UUID be valid at namespace creation (§4.5.1).

The key, normalization, and failure handling

1. Primary key: strip(subnqn) read from the kernel path (via libnvme; see next section). Values are ASCII/UTF-8 with code values 0x20–0x7E, left-justified and space-padded per the spec's string rules (§1.4.2 ASCII/UTF-8 string conventions); normalize by stripping trailing (and leading) spaces only. No case folding: NQNs are compared "as binary strings without any text processing (e.g., case folding)" (§4.5, NQN processing rules), and SN/MN have no canonical case either.
2. Fallback ladder (defensive; on Linux ≥ 4.8 the kernel fallback already fires before Fenris ever sees an empty value): (a) device-provided SUBNQN; (b) the kernel's composite nqn.2014.08.org.nvmexpress:{vid}{ssvid}{sn}{mn} built from Identify — the kernel spells the date with dots and concatenates the raw fixed-width SN and MN, "slightly different from the format specified" in §4.5.1's NQN construction "for historic reasons" (core.c nvme_init_subnqn); Fenris's fallback reproduces the kernel spelling so a fallback-built key equals the sysfs value byte-for-byte; (c) strip(mn)|strip(sn). Every rung changes on drive replacement and survives firmware updates; the ladder exists so a missing rung never yields a blank key from a perfectly good physical drive.
3. Blank key (all components empty — e.g. a virtual controller reporting nothing): record identity_key = "" plus a degraded marker and segment by DUW monotonicity alone; surface as a fact, never fabricate uniqueness (matches ADR 0005's "facts, not alerts" posture).
4. Duplicate keys across physical units are undetectable by construction when both units report identical SN/MN/SUBNQN. The mitigation already exists in ADR 0001: a replacement drive almost certainly reports a lower DUW than the accumulated history, and any DUW decrease forces a segment boundary regardless of identity. Write deltas remain quarantined even though identity cannot distinguish the units.
5. Recorded metadata per segment: normalized subnqn, sn, mn, fr, plus transport — diagnostics for humans, not key components.

How the collector obtains the key via libnvme

Three concrete paths, all first-party:

1. libnvme Python bindings (from libnvme import nvme, official SWIG bindings, "python bindings for libnvme"). nvme.root() scans sysfs (nvme.i: nvme_root() calls nvme_scan); controller objects expose model, serial, firmware, subsysnqn, name, sysfs_dir as attributes (nvme.i struct nvme_ctrl attributes); namespace objects expose nsid, nguid, eui64, uuid (nvme.i struct nvme_ns). These getters read sysfs via nvme_get_ctrl_attr (tree.c populates ctrl fields from sysfs attributes), and __nvme_get_attr strips the trailing newline and trailing spaces and returns NULL when the result is empty (linux.c L526–552) — i.e., the bindings deliver exactly the normalization this decision requires, and a blank field arrives as None.
2. Admin passthrough for raw Identify: nvme_ctrl_identify(c, &id) fills struct nvme_id_ctrl (man page: "Issues an 'identify controller' command"), whose sn[20]/mn[40]/fr[8] and subnqn[256] members are documented in the nvme_id_ctrl(2) man page (subnqn member); namespace identifiers come from nvme_ns_identify filling struct nvme_id_ns (tree.h nvme_ns_identify). Here the collector must strip trailing spaces itself.
3. nvme-cli JSON (built on libnvme): nvme id-ctrl -o json emits sn, mn, fr, cntlid with strings copied verbatim including padding (nvme-print-json.c L409–L422) plus subnqn, which is included only when non-empty (L522–L523); nvme list -o json emits per device DevicePath, Firmware, ModelNumber, SerialNumber (v2.16 json_list_item_obj). So stripping trailing spaces is the collector's job in both nvme-cli paths.

What libnvme/nvme-cli offer beyond the current smartctl -j collector: subnqn (the chosen key), cntlid, vid/ssvid, and the namespace identifiers — smartmontools' NVMe JSON device section carries model_name, serial_number, firmware_version (from id_ctrl.mn/sn/fr, nvmeprint.cpp print_drive_info) and no NQN. smartmontools also trims the strings it copies (utility.cpp format_char_array strips leading/trailing spaces), so normalization is compatible across both collectors.

Empirical check (one real drive, sysfs only)

$ cat /sys/class/nvme/nvme0/{model,serial,firmware_rev,subsysnqn}
Micron_2400_MTFDKBA512QFM<spaces to 40>   # sysfs preserves Identify padding
233542F44436<spaces to 20>
V3MA001<space>                            # FR padded to 8
nqn.2016-08.com.micron:nvme:nvm-subsystem-sn-233542F44436
$ cat /sys/block/nvme0n1/{nguid,eui,wwid}
00000000-0000-0001-00a0-752342f44436
00 a0 75 01 42 f4 44 36
eui.000000000000000100a0752342f44436

Confirms: raw sysfs keeps the spec's trailing-space padding (strip before storing); the vendor NQN embeds the serial; NGUID/EUI-64 are present here but are per-namespace; /sys/class/nvme-subsystem/nvme-subsys0/{model,serial,firmware_rev,subsysnqn} carries the same subsystem-level values (sysfs-nvme nvme-subsystem entries, added 4.15). The kernel's own namespace wwid uses the priority ladder uuid.{UUID} → eui.{NGUID} → eui.{EUI64} → nvme.{VID}-{SERIAL}-{MODEL}-{NSID} (sysfs-nvme wwid) — a first-party precedent for a serial+model fallback when better identifiers are absent.

What ADR 0001's segment and migration rules must assume

1. Legacy history.jsonl cannot carry a full identity. The legacy collector records device (/dev/nvme0), model (smartctl model_name), capacity_bytes, and the counters — no serial, no NQN, no firmware (fenris.py sample(): the identity-adjacent fields are device and model only). Migration may therefore only import legacy samples under a model-scoped legacy identity, explicitly labeled incomplete. It cannot prove the legacy samples came from the current drive, cannot detect a same-model swap that happened before migration, and must not backfill serials retroactively.
2. The first new-version collection run starts a new controller segment for the monitored drive, recording full identity_key plus sn/mn/fr/subnqn metadata, because the legacy and new identities are not comparable — the identity change is an epistemic boundary, not a detected drive swap. Imported legacy rows keep their legacy segment; the DUW-monotonic rule already governs deltas within it.
3. Two independent segmentation axes stay independent: identity-key change ⇒ boundary (drive replacement); DUW decrease ⇒ boundary (counter reset, firmware quirk, or same-identity replacement). Neither implies the other; ADR 0001's controller_segments wording ("boundaries where controller identity changes or DUW decreases") already encodes this, and this research fixes "controller identity" to mean the normalized kernel-exposed subsystem NQN with the fallback ladder above.
4. The key is a string with rules, not just a field choice: normalization (space-stripping, no case folding) must be specified once and applied at write time, or the same drive could split its own history across a collector implementation change (smartctl, nvme-cli, and libnvme deliver differently-trimmed values, as shown above).

Newly surfaced questions

• Should the store record vid/ssvid/cntlid alongside segment metadata now (cheap) for future composite needs, or keep segments minimal?
• Should a detected blank/degraded identity degrade the projection confidence category (it weakens the "identity/counter discontinuity" axis of the Unavailable state)?
• Does the collector pin to one acquisition path (libnvme bindings vs nvme list -o json vs continued smartctl -j plus a sysfs read for subnqn) — an operational choice this research does not settle?