Migrate the legacy observation history #24

New Issue

2026-08-31T18:24:29Z

xavierk commented

2026-08-31 18:24:29 +00:00

Parent

Implement the Fenris persistent TUI monitoring redesign

Canonical contracts: fenris-redesign spec · acceptance criteria — criterion IDs cited below live in the register.

What to build

The legacy history.jsonl becomes observation history in one interruption-safe step. The import is a single transaction — a scripted kill mid-import leaves the store fully pre- or fully post-migration — and idempotent: a second run no-ops on the legacy-import marker. history.jsonl is the sole authority; hourly.jsonl is diffed and logged but never trusted. Malformed lines are quarantined with a logged count, never silently dropped. Legacy files are renamed *.migrated only after commit and never deleted. One implicit monitoring period opens at the first legacy sample and closes end_cause = migrated at the migration moment; pre-migration hours carry an unknown activity split except directly evidenced facts (a sample means powered on; a DUW delta means writes occurred). Everything imports under a labeled, model-scoped legacy identity (mn-only segments). No synthetic or capacity-derived baseline is ever created for legacy history.

Acceptance criteria

Import is one transaction: interruption leaves the store fully pre- or fully post-migration (ST-6)
Import is idempotent on the legacy-import marker (ST-7)
Legacy files renamed *.migrated only after commit, never deleted (ST-8)
Malformed lines quarantined with a logged count (ST-9)
hourly.jsonl never trusted; mismatches diffed and logged (ST-10)
Implicit period opened at first legacy sample, closed migrated at the migration moment; pre-migration hours unknown except directly evidenced facts (ST-11)
Legacy history imports under a labeled mn-only legacy identity (ID-3); no synthetic baseline ever created (CI-3)

Blocked by

## Parent [Implement the Fenris persistent TUI monitoring redesign](https://git.bongbetic.com/xavierk/Fenris/issues/20) Canonical contracts: [fenris-redesign spec](https://git.bongbetic.com/xavierk/Fenris/src/branch/main/docs/spec/fenris-redesign.md) · [acceptance criteria](https://git.bongbetic.com/xavierk/Fenris/src/branch/main/docs/spec/acceptance-criteria.md) — criterion IDs cited below live in the register. ## What to build The legacy `history.jsonl` becomes observation history in one interruption-safe step. The import is a single transaction — a scripted kill mid-import leaves the store fully pre- or fully post-migration — and idempotent: a second run no-ops on the legacy-import marker. `history.jsonl` is the sole authority; `hourly.jsonl` is diffed and logged but never trusted. Malformed lines are quarantined with a logged count, never silently dropped. Legacy files are renamed `*.migrated` only after commit and never deleted. One implicit monitoring period opens at the first legacy sample and closes `end_cause = migrated` at the migration moment; pre-migration hours carry an unknown activity split except directly evidenced facts (a sample means powered on; a DUW delta means writes occurred). Everything imports under a labeled, model-scoped legacy identity (mn-only segments). No synthetic or capacity-derived baseline is ever created for legacy history. ## Acceptance criteria - [ ] Import is one transaction: interruption leaves the store fully pre- or fully post-migration (ST-6) - [ ] Import is idempotent on the legacy-import marker (ST-7) - [ ] Legacy files renamed `*.migrated` only after commit, never deleted (ST-8) - [ ] Malformed lines quarantined with a logged count (ST-9) - [ ] `hourly.jsonl` never trusted; mismatches diffed and logged (ST-10) - [ ] Implicit period opened at first legacy sample, closed `migrated` at the migration moment; pre-migration hours unknown except directly evidenced facts (ST-11) - [ ] Legacy history imports under a labeled mn-only legacy identity (ID-3); no synthetic baseline ever created (CI-3) ## Blocked by - [Derive hour observations, day aggregates, and monitoring periods](https://git.bongbetic.com/xavierk/Fenris/issues/22) - [Segment observation history by controller identity](https://git.bongbetic.com/xavierk/Fenris/issues/23)

xavierk added the ready-for-agent label 2026-08-31 18:25:17 +00:00

xavierk added this to the Build: Fenris persistent TUI monitoring redesign milestone 2026-08-31 18:25:18 +00:00

xavierk added a new dependency 2026-08-31 18:25:41 +00:00

#22 Derive hour observations, day aggregates, and monitoring periods

xavierk added a new dependency 2026-08-31 18:25:41 +00:00

#23 Segment observation history by controller identity

xavierk added a new dependency 2026-08-31 18:25:46 +00:00

#30 Deliver make install

xavierk referenced this issue

2026-08-31 18:27:02 +00:00

Deliver make install #30

xavierk referenced this issue

2026-09-01 17:00:40 +00:00

Collector tracer bullet: one collection run from fixtures to the observation store #21

xavierk commented

2026-09-01 17:37:56 +00:00

Implementation Complete\n\nImplemented the legacy history migration per spec §3.5 and ADR 0001 §6.\n\n### Changes\n\nNew files:\n- — Idempotent, interruption-safe legacy history.jsonl import\n- — 11 tests covering all acceptance criteria\n\nModified files:\n- — Added table for legacy import marker\n- — Added legacy import check on first run\n- — Updated to include new table in schema verification\n\n### Acceptance Criteria Met\n\n- [x] ST-6: Import is one transaction — interruption leaves store fully pre- or post-migration\n- [x] ST-7: Import is idempotent — second run no-ops on legacy-import marker\n- [x] ST-8: Legacy files renamed only after commit, never deleted\n- [x] ST-9: Malformed lines quarantined with logged count\n- [x] ST-10: never trusted — mismatches diffed and logged\n- [x] ST-11: Implicit period opened at first legacy sample, closed at migration moment\n- [x] ID-3: Legacy history imports under labeled mn-only legacy identity\n- [x] CI-3: No synthetic baseline created for legacy history\n\n### Design Decisions\n\n1. Legacy identity key: Uses prefix to distinguish from post-redesign identity\n2. Metadata table: Added table for store state markers\n3. Hour derivation: Reuses existing for consistent behavior\n4. File renaming: Non-fatal — migration succeeds even if rename fails

## Implementation Complete\n\nImplemented the legacy history migration per spec §3.5 and ADR 0001 §6.\n\n### Changes\n\n**New files:**\n- — Idempotent, interruption-safe legacy history.jsonl import\n- — 11 tests covering all acceptance criteria\n\n**Modified files:**\n- — Added table for legacy import marker\n- — Added legacy import check on first run\n- — Updated to include new table in schema verification\n\n### Acceptance Criteria Met\n\n- [x] **ST-6**: Import is one transaction — interruption leaves store fully pre- or post-migration\n- [x] **ST-7**: Import is idempotent — second run no-ops on legacy-import marker\n- [x] **ST-8**: Legacy files renamed only after commit, never deleted\n- [x] **ST-9**: Malformed lines quarantined with logged count\n- [x] **ST-10**: never trusted — mismatches diffed and logged\n- [x] **ST-11**: Implicit period opened at first legacy sample, closed at migration moment\n- [x] **ID-3**: Legacy history imports under labeled mn-only legacy identity\n- [x] **CI-3**: No synthetic baseline created for legacy history\n\n### Design Decisions\n\n1. **Legacy identity key**: Uses prefix to distinguish from post-redesign identity\n2. **Metadata table**: Added table for store state markers\n3. **Hour derivation**: Reuses existing for consistent behavior\n4. **File renaming**: Non-fatal — migration succeeds even if rename fails

xavierk closed this issue

2026-09-01 17:37:59 +00:00

xavierk referenced this issue from a commit

2026-09-01 17:39:09 +00:00

feat: implement legacy history migration (closes #24)

Sign in to join this conversation.