xavierk/Fenris
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases 11 Wiki Activity

Deepen observation-history publication, local-day evidence, and activity selection #95

New Issue
Closed
opened 2026-09-27 19:54:14 +00:00 by xavierk · 1 comment
xavierk commented 2026-09-27 19:54:14 +00:00
Owner
Copy Link
Copy Source

Problem Statement

Fenris can show a plausible but incorrect picture of monitored drive activity. Local-day read and write totals currently derive from whole UTC hours: an hour straddling a local midnight can be counted on both adjacent dates, while a measured interval spanning UTC hours can disappear from the local total even when it belongs to one local date. Publication is split across commits, derivation failures can report collection success, repair and normal collection do not consistently apply the same evidence rules, and the advertised 14-day detail pruning is not reached by normal collection. Meanwhile, overlapping activity selection state in the TUI makes refreshing, browsing, and inspecting a live point fragile. A drive owner needs trustworthy measured volume, an honest account of incomplete evidence and failures, and activity navigation that stays put when requested without hiding new live data.

Solution

Keep one observation store and the existing collector-only write path, but deepen three modules together. A collection-focused observation-history publication module owns recovery, validation, consistent publication and safe retention, with local-day evidence attribution inside its implementation. A coherent read-only observation-history reader makes published evidence, pending publication, status and projection inputs agree for both TUI and CLI. An in-process activity selection module owns the user's live-following, inspected-point and historical browsing choices, leaving Textual to render. Deliver local-day evidence plus publication/recovery/retention together first; deliver navigation second.

Present known read and write volumes separately as known amounts, incomplete when some measured volume cannot be assigned to a date. Retain ambiguous midnight-spanning read/write volume exactly once as shared local-day evidence, separate from both adjacent dates' known totals. Never divide, invent, double-count or quietly present a partial amount as a full-day total. Valid observations whose derivation fails survive privately as pending publication, while readers continue to see the last consistent publication and explain the pending condition. A failed publication is not a successful collection. Historical summaries with insufficient surviving evidence become unavailable at local-day precision without destroying trustworthy UTC history.

User Stories

  1. As a drive owner, I want each local-day write amount to contain only writes supported for that date, so that I can trust its contribution to observed activity.
  2. As a drive owner, I want each local-day read amount kept distinct from writes, so that I do not mistake reads for endurance consumption.
  3. As a drive owner, I want a measured interval that crosses a UTC-hour boundary but lies inside one local day to count for that day, so that short bursts are not lost.
  4. As a drive owner, I want a measured interval crossing local midnight kept once as shared evidence, so that neither adjacent day claims volume it cannot prove.
  5. As a drive owner, I want the known amount shown with an explicit incomplete label when a date has ambiguous or missing evidence, so that a partial measurement does not look like a complete daily total.
  6. As a drive owner, I want shared read and write volume visibly explained apart from known daily amounts, so that I understand why a day is incomplete.
  7. As a drive owner, I want measured zero distinguished from absent evidence, so that an idle drive is not mistaken for missing observations.
  8. As a drive owner, I want today's amounts labelled as so far, so that a live partial day is not presented as finished.
  9. As a drive owner in a half-hour-offset timezone, I want local midnight measured at the actual boundary, so that adjacent dates do not share the same whole UTC hour as known activity.
  10. As a drive owner crossing a daylight-saving transition, I want the actual 23- or 25-hour local day respected, so that coverage and volume are not based on a fixed day length.
  11. As a drive owner browsing a repeated local clock hour, I want points distinguished by their actual instants and timezone labels, so that two observations do not collapse into one.
  12. As a drive owner after a timezone change, I want earlier local-day summaries to retain their recorded timezone and midnight boundaries, so that history is not silently reinterpreted.
  13. As a drive owner with old coarse observation history, I want unprovable local-day totals identified as unavailable, so that migration does not manufacture precise dates.
  14. As a drive owner, I want old measured evidence preserved even when a legacy local total cannot be trusted, so that trustworthy UTC history remains useful.
  15. As a drive owner, I want an applicable usage-adjusted theoretical lifespan computed only from published, compatible observation history, so that pending counters cannot change the outlook prematurely.
  16. As a drive owner, I want projection confidence and the complete-observation-day gate to retain their existing meaning, so that the new activity evidence does not invent certainty.
  17. As a drive owner, I want controller changes and counter discontinuities respected during derivation, so that writes from distinct controller segments are not combined.
  18. As a drive owner who deliberately disables monitoring, I want that time excluded as before, so that missing activity is not mistaken for a monitoring period.
  19. As a drive owner, I want readings within a monitoring period that cannot be classified to remain unknown, so that gaps are not silently filled.
  20. As a drive owner, I want a successful collection to update the relevant read views together, so that a recent sample cannot disagree with its activity summaries.
  21. As a drive owner, I want a derivation failure reported as a failed collection with a clear pending explanation, so that I do not trust partially published measurements.
  22. As a drive owner, I want valid acquired observations retained for retry after a derivation failure, so that real measured evidence is not discarded.
  23. As a drive owner, I want the most recent published observation, rather than a pending one, to determine displayed freshness, so that unpublished work cannot masquerade as fresh history.
  24. As a drive owner, I want pending observations excluded from drive facts, identity interpretation and projection inputs, so that every visible measurement describes one coherent publication.
  25. As a drive owner, I want each scheduled collection to retry pending publication before considering new acquisition, so that older unfinished evidence is not skipped.
  26. As a drive owner, I want recovery after a restart to avoid counting any interval twice, so that repeated runs conserve measured volume.
  27. As a drive owner, I want an invalid acquired observation rejected without retaining it as pending, so that store invariants remain trustworthy.
  28. As a drive owner, I want a store fault or newer schema to retain the existing refusal behavior, so that a failed store is not replaced or misread.
  29. As a drive owner, I want pending evidence protected when admission reaches 6,720 observations, so that capacity pressure does not erase previously measured activity.
  30. As a drive owner, I want collection to fail visibly and avoid acquiring more observations while that capacity remains exhausted, so that backlog growth is bounded and the resulting gap is not fabricated.
  31. As a drive owner, I want recovery to continue on the normal schedule despite full admission, so that new acquisition can resume when the backlog clears.
  32. As a drive owner, I want detail older than 14 days removed by normal collection only after the durable local and UTC evidence it supports is safe, so that long-term history remains trustworthy.
  33. As a drive owner, I want pending observations and required anchors retained during pruning, so that repair can still finish after detail ages out.
  34. As a TUI user, I want the opening live graph to follow the newest activity point, so that I see new readings as they arrive.
  35. As a TUI user, I want deliberate point inspection to hold that point through refresh, so that new readings do not interrupt inspection.
  36. As a TUI user, I want an expired inspected live point explained before following resumes, so that my selection does not silently jump.
  37. As a TUI user, I want the today action to resume live following immediately, so that returning from browsing is predictable.
  38. As a TUI user browsing an older date, I want the selected date, hour and read/write measurement preserved through refresh, so that I can compare evidence without losing context.
  39. As a TUI user, I want an unavailable historical selection explained rather than silently replaced, so that I know what happened to the requested date.
  40. As a TUI user, I want cancelling date entry to retain my previous selection, so that a cancelled action changes nothing.
  41. As a TUI user, I want date entry keystrokes isolated from graph shortcuts, so that entering a date does not trigger another action.
  42. As a keyboard user, I want the existing previous/next, go-to-date, today, point inspection and measurement controls to remain discoverable from ordinary launch, so that navigation does not require hidden graph focus.
  43. As a mouse user, I want clickable activity controls to follow the same selection rules as keyboard controls, so that input method does not alter behavior.
  44. As a TUI user on a constrained terminal, I want dates, known amounts, evidence state, selection and confidence accessible in text, so that small displays do not conceal uncertainty.
  45. As a CLI user, I want the same publication and store-fault facts as the TUI, so that headless and interactive views tell the same truth.
  46. As an operator, I want native monitoring, sanctioned pause/resume and read-only TUI responsibilities unchanged, so that this repair does not require a new process or operational workflow.

Implementation Decisions

  • Design all three modules together; deliver local-day evidence and publication/recovery/retention in the first correctness stage, then activity selection in the second. Do not split a partial publication model into a shippable stage ahead of its coherent reader and local evidence.
  • Use a collection-focused module with a small caller interface for one collection run. It owns recovery of pending evidence, bounded admission before acquisition, validation, derivation, publication and safe retention. Keep the existing device-acquisition and native monitoring adapters; a normal collection remains collector-owned. No separate sampler or generic evidence-plan framework.
  • Derive local-day evidence from compatible measured intervals and trustworthy surviving coarse evidence, rather than summing overlapping whole UTC hours. Count an interval fully when its measured bounds lie inside one recorded local day even if it crosses a UTC-hour boundary. A measurement spanning local midnight remains one shared item and contributes to neither adjacent day's known amount. Do not add the same volume once as coarse evidence and again as finer evidence.
  • Record local-day identity and provenance sufficient for historical interpretation: recorded timezone, actual UTC midnight bounds, relevant controller segment and monitoring-period context, known read/write amounts, shared boundary volume and evidence/completeness state. Local-day volume completeness and the existing full-observation-day projection gate are different facts. Preserve the established UTC hour/day evidence and projection algorithm.
  • Keep one versioned SQLite observation store. Stage validated pending observations privately inside it, separate from published samples and derived evidence. Give pending work stable identity and captured calendar/identity context for deterministic recovery. Ordinary published queries—including projection sample, wear, baseline-applicability and freshness queries—cannot see staged observations. Expose only a separate pending explanation and the last published snapshot to readers.
  • On a normal run, attempt ordered pending recovery before admission and acquisition. Do not let a later arrival bypass unresolved predecessor intervals. If a newly acquired observation cannot be derived for a non-invariant reason, retain its validated raw evidence privately, leave public evidence unchanged, report unsuccessful collection and retry on the next scheduled run. If an invariant fails, reject the new work without writing it, including any newly proposed segment or monitoring-period change. A store fault retains refusal/degradation behavior rather than being converted into recoverable pending work.
  • Replace helper-owned intermediate commits with publication owned by the collection module. A complete publication and consumption of its pending work commit together; a crash exposes the old publication or the fully new one. Repeat/restart recovery cannot add measured volume twice. Repair and normal collection share derivation and coverage policy instead of divergent calculations.
  • Initial pending-admission capacity is 6,720 observations (14 days at the default three-minute target, not a time-based expiry guarantee). Retry recovery before checking the limit. If still full, skip acquisition, report failure visibly, retain existing pending evidence, and keep subsequent scheduled recovery attempts. This does not constitute a deliberate disable or permission to fill the gap later with fabricated activity.
  • Make normal collection own retention after durable publication. Remove published detail older than 14 days only after trustworthy replacement UTC/local summaries, boundary evidence, controller context, projection prerequisites and successor anchors are durable; do not delete pending work or evidence needed for recovery. Keep durable summaries indefinitely as already decided. Do not retain all fine-grained intervals forever merely to support arbitrary timezone reinterpretation.
  • Migrate and repair in the existing ordered transactional store path. Preserve historical evidence and unknown/newer-schema refusal. Mark existing unprovable local-day totals unavailable to trusted local displays and eligibility; rebuild only if surviving evidence warrants it. A timezone change does not rewrite earlier boundaries. No destructive reset of the user's observation store.
  • Deepen the existing read-only snapshot seam so TUI and CLI consume consistent published history, publication state, drive facts, status and projection inputs from one observation-store snapshot. Service state and legitimate endurance-baseline edits remain independently authoritative. Pending evidence cannot make a published observation appear fresh, alter a controller-segment interpretation or feed the usage-adjusted theoretical lifespan. Preserve existing store-fault, deliberate-disable and categorical-confidence rules.
  • Give activity selection one in-process module: distinguish live following, live point pinned by a stable interval identity, and historical date/hour selection; track read/write measurement independently. Refresh preserves an explicit pin or historical selection; if the pinned live point leaves the rolling window, explain it and resume following. The today action resets to live following. A historical point uses unambiguous time/evidence identity across repeated local clock labels. Keyboard, mouse, tabs and modal date entry use the same selection rules; widgets and plotting render rather than reconstruct private navigation state.
  • Preserve the existing TUI controls, plot behavior, reduced-motion and theme preferences, accessibility and constrained-terminal text fallback. The new interface earns depth by hiding attribution, transaction ordering and selection reconciliation, not by moving pass-through functions into new files. SQLite remains locally substitutable for tests without adding a single-adapter seam.
  • Follow ADRs 0001, 0002, 0003, 0005, 0010 and 0011. ADR 0011 explicitly clarifies that pending-work metadata is not a new health grade and freshest-sample freshness refers to the latest published sample; retain journalled/native last-collection outcomes. The established live-activity specification remains authoritative for existing accepted presentation and cadence behavior.

Testing Decisions

  • Use one primary acceptance seam: controlled SMART/sysfs acquisition fixtures and an injected clock enter the normal collection run, which uses a real temporary observation store; read through the ordinary TUI and CLI/status paths. Assert public collection outcomes, rendered known/shared volumes, evidence labels, published freshness, projection availability and selected activity. A good test fails when a user's observed fact becomes false but survives internal refactors. Do not mock the internal derivation or substitute canned reader rows for the main test path.
  • Reuse existing collection/history tracer fixtures, real-SQLite migration and repair tests, shared status/projection parity checks, and headless Textual interaction tests. Exercise navigation with user actions and assert visible date, measurement and point readouts instead of private widget indices. Keep focused store/migration and navigation transition tests only where the main seam cannot isolate a fault.
  • Probe exact conservation for both reads and writes: a known 5,120,000-byte interval crossing a UTC hour but staying in one local day must appear as known local volume; a measured 100-byte midnight-straddling Kolkata interval must exist once as shared evidence, not as 200 bytes across adjacent local dates. Include first anchor, measured zero, repeated collection, same-hour accumulation, multi-hour intervals, segment changes and deliberate disable.
  • Exercise actual local midnight boundaries for fractional offsets, 23- and 25-hour daylight-saving days, repeated local clock labels and a timezone change. Verify known versus missing versus shared evidence, recorded historical labels, truthful coverage and unchanged UTC projection evidence. Older local totals without sufficient provenance remain unavailable without deleting valid UTC history.
  • Inject a real derivation failure, not merely two successful runs: verify unsuccessful collection, durable validated pending evidence, unchanged published sample/segment/day/projection facts and visible pending state before and after restart. Then retry with an older pending item plus a newer arrival; assert ordered, idempotent publication, no duplication and no skipped evidence. Separately test invariant rollback, store-fault refusal and concurrent read snapshots.
  • Verify capacity through the normal entry point: with 6,720 pending observations, recovery still executes; acquisition is not invoked if the capacity remains full, and the failed outcome explains the gap. If recovery frees admission, collection resumes. Pending observations do not age out merely because 14 days pass.
  • Advance controlled time beyond 14 days, run the normal collection path and restart readers. Verify published detail is pruned only once replacement evidence and required anchors exist; pending work survives; old local/UTC summaries, status and eligible projection inputs remain truthful. Include interrupted migration, repeated repair and newer-schema refusal.
  • Cover CLI/TUI parity for freshness during pending work, latest counters, controller identity, available baseline, projection confidence/full-day eligibility, missing store, unreadable store and newer schema. Do not use staging-only data as published evidence merely to make a test pass.
  • Cover initial live following, user pin across refresh, pin expiry with explanation and resumed following, today reset, historical date/hour/measurement continuity, unavailable dates, invalid entry, cancellation and modal keystroke isolation. Verify keyboard and clickable equivalents at ordinary and constrained sizes, including 80×24 and below, without requiring undocumented graph focus; preserve motion/theme/help behavior.
  • Run targeted tests first, then relevant broader suites and direct behavioral probes. Use an isolated development environment to obtain missing pytest; do not operate on the installed observation store, real drive, native units or user monitoring. Report unavailable platform verification instead of claiming it passed. A build or static check alone is insufficient.

Out of Scope

  • A second observation store, generic event bus, web UI, new long-running process, separate sampler, network adapter or generic plotting framework.
  • A new endurance projection formula, fabricated baseline, physical hardware-failure forecast, numeric confidence percentage, transfer-speed-first graph or multi-drive/SATA support.
  • Guessing the location or amount of missing activity, prorating ambiguous midnight volume, counting it twice, retroactively rewriting historical timezones, or retaining unlimited fine detail solely for future timezone reinterpretation.
  • Changing the existing three-minute native cadence, ordinary retry schedule, authenticated monitoring controls, package publication or installed host state as part of this architecture work.
  • A separate test-only production interface or claims that existing tests prove correct behavior when they do not exercise the normal path.

Further Notes

The architecture review at commit 4550dd5 reproduced two defects using synthetic data in temporary observation stores: a 100-byte UTC hour contributed 100 known bytes to each of two adjacent Asia/Kolkata local days, and a cross-UTC-hour 5,120,000-byte measured interval appeared in UTC unallocated evidence but not in local-day totals. Source navigation found no production caller for pruning. These are investigation and regression targets; they are not statements about the user's deployed observation store. The original hourly-input probe is not a proof that the interval crossed local midnight; the acceptance test must use an actual measured midnight-straddling interval.

Earlier live-activity delivery issues are closed; this spec deepens and corrects the existing implementation rather than replacing the accepted product design. The latest domain glossary defines Local-day evidence, Shared local-day evidence, Pending publication and Activity selection. The previously accepted live-activity specification and ADR 0010 remain pertinent; ADR 0011 records the newly agreed recovery and admission trade-off. Design decisions and the acceptance ledger were also captured in a temporary architecture handoff, but this issue is the durable tracker source for subsequent tickets.

At review time neither the project virtual environment nor system Python had pytest available, so no pytest pass is claimed. Before implementation, arrange isolated test dependencies and establish a baseline. Do not infer a production fault or perform any host-monitoring operation merely because a synthetic probe failed.

## Problem Statement Fenris can show a plausible but incorrect picture of monitored drive activity. Local-day read and write totals currently derive from whole UTC hours: an hour straddling a local midnight can be counted on both adjacent dates, while a measured interval spanning UTC hours can disappear from the local total even when it belongs to one local date. Publication is split across commits, derivation failures can report collection success, repair and normal collection do not consistently apply the same evidence rules, and the advertised 14-day detail pruning is not reached by normal collection. Meanwhile, overlapping activity selection state in the TUI makes refreshing, browsing, and inspecting a live point fragile. A drive owner needs trustworthy measured volume, an honest account of incomplete evidence and failures, and activity navigation that stays put when requested without hiding new live data. ## Solution Keep one observation store and the existing collector-only write path, but deepen three modules together. A collection-focused observation-history publication module owns recovery, validation, consistent publication and safe retention, with local-day evidence attribution inside its implementation. A coherent read-only observation-history reader makes published evidence, pending publication, status and projection inputs agree for both TUI and CLI. An in-process activity selection module owns the user's live-following, inspected-point and historical browsing choices, leaving Textual to render. Deliver local-day evidence plus publication/recovery/retention together first; deliver navigation second. Present known read and write volumes separately as **known amounts, incomplete** when some measured volume cannot be assigned to a date. Retain ambiguous midnight-spanning read/write volume exactly once as shared local-day evidence, separate from both adjacent dates' known totals. Never divide, invent, double-count or quietly present a partial amount as a full-day total. Valid observations whose derivation fails survive privately as pending publication, while readers continue to see the last consistent publication and explain the pending condition. A failed publication is not a successful collection. Historical summaries with insufficient surviving evidence become unavailable at local-day precision without destroying trustworthy UTC history. ## User Stories 1. As a drive owner, I want each local-day write amount to contain only writes supported for that date, so that I can trust its contribution to observed activity. 2. As a drive owner, I want each local-day read amount kept distinct from writes, so that I do not mistake reads for endurance consumption. 3. As a drive owner, I want a measured interval that crosses a UTC-hour boundary but lies inside one local day to count for that day, so that short bursts are not lost. 4. As a drive owner, I want a measured interval crossing local midnight kept once as shared evidence, so that neither adjacent day claims volume it cannot prove. 5. As a drive owner, I want the known amount shown with an explicit incomplete label when a date has ambiguous or missing evidence, so that a partial measurement does not look like a complete daily total. 6. As a drive owner, I want shared read and write volume visibly explained apart from known daily amounts, so that I understand why a day is incomplete. 7. As a drive owner, I want measured zero distinguished from absent evidence, so that an idle drive is not mistaken for missing observations. 8. As a drive owner, I want today's amounts labelled as so far, so that a live partial day is not presented as finished. 9. As a drive owner in a half-hour-offset timezone, I want local midnight measured at the actual boundary, so that adjacent dates do not share the same whole UTC hour as known activity. 10. As a drive owner crossing a daylight-saving transition, I want the actual 23- or 25-hour local day respected, so that coverage and volume are not based on a fixed day length. 11. As a drive owner browsing a repeated local clock hour, I want points distinguished by their actual instants and timezone labels, so that two observations do not collapse into one. 12. As a drive owner after a timezone change, I want earlier local-day summaries to retain their recorded timezone and midnight boundaries, so that history is not silently reinterpreted. 13. As a drive owner with old coarse observation history, I want unprovable local-day totals identified as unavailable, so that migration does not manufacture precise dates. 14. As a drive owner, I want old measured evidence preserved even when a legacy local total cannot be trusted, so that trustworthy UTC history remains useful. 15. As a drive owner, I want an applicable usage-adjusted theoretical lifespan computed only from published, compatible observation history, so that pending counters cannot change the outlook prematurely. 16. As a drive owner, I want projection confidence and the complete-observation-day gate to retain their existing meaning, so that the new activity evidence does not invent certainty. 17. As a drive owner, I want controller changes and counter discontinuities respected during derivation, so that writes from distinct controller segments are not combined. 18. As a drive owner who deliberately disables monitoring, I want that time excluded as before, so that missing activity is not mistaken for a monitoring period. 19. As a drive owner, I want readings within a monitoring period that cannot be classified to remain unknown, so that gaps are not silently filled. 20. As a drive owner, I want a successful collection to update the relevant read views together, so that a recent sample cannot disagree with its activity summaries. 21. As a drive owner, I want a derivation failure reported as a failed collection with a clear pending explanation, so that I do not trust partially published measurements. 22. As a drive owner, I want valid acquired observations retained for retry after a derivation failure, so that real measured evidence is not discarded. 23. As a drive owner, I want the most recent published observation, rather than a pending one, to determine displayed freshness, so that unpublished work cannot masquerade as fresh history. 24. As a drive owner, I want pending observations excluded from drive facts, identity interpretation and projection inputs, so that every visible measurement describes one coherent publication. 25. As a drive owner, I want each scheduled collection to retry pending publication before considering new acquisition, so that older unfinished evidence is not skipped. 26. As a drive owner, I want recovery after a restart to avoid counting any interval twice, so that repeated runs conserve measured volume. 27. As a drive owner, I want an invalid acquired observation rejected without retaining it as pending, so that store invariants remain trustworthy. 28. As a drive owner, I want a store fault or newer schema to retain the existing refusal behavior, so that a failed store is not replaced or misread. 29. As a drive owner, I want pending evidence protected when admission reaches 6,720 observations, so that capacity pressure does not erase previously measured activity. 30. As a drive owner, I want collection to fail visibly and avoid acquiring more observations while that capacity remains exhausted, so that backlog growth is bounded and the resulting gap is not fabricated. 31. As a drive owner, I want recovery to continue on the normal schedule despite full admission, so that new acquisition can resume when the backlog clears. 32. As a drive owner, I want detail older than 14 days removed by normal collection only after the durable local and UTC evidence it supports is safe, so that long-term history remains trustworthy. 33. As a drive owner, I want pending observations and required anchors retained during pruning, so that repair can still finish after detail ages out. 34. As a TUI user, I want the opening live graph to follow the newest activity point, so that I see new readings as they arrive. 35. As a TUI user, I want deliberate point inspection to hold that point through refresh, so that new readings do not interrupt inspection. 36. As a TUI user, I want an expired inspected live point explained before following resumes, so that my selection does not silently jump. 37. As a TUI user, I want the today action to resume live following immediately, so that returning from browsing is predictable. 38. As a TUI user browsing an older date, I want the selected date, hour and read/write measurement preserved through refresh, so that I can compare evidence without losing context. 39. As a TUI user, I want an unavailable historical selection explained rather than silently replaced, so that I know what happened to the requested date. 40. As a TUI user, I want cancelling date entry to retain my previous selection, so that a cancelled action changes nothing. 41. As a TUI user, I want date entry keystrokes isolated from graph shortcuts, so that entering a date does not trigger another action. 42. As a keyboard user, I want the existing previous/next, go-to-date, today, point inspection and measurement controls to remain discoverable from ordinary launch, so that navigation does not require hidden graph focus. 43. As a mouse user, I want clickable activity controls to follow the same selection rules as keyboard controls, so that input method does not alter behavior. 44. As a TUI user on a constrained terminal, I want dates, known amounts, evidence state, selection and confidence accessible in text, so that small displays do not conceal uncertainty. 45. As a CLI user, I want the same publication and store-fault facts as the TUI, so that headless and interactive views tell the same truth. 46. As an operator, I want native monitoring, sanctioned pause/resume and read-only TUI responsibilities unchanged, so that this repair does not require a new process or operational workflow. ## Implementation Decisions - Design all three modules together; deliver local-day evidence and publication/recovery/retention in the first correctness stage, then activity selection in the second. Do not split a partial publication model into a shippable stage ahead of its coherent reader and local evidence. - Use a collection-focused module with a small caller interface for one collection run. It owns recovery of pending evidence, bounded admission before acquisition, validation, derivation, publication and safe retention. Keep the existing device-acquisition and native monitoring adapters; a normal collection remains collector-owned. No separate sampler or generic evidence-plan framework. - Derive local-day evidence from compatible measured intervals and trustworthy surviving coarse evidence, rather than summing overlapping whole UTC hours. Count an interval fully when its measured bounds lie inside one recorded local day even if it crosses a UTC-hour boundary. A measurement spanning local midnight remains one shared item and contributes to neither adjacent day's known amount. Do not add the same volume once as coarse evidence and again as finer evidence. - Record local-day identity and provenance sufficient for historical interpretation: recorded timezone, actual UTC midnight bounds, relevant controller segment and monitoring-period context, known read/write amounts, shared boundary volume and evidence/completeness state. Local-day volume completeness and the existing full-observation-day projection gate are different facts. Preserve the established UTC hour/day evidence and projection algorithm. - Keep one versioned SQLite observation store. Stage validated pending observations privately inside it, separate from published samples and derived evidence. Give pending work stable identity and captured calendar/identity context for deterministic recovery. Ordinary published queries—including projection sample, wear, baseline-applicability and freshness queries—cannot see staged observations. Expose only a separate pending explanation and the last published snapshot to readers. - On a normal run, attempt ordered pending recovery before admission and acquisition. Do not let a later arrival bypass unresolved predecessor intervals. If a newly acquired observation cannot be derived for a non-invariant reason, retain its validated raw evidence privately, leave public evidence unchanged, report unsuccessful collection and retry on the next scheduled run. If an invariant fails, reject the new work without writing it, including any newly proposed segment or monitoring-period change. A store fault retains refusal/degradation behavior rather than being converted into recoverable pending work. - Replace helper-owned intermediate commits with publication owned by the collection module. A complete publication and consumption of its pending work commit together; a crash exposes the old publication or the fully new one. Repeat/restart recovery cannot add measured volume twice. Repair and normal collection share derivation and coverage policy instead of divergent calculations. - Initial pending-admission capacity is **6,720 observations** (14 days at the default three-minute target, not a time-based expiry guarantee). Retry recovery before checking the limit. If still full, skip acquisition, report failure visibly, retain existing pending evidence, and keep subsequent scheduled recovery attempts. This does not constitute a deliberate disable or permission to fill the gap later with fabricated activity. - Make normal collection own retention after durable publication. Remove published detail older than 14 days only after trustworthy replacement UTC/local summaries, boundary evidence, controller context, projection prerequisites and successor anchors are durable; do not delete pending work or evidence needed for recovery. Keep durable summaries indefinitely as already decided. Do not retain all fine-grained intervals forever merely to support arbitrary timezone reinterpretation. - Migrate and repair in the existing ordered transactional store path. Preserve historical evidence and unknown/newer-schema refusal. Mark existing unprovable local-day totals unavailable to trusted local displays and eligibility; rebuild only if surviving evidence warrants it. A timezone change does not rewrite earlier boundaries. No destructive reset of the user's observation store. - Deepen the existing read-only snapshot seam so TUI and CLI consume consistent published history, publication state, drive facts, status and projection inputs from one observation-store snapshot. Service state and legitimate endurance-baseline edits remain independently authoritative. Pending evidence cannot make a published observation appear fresh, alter a controller-segment interpretation or feed the usage-adjusted theoretical lifespan. Preserve existing store-fault, deliberate-disable and categorical-confidence rules. - Give activity selection one in-process module: distinguish live following, live point pinned by a stable interval identity, and historical date/hour selection; track read/write measurement independently. Refresh preserves an explicit pin or historical selection; if the pinned live point leaves the rolling window, explain it and resume following. The today action resets to live following. A historical point uses unambiguous time/evidence identity across repeated local clock labels. Keyboard, mouse, tabs and modal date entry use the same selection rules; widgets and plotting render rather than reconstruct private navigation state. - Preserve the existing TUI controls, plot behavior, reduced-motion and theme preferences, accessibility and constrained-terminal text fallback. The new interface earns depth by hiding attribution, transaction ordering and selection reconciliation, not by moving pass-through functions into new files. SQLite remains locally substitutable for tests without adding a single-adapter seam. - Follow ADRs 0001, 0002, 0003, 0005, 0010 and 0011. ADR 0011 explicitly clarifies that pending-work metadata is not a new health grade and freshest-sample freshness refers to the latest **published** sample; retain journalled/native last-collection outcomes. The established live-activity specification remains authoritative for existing accepted presentation and cadence behavior. ## Testing Decisions - Use one primary acceptance seam: controlled SMART/sysfs acquisition fixtures and an injected clock enter the normal collection run, which uses a real temporary observation store; read through the ordinary TUI and CLI/status paths. Assert public collection outcomes, rendered known/shared volumes, evidence labels, published freshness, projection availability and selected activity. A good test fails when a user's observed fact becomes false but survives internal refactors. Do not mock the internal derivation or substitute canned reader rows for the main test path. - Reuse existing collection/history tracer fixtures, real-SQLite migration and repair tests, shared status/projection parity checks, and headless Textual interaction tests. Exercise navigation with user actions and assert visible date, measurement and point readouts instead of private widget indices. Keep focused store/migration and navigation transition tests only where the main seam cannot isolate a fault. - Probe exact conservation for both reads and writes: a known 5,120,000-byte interval crossing a UTC hour but staying in one local day must appear as known local volume; a measured 100-byte midnight-straddling Kolkata interval must exist once as shared evidence, not as 200 bytes across adjacent local dates. Include first anchor, measured zero, repeated collection, same-hour accumulation, multi-hour intervals, segment changes and deliberate disable. - Exercise actual local midnight boundaries for fractional offsets, 23- and 25-hour daylight-saving days, repeated local clock labels and a timezone change. Verify known versus missing versus shared evidence, recorded historical labels, truthful coverage and unchanged UTC projection evidence. Older local totals without sufficient provenance remain unavailable without deleting valid UTC history. - Inject a **real derivation failure**, not merely two successful runs: verify unsuccessful collection, durable validated pending evidence, unchanged published sample/segment/day/projection facts and visible pending state before and after restart. Then retry with an older pending item plus a newer arrival; assert ordered, idempotent publication, no duplication and no skipped evidence. Separately test invariant rollback, store-fault refusal and concurrent read snapshots. - Verify capacity through the normal entry point: with 6,720 pending observations, recovery still executes; acquisition is not invoked if the capacity remains full, and the failed outcome explains the gap. If recovery frees admission, collection resumes. Pending observations do not age out merely because 14 days pass. - Advance controlled time beyond 14 days, run the normal collection path and restart readers. Verify published detail is pruned only once replacement evidence and required anchors exist; pending work survives; old local/UTC summaries, status and eligible projection inputs remain truthful. Include interrupted migration, repeated repair and newer-schema refusal. - Cover CLI/TUI parity for freshness during pending work, latest counters, controller identity, available baseline, projection confidence/full-day eligibility, missing store, unreadable store and newer schema. Do not use staging-only data as published evidence merely to make a test pass. - Cover initial live following, user pin across refresh, pin expiry with explanation and resumed following, today reset, historical date/hour/measurement continuity, unavailable dates, invalid entry, cancellation and modal keystroke isolation. Verify keyboard and clickable equivalents at ordinary and constrained sizes, including 80×24 and below, without requiring undocumented graph focus; preserve motion/theme/help behavior. - Run targeted tests first, then relevant broader suites and direct behavioral probes. Use an isolated development environment to obtain missing pytest; do not operate on the installed observation store, real drive, native units or user monitoring. Report unavailable platform verification instead of claiming it passed. A build or static check alone is insufficient. ## Out of Scope - A second observation store, generic event bus, web UI, new long-running process, separate sampler, network adapter or generic plotting framework. - A new endurance projection formula, fabricated baseline, physical hardware-failure forecast, numeric confidence percentage, transfer-speed-first graph or multi-drive/SATA support. - Guessing the location or amount of missing activity, prorating ambiguous midnight volume, counting it twice, retroactively rewriting historical timezones, or retaining unlimited fine detail solely for future timezone reinterpretation. - Changing the existing three-minute native cadence, ordinary retry schedule, authenticated monitoring controls, package publication or installed host state as part of this architecture work. - A separate test-only production interface or claims that existing tests prove correct behavior when they do not exercise the normal path. ## Further Notes The architecture review at commit 4550dd5 reproduced two defects using synthetic data in temporary observation stores: a 100-byte UTC hour contributed 100 known bytes to each of two adjacent Asia/Kolkata local days, and a cross-UTC-hour 5,120,000-byte measured interval appeared in UTC unallocated evidence but not in local-day totals. Source navigation found no production caller for pruning. These are investigation and regression targets; they are **not** statements about the user's deployed observation store. The original hourly-input probe is not a proof that the interval crossed local midnight; the acceptance test must use an actual measured midnight-straddling interval. Earlier live-activity delivery issues are closed; this spec deepens and corrects the existing implementation rather than replacing the accepted product design. The latest domain glossary defines Local-day evidence, Shared local-day evidence, Pending publication and Activity selection. The previously accepted live-activity specification and ADR 0010 remain pertinent; ADR 0011 records the newly agreed recovery and admission trade-off. Design decisions and the acceptance ledger were also captured in a temporary architecture handoff, but this issue is the durable tracker source for subsequent tickets. At review time neither the project virtual environment nor system Python had pytest available, so no pytest pass is claimed. Before implementation, arrange isolated test dependencies and establish a baseline. Do not infer a production fault or perform any host-monitoring operation merely because a synthetic probe failed.
xavierk added the ready-for-agent label 2026-09-27 19:54:14 +00:00
xavierk commented 2026-10-02 08:22:02 +00:00
Author
Owner
Copy Link
Copy Source

Delivered in v0.6.0 through tickets #96 to #103: single publication transaction, pending publication retention and recovery, local-day evidence conservation and historical trust, safe pruning through normal collection, bounded pending admission, live follow/pin, and historical activity selection. Closing the parent.

Delivered in v0.6.0 through tickets #96 to #103: single publication transaction, pending publication retention and recovery, local-day evidence conservation and historical trust, safe pruning through normal collection, bounded pending admission, live follow/pin, and historical activity selection. Closing the parent.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
ready-for-agent
Spec triaged and ready for an implementing agent
 wayfinder:grilling
HITL conversational decision ticket
 wayfinder:map
Canonical Wayfinder decision map
 wayfinder:prototype
HITL prototype decision ticket
 wayfinder:research
AFK research decision ticket
 wayfinder:task
Prerequisite task ticket
No labels ready-for-agent
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
xavierk (Xavier Karma)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: xavierk/Fenris#95