Choose Odin’s workload suite and run profiles #8

New Issue

2026-09-25T17:48:50Z

xavierk commented

2026-09-25 17:48:50 +00:00

Part of Find the way to Odin’s build-ready specification.

Question

Which exact workloads and checks belong in the first release and each quick/standard/extended profile? Choose workload versions, measurement definitions, repetition/budget policy, optional dependencies, installed versus reference browser/toolchain behavior, unavailable/error/cancelled states, and the boundary between online diagnostics and separately requested exhaustive tests. Account for all nine requested areas or explicitly resolve a proposed deferral with the user. The standard run targets 10–20 minutes. Decide separately whether GPU presentation workloads may open a visible window; the approved headed-browser mode does not settle that choice.

Use the linked research to make selections with the human, including tradeoffs and acceptance criteria; do not implement them.

Part of [Find the way to Odin’s build-ready specification](https://git.bongbetic.com/xavierk/odin/issues/1).  ## Question Which exact workloads and checks belong in the first release and each quick/standard/extended profile? Choose workload versions, measurement definitions, repetition/budget policy, optional dependencies, installed versus reference browser/toolchain behavior, unavailable/error/cancelled states, and the boundary between online diagnostics and separately requested exhaustive tests. Account for all nine requested areas or explicitly resolve a proposed deferral with the user. The standard run targets 10–20 minutes. Decide separately whether GPU presentation workloads may open a visible window; the approved headed-browser mode does not settle that choice. Use the linked research to make selections with the human, including tradeoffs and acceptance criteria; do not implement them.

xavierk added the wayfinder:grilling label 2026-09-25 17:48:50 +00:00

xavierk added a new dependency 2026-09-25 17:49:47 +00:00

#2 Establish safe CPU, memory, and kernel measurements

xavierk added a new dependency 2026-09-25 17:49:48 +00:00

#3 Establish GPU driver evidence and performance workloads

xavierk added a new dependency 2026-09-25 17:49:48 +00:00

#4 Establish storage measurements and trustworthy health advice

xavierk added a new dependency 2026-09-25 17:49:49 +00:00

#5 Establish browser, shell, and language workload validity

xavierk added a new dependency 2026-09-25 17:49:51 +00:00

#9 Choose Odin’s runtime and Linux compatibility contract

xavierk added a new dependency 2026-09-25 17:49:54 +00:00

#10 Define Odin’s median score and comparability contract

xavierk added a new dependency 2026-09-25 17:49:57 +00:00

#11 Define health findings and the optimization advice rubric

xavierk added a new dependency 2026-09-25 17:50:05 +00:00

#14 Set distro, VM, and physical-hardware validation gates

xavierk added a new dependency 2026-09-25 17:50:08 +00:00

#15 Approve Odin’s build-ready specification

xavierk self-assigned this 2026-09-25 18:22:00 +00:00

xavierk commented

2026-09-25 18:47:13 +00:00

Resolution

Odin v1 has three run profiles. Quick is a read-only, roughly 2–3 minute triage run. Standard attempts every requested area for which the system and prepared workload pack have capability, targeting 10–20 minutes. Extended is a menu of separately selected modules with individual time, memory, and write estimates. A missing capability is a named outcome, never a numerical zero or a silent replacement workload. One selected filesystem path and one selected GPU are measured per standard run; safe health discovery can inspect other devices. The display GPU is the default; an ambiguous or headless multi-GPU selection requires a choice.

Standard workload contract

Area	Selected v1 workload and measurement
CPU	Pinned sysbench 1.0.20 prime-search work with `cpu-max-prime=20000`, one worker and workers matched to the effective CPU entitlement in separate tests. The parallel worker count is `max(1, ceil(min(allowed logical CPUs, effective cgroup CPU quota)))`, treating an absent quota as the allowed count. Three 10-second timed runs per configuration; record events/s, actual worker count, affinity, quota, and throttling. This is a narrow integer workload, not a general CPU rating.
Memory and loaded responsiveness	STREAM 5.10 Copy/Triad with an array size recorded per run and large enough to exceed the relevant last-level cache; preserve STREAM's own best-iteration statistic and take any cross-run median only over three independent runs. Pinned schbench revision `6300b8f3a8922c61ea6bb2cdfa1901a42c0cc6fc` measures request and wakeup latency alone and under bounded stress-ng 0.22.01 CPU load. Its v1 protocol uses one message/worker thread, 64 KiB work footprint, one matrix operation, 100 µs sleep, 100 requested requests/s, explicit zero internal warmup, and 15-second timed runs; three idle/loaded pairs retain sample counts, p50/p95/p99, achieved throughput, requested/achieved load, and PSI deltas. A separate untimed preparation phase handles warmup because this revision does not reliably reset warmup in rate-limited mode. Stress-ng bogo-ops are not a performance metric.
Memory errors	One finite memtester 4.7.1 pass over at most 256 MiB, only when safe headroom remains; record actual allocation, locked bytes, completed patterns, errors and timeout. Read available EDAC/RAS counters and pressure telemetry before and after. Passing says only that no error was detected in tested bytes. Active memory-pressure generation belongs only to an explicitly selected, contained extended module.
GPU	Enumerate device, kernel driver, selected userspace renderer/API and permissions first. For one selected hardware GPU, use a qualified Vulkan-only clpeak 2.1.4 build for fixed FP32 and global-memory-bandwidth tests, and glmark2 2023.01 for fixed 800×600 offscreen GL/GLES shading and texture scenes where that path is available. Keep compute and rendering metrics separate; repeat short timed tests three times. A separate known-output operation checks execution correctness. Software/virtual renderers have distinct identities and do not silently count as hardware-GPU results. Standard opens no GPU presentation window.
Storage	On one user-selected filesystem path, fio 3.43 uses a private Odin-owned 256 MiB fixture. Run 1 MiB sequential read (256 MiB) and write (128 MiB), then 4 KiB random read and write (4 MiB each), using `psync` at queue depth 1 and three repeats each. Use aligned direct I/O when supported; a buffered fallback is a separately named result. Record bytes, elapsed time, bandwidth/IOPS, completion-latency distribution, engine, cache/flush mode, filesystem and device mapping. This is a bounded burst-path test, not sustained SSD speed. Fixture preparation plus repeated writes total 652 MiB before metadata; enforce a 1 GiB host-write ceiling and leave at least 2 GiB plus 10% of filesystem capacity free after the run. Show the estimate and require a clear prompt; if declined or unsafe, skip writable performance tests. Never target an existing user file or raw device.
Browser	Locally host a complete Speedometer 3.1 pack based on release commit `1386415be8fef2f6b6bbdbe1828872471c5d802a` on loopback, pinning its complete asset digest at packaging. The standard run uses the official full default suite, ten internal iterations and 800×600 workload viewport, on one installed browser with matching automation. Use headed mode when a display is available and headless otherwise; each is a distinct result identity. Preserve the native score, mean, confidence interval, raw measurements and settings. One full suite is run in standard; a 5-minute end-to-end deadline marks an unfinished suite incomplete rather than changing its protocol. Reference-browser mode is separately prepared and labelled.
Bash shell	Measure clean direct Bash process startup and clean interactive PTY prompt readiness as distinct distributions (30 samples each), plus three timed runs of a pinned builtin arithmetic script: for integer `i=0..99999`, sum `(17i+23) mod 1000000` using Bash arithmetic and verify `43824450000`. Installed Bash/version is recorded. The user's startup files are not run by default.
Python, Rust, C++, Java	Use an Odin-owned `text-parse-v1` source-and-input pack, prepared before timing with each installed toolchain. The same 1,048,576 ASCII records are parsed and aggregated in memory; each language verifies the record count and weighted sum `Σ(key+1)×value = 266529503798272`. For `i=0..1048575`, input line `i` is zero-padded `key,value\n`, with `key=(73i+19) mod 1024` as four digits and `value=(17i+23) mod 1000000` as six digits. The 12 MiB corpus SHA-256 is `1f7707f5f5e19ec9292a0aecdb67fc241176e58b86dccc04b8cc13acf421633b`. Report completed passes/s from three fixed five-second timed windows after a declared warmup, plus 30 clean process-to-marker startup trials. For Rust/C++, startup means launching the prepared binary, not compiler startup. Results describe these implementations and toolchains; Odin does not rank programming languages intrinsically.
Overall health	Before and after load, read available temperature/alarm sensors, CPU/memory/I/O PSI, EDAC/RAS, kernel/device I/O errors, and safe protocol-specific SMART data for discoverable drives. Use supported installed smartctl parsers, optionally qualified nvme-cli data, and preserve tool versions, collection scope and permission failures. Do not start self-tests or guess transports with potential writes. Health findings remain separate from speed results.

For memory allocation, retain at least max(1 GiB, 25% of effective available memory) outside Odin's workers. STREAM's three arrays are each at least max(128 MiB, 4× reported last-level cache) and together at most 1.5 GiB; if this cannot fit safely or cache size cannot be established, skip bandwidth with a reason. The online memory check is capped at 256 MiB and one pass. GPU performance workers must be confined to a qualified VRAM limit no greater than 256 MiB or 25% of known free VRAM, whichever is smaller; otherwise skip the GPU performance worker. Active CPU load targets about half the effective CPU entitlement and is recorded, never interpreted from bogo-ops.

The standard time allocation starts at roughly 1 minute for preflight/health, 4.5 for CPU/memory/responsiveness, 3 for GPU, 2.5 for storage, 5 for browser, 0.5 for Bash, 3 for the four languages, and 0.5 for transitions/cleanup. These are initial stage targets, not a global kill switch. Every worker has a fixed end-to-end deadline that includes warmup, initialization and calibration; slow or unsupported systems may yield incomplete/partial results. Neither upstream workloads nor official statistics are silently shortened to fit the target. Validate these budgets on low-end physical hardware before release.

Other profiles and boundaries

Quick collects safe read-only health and capability evidence, short CPU/Bash/language probes, memory/PSI context, GPU discovery and a bounded correctness probe, and browser automation availability. Read-only here means no storage writes, device-changing commands, or self-tests; short computation can still occur in RAM/GPU memory. It does not create a storage fixture, publish a Speedometer score, or claim exhaustive memory/GPU coverage. Its shortened probes have their own identities.

Extended modules can be selected independently: contained active memory pressure and longer scheduler tests; visible GPU presentation and additional devices; queued random read, durable/sync write, integrity verification and larger sustained-media file work under a separately shown write budget; three complete Speedometer runs, JetStream 3.0 and MotionMark 1.3.2; configured-shell prompt and fixed pipeline tests; clean Rust/C++/Java compilation and longer language-pack runtime tests. A GPU presentation module may open a window only after explicit selection and never takes over KMS as an automatic fallback. Offline Memtest86+ and long drive self-tests are separate follow-up flows outside every timed profile.

Performance helper builds and workload assets are pinned and qualified for each supported architecture/libc lane. Browser and language software use installed mode by default; prepared reference mode is available and is a distinct comparison cohort. Missing software yields a reasoned skip and an offer of separate preparation with consent; no ordinary run installs packages or downloads assets. Pin source/artifact hashes, parser and protocol revisions, license notices, and resource closure before distribution. A helper with the wrong version never silently substitutes into the same workload identity.

The outcome vocabulary distinguishes completed, not scheduled, unavailable (dependency, hardware/API, permissions, display or safe resources), invalid output, execution failure, timeout and cancellation. Retain partial samples and cleanup status without calling them valid completed measurements. Preserve kernel, governor, scheduler, mount, thermal/power and toolchain conditions as measured; the run does not tune them automatically. Workload identity, score eligibility and normalization will be finalized in Define Odin’s median score and comparability contract; warning thresholds and stop rules in Define health findings and the optimization advice rubric; delivery floors and packaging in Choose Odin’s runtime and Linux compatibility contract; and the validation matrix in Set distro, VM, and physical-hardware validation gates.

This selection follows the cited findings in Establish safe CPU, memory, and kernel measurements, Establish GPU driver evidence and performance workloads, Establish storage measurements and trustworthy health advice, and Establish browser, shell, and language workload validity. All pinned candidates remain subject to build, license, correctness, cancellation and physical-hardware qualification; a failed qualification reopens the relevant workload choice rather than silently changing it.

## Resolution Odin v1 has three run profiles. **Quick** is a read-only, roughly 2–3 minute triage run. **Standard** attempts every requested area for which the system and prepared workload pack have capability, targeting 10–20 minutes. **Extended** is a menu of separately selected modules with individual time, memory, and write estimates. A missing capability is a named outcome, never a numerical zero or a silent replacement workload. One selected filesystem path and one selected GPU are measured per standard run; safe health discovery can inspect other devices. The display GPU is the default; an ambiguous or headless multi-GPU selection requires a choice. ### Standard workload contract | Area | Selected v1 workload and measurement | | --- | --- | | CPU | Pinned sysbench 1.0.20 prime-search work with `cpu-max-prime=20000`, one worker and workers matched to the effective CPU entitlement in separate tests. The parallel worker count is `max(1, ceil(min(allowed logical CPUs, effective cgroup CPU quota)))`, treating an absent quota as the allowed count. Three 10-second timed runs per configuration; record events/s, actual worker count, affinity, quota, and throttling. This is a narrow integer workload, not a general CPU rating. | | Memory and loaded responsiveness | STREAM 5.10 Copy/Triad with an array size recorded per run and large enough to exceed the relevant last-level cache; preserve STREAM's own best-iteration statistic and take any cross-run median only over three independent runs. Pinned schbench revision `6300b8f3a8922c61ea6bb2cdfa1901a42c0cc6fc` measures request and wakeup latency alone and under bounded stress-ng 0.22.01 CPU load. Its v1 protocol uses one message/worker thread, 64 KiB work footprint, one matrix operation, 100 µs sleep, 100 requested requests/s, explicit zero internal warmup, and 15-second timed runs; three idle/loaded pairs retain sample counts, p50/p95/p99, achieved throughput, requested/achieved load, and PSI deltas. A separate untimed preparation phase handles warmup because this revision does not reliably reset warmup in rate-limited mode. Stress-ng bogo-ops are not a performance metric. | | Memory errors | One finite memtester 4.7.1 pass over at most 256 MiB, only when safe headroom remains; record actual allocation, locked bytes, completed patterns, errors and timeout. Read available EDAC/RAS counters and pressure telemetry before and after. Passing says only that no error was detected in tested bytes. Active memory-pressure generation belongs only to an explicitly selected, contained extended module. | | GPU | Enumerate device, kernel driver, selected userspace renderer/API and permissions first. For one selected hardware GPU, use a qualified Vulkan-only clpeak 2.1.4 build for fixed FP32 and global-memory-bandwidth tests, and glmark2 2023.01 for fixed 800×600 offscreen GL/GLES shading and texture scenes where that path is available. Keep compute and rendering metrics separate; repeat short timed tests three times. A separate known-output operation checks execution correctness. Software/virtual renderers have distinct identities and do not silently count as hardware-GPU results. Standard opens no GPU presentation window. | | Storage | On one user-selected filesystem path, fio 3.43 uses a private Odin-owned 256 MiB fixture. Run 1 MiB sequential read (256 MiB) and write (128 MiB), then 4 KiB random read and write (4 MiB each), using `psync` at queue depth 1 and three repeats each. Use aligned direct I/O when supported; a buffered fallback is a separately named result. Record bytes, elapsed time, bandwidth/IOPS, completion-latency distribution, engine, cache/flush mode, filesystem and device mapping. This is a bounded burst-path test, not sustained SSD speed. Fixture preparation plus repeated writes total 652 MiB before metadata; enforce a 1 GiB host-write ceiling and leave at least 2 GiB plus 10% of filesystem capacity free after the run. Show the estimate and require a clear prompt; if declined or unsafe, skip writable performance tests. Never target an existing user file or raw device. | | Browser | Locally host a complete Speedometer 3.1 pack based on release commit `1386415be8fef2f6b6bbdbe1828872471c5d802a` on loopback, pinning its complete asset digest at packaging. The standard run uses the official full default suite, ten internal iterations and 800×600 workload viewport, on one installed browser with matching automation. Use headed mode when a display is available and headless otherwise; each is a distinct result identity. Preserve the native score, mean, confidence interval, raw measurements and settings. One full suite is run in standard; a 5-minute end-to-end deadline marks an unfinished suite incomplete rather than changing its protocol. Reference-browser mode is separately prepared and labelled. | | Bash shell | Measure clean direct Bash process startup and clean interactive PTY prompt readiness as distinct distributions (30 samples each), plus three timed runs of a pinned builtin arithmetic script: for integer `i=0..99999`, sum `(17i+23) mod 1000000` using Bash arithmetic and verify `43824450000`. Installed Bash/version is recorded. The user's startup files are not run by default. | | Python, Rust, C++, Java | Use an Odin-owned `text-parse-v1` source-and-input pack, prepared before timing with each installed toolchain. The same 1,048,576 ASCII records are parsed and aggregated in memory; each language verifies the record count and weighted sum `Σ(key+1)×value = 266529503798272`. For `i=0..1048575`, input line `i` is zero-padded `key,value\n`, with `key=(73i+19) mod 1024` as four digits and `value=(17i+23) mod 1000000` as six digits. The 12 MiB corpus SHA-256 is `1f7707f5f5e19ec9292a0aecdb67fc241176e58b86dccc04b8cc13acf421633b`. Report completed passes/s from three fixed five-second timed windows after a declared warmup, plus 30 clean process-to-marker startup trials. For Rust/C++, startup means launching the prepared binary, not compiler startup. Results describe these implementations and toolchains; Odin does not rank programming languages intrinsically. | | Overall health | Before and after load, read available temperature/alarm sensors, CPU/memory/I/O PSI, EDAC/RAS, kernel/device I/O errors, and safe protocol-specific SMART data for discoverable drives. Use supported installed smartctl parsers, optionally qualified nvme-cli data, and preserve tool versions, collection scope and permission failures. Do not start self-tests or guess transports with potential writes. Health findings remain separate from speed results. | For memory allocation, retain at least `max(1 GiB, 25% of effective available memory)` outside Odin's workers. STREAM's three arrays are each at least `max(128 MiB, 4× reported last-level cache)` and together at most 1.5 GiB; if this cannot fit safely or cache size cannot be established, skip bandwidth with a reason. The online memory check is capped at 256 MiB and one pass. GPU performance workers must be confined to a qualified VRAM limit no greater than 256 MiB or 25% of known free VRAM, whichever is smaller; otherwise skip the GPU performance worker. Active CPU load targets about half the effective CPU entitlement and is recorded, never interpreted from bogo-ops. The standard time allocation starts at roughly 1 minute for preflight/health, 4.5 for CPU/memory/responsiveness, 3 for GPU, 2.5 for storage, 5 for browser, 0.5 for Bash, 3 for the four languages, and 0.5 for transitions/cleanup. These are initial stage targets, not a global kill switch. Every worker has a fixed end-to-end deadline that includes warmup, initialization and calibration; slow or unsupported systems may yield incomplete/partial results. Neither upstream workloads nor official statistics are silently shortened to fit the target. Validate these budgets on low-end physical hardware before release. ### Other profiles and boundaries Quick collects safe read-only health and capability evidence, short CPU/Bash/language probes, memory/PSI context, GPU discovery and a bounded correctness probe, and browser automation availability. Read-only here means no storage writes, device-changing commands, or self-tests; short computation can still occur in RAM/GPU memory. It does not create a storage fixture, publish a Speedometer score, or claim exhaustive memory/GPU coverage. Its shortened probes have their own identities. Extended modules can be selected independently: contained active memory pressure and longer scheduler tests; visible GPU presentation and additional devices; queued random read, durable/sync write, integrity verification and larger sustained-media file work under a separately shown write budget; three complete Speedometer runs, JetStream 3.0 and MotionMark 1.3.2; configured-shell prompt and fixed pipeline tests; clean Rust/C++/Java compilation and longer language-pack runtime tests. A GPU presentation module may open a window only after explicit selection and never takes over KMS as an automatic fallback. Offline Memtest86+ and long drive self-tests are separate follow-up flows outside every timed profile. Performance helper builds and workload assets are pinned and qualified for each supported architecture/libc lane. Browser and language software use installed mode by default; prepared reference mode is available and is a distinct comparison cohort. Missing software yields a reasoned skip and an offer of separate preparation with consent; no ordinary run installs packages or downloads assets. Pin source/artifact hashes, parser and protocol revisions, license notices, and resource closure before distribution. A helper with the wrong version never silently substitutes into the same workload identity. The outcome vocabulary distinguishes completed, not scheduled, unavailable (dependency, hardware/API, permissions, display or safe resources), invalid output, execution failure, timeout and cancellation. Retain partial samples and cleanup status without calling them valid completed measurements. Preserve kernel, governor, scheduler, mount, thermal/power and toolchain conditions as measured; the run does not tune them automatically. Workload identity, score eligibility and normalization will be finalized in [Define Odin’s median score and comparability contract](https://git.bongbetic.com/xavierk/odin/issues/10); warning thresholds and stop rules in [Define health findings and the optimization advice rubric](https://git.bongbetic.com/xavierk/odin/issues/11); delivery floors and packaging in [Choose Odin’s runtime and Linux compatibility contract](https://git.bongbetic.com/xavierk/odin/issues/9); and the validation matrix in [Set distro, VM, and physical-hardware validation gates](https://git.bongbetic.com/xavierk/odin/issues/14). This selection follows the cited findings in [Establish safe CPU, memory, and kernel measurements](https://git.bongbetic.com/xavierk/odin/issues/2), [Establish GPU driver evidence and performance workloads](https://git.bongbetic.com/xavierk/odin/issues/3), [Establish storage measurements and trustworthy health advice](https://git.bongbetic.com/xavierk/odin/issues/4), and [Establish browser, shell, and language workload validity](https://git.bongbetic.com/xavierk/odin/issues/5). All pinned candidates remain subject to build, license, correctness, cancellation and physical-hardware qualification; a failed qualification reopens the relevant workload choice rather than silently changing it.

xavierk closed this issue