Skip to content

metrics: record MLX nax-capability in run identity - #290

Merged
Pushkinist merged 1 commit into
mainfrom
feat/275-mlx-nax-identity
Jul 21, 2026
Merged

metrics: record MLX nax-capability in run identity#290
Pushkinist merged 1 commit into
mainfrom
feat/275-mlx-nax-identity

Conversation

@Pushkinist

Copy link
Copy Markdown
Owner

Summary

  • Stamp the metallib nax-kernel scan crates/rmlx-mlx/build.rs already computes (fix(mlx): gate the loud nax-kernel warning on Neural-Accelerator-class hosts #262's kernels_present) into a new RMLX_MLX_NAX compile-time constant (present/absent/unknown) — not a second detection path, the same scan the missing-kernel build warning already uses.
  • Thread it into run identity: rmlx-cli::main() calls rmlx_metrics::identity::set_mlx_nax(rmlx_mlx::NAX_CAPABILITY) once at startup (same process-wide one-shot pattern as install_rotor_qjl/install_planar_fused_qk), because rmlx-metrics cannot depend on rmlx-mlx (whose build script hard-requires a working Homebrew MLX/mlx-c install — a dependency this generic, cross-backend metrics crate must not carry) and cargo:rustc-env only reaches the compiler invocation of the crate whose build script set it, never a different crate's.
  • New migration 004_events_mlx_nax.sql adds events.mlx_nax (nullable, append-only); EventRecorder::record now populates it alongside backend_version/build_profile.
  • observations / the §8.5 ingest shape is left untouched — that table's identity fields are strictly binary-known-or-caller-supplied-via---git-sha-style flag, and mlx_nax doesn't fit either slot as a forced column there; a bench script that wants it can already carry it in RunRecord.notes (free-form).
  • Docs: docs/METRICS_DB.md (new column + why the propagation goes through a runtime setter), docs/FFI.md (points at the new run-identity use of the scan), docs/PERF_BASELINE.md (new top-of-file caveat: prefill numbers recorded ~2026-07-13 through the pin restoration on 2026-07-17 are suspect — 2-3.7x inflated, decode unaffected).

#275's historical-row backfill — declined, not done

#275's title also asks to "annotate post-Jul-13 no-NAX prefill rows in runs.db." This PR does not touch runs.db. events is explicitly append-only with no UPDATE path (migration 003_events_identity.sql's stated contract), and CLAUDE.md lists metrics-table mutation as an Ask-before item. Retroactively writing mlx_nax into rows that never recorded it at insert time would fabricate identity the binary didn't know at the time. Documented instead of mutated — see the caveat sections above. The coordinator should confirm with the maintainer whether any other form of historical annotation (e.g. a documented exclusion range in downstream queries) is wanted; this PR takes no action on the DB itself.

Closes #275 (the run-identity half; the historical-row half is intentionally left to the maintainer per the above).

Test plan

  • Unit: EventRecorder::record writes a non-NULL, self-consistent mlx_nax (crates/rmlx-metrics/src/events_tests.rs::record_populates_mlx_nax)
  • Unit: migration 004 adds the column, applies cleanly + idempotently after 003 (schema_tests.rs::events_table_has_mlx_nax_column, existing init_schema_idempotent generically covers ordering/idempotency since MIGRATIONS is a plain ordered slice)
  • Unit: set_mlx_nax/first-writer-wins semantics (identity_tests.rs::mlx_nax_defaults_to_unknown_then_takes_the_first_set_value)
  • Unit: tri-state string mapping (crates/rmlx-mlx/tests/mlx_pin.rs::nax_capability_str_matrix)
  • Real run: rmlx baseline --model gemma-4-e2b-it-mxfp8 --kv-quant none --prompt-tokens 4096 --max-ctx 8192 --record against a scratch RMLX_HOMESELECT DISTINCT mlx_nax FROM events returns present (this dev box's pinned MLX), confirming the real end-to-end plumbing, not just the unit tests. Arch-independent metrics plumbing, so one real model is sufficient (not the ≥2-arch perf-proof rule, which is for perf/kernel claims).
  • make ci: fmt/clippy/deny/audit/ci-metrics/gate-scripts all pass; full cargo test --workspace (unit+integration+doc, every crate) ran to completion with 0 failures — the background invocation was killed by the harness mid doc-tests (external timeout, not a test failure) after every unit/integration test in the workspace had already passed, so deny/audit/ci-metrics/the six gate scripts were re-run standalone afterward and are all green.

🤖 Generated with Claude Code

Stamp the build.rs metallib nax-kernel scan (#262's existing
kernels_present detection) into RMLX_MLX_NAX at compile time, thread it
into rmlx-metrics run identity via a runtime setter (rmlx-metrics
cannot depend on rmlx-mlx or read its cargo:rustc-env directly), and
record it as events.mlx_nax (migration 004) so bench rows are
self-describing about whether they ran against a nax-capable MLX.

Declines the runs.db historical-row backfill from #275's title: events
is append-only with no UPDATE path, so the suspect ~2026-07-13 no-NAX
window is documented instead (docs/PERF_BASELINE.md,
.rmlx/mlx-homebrew-nax-regression.md) rather than mutated in.
@Pushkinist
Pushkinist merged commit d8d7463 into main Jul 21, 2026
9 checks passed
@Pushkinist
Pushkinist deleted the feat/275-mlx-nax-identity branch July 21, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

metrics: record MLX nax-capability in run identity + annotate post-Jul-13 no-NAX prefill rows in runs.db

1 participant