metrics: record MLX nax-capability in run identity - #290
Merged
Conversation
Stamp the build.rs metallib nax-kernel scan (#262's existing kernels_present detection) into RMLX_MLX_NAX at compile time, thread it into rmlx-metrics run identity via a runtime setter (rmlx-metrics cannot depend on rmlx-mlx or read its cargo:rustc-env directly), and record it as events.mlx_nax (migration 004) so bench rows are self-describing about whether they ran against a nax-capable MLX. Declines the runs.db historical-row backfill from #275's title: events is append-only with no UPDATE path, so the suspect ~2026-07-13 no-NAX window is documented instead (docs/PERF_BASELINE.md, .rmlx/mlx-homebrew-nax-regression.md) rather than mutated in.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
crates/rmlx-mlx/build.rsalready computes (fix(mlx): gate the loud nax-kernel warning on Neural-Accelerator-class hosts #262'skernels_present) into a newRMLX_MLX_NAXcompile-time constant (present/absent/unknown) — not a second detection path, the same scan the missing-kernel build warning already uses.rmlx-cli::main()callsrmlx_metrics::identity::set_mlx_nax(rmlx_mlx::NAX_CAPABILITY)once at startup (same process-wide one-shot pattern asinstall_rotor_qjl/install_planar_fused_qk), becausermlx-metricscannot depend onrmlx-mlx(whose build script hard-requires a working Homebrew MLX/mlx-c install — a dependency this generic, cross-backend metrics crate must not carry) andcargo:rustc-envonly reaches the compiler invocation of the crate whose build script set it, never a different crate's.004_events_mlx_nax.sqladdsevents.mlx_nax(nullable, append-only);EventRecorder::recordnow populates it alongsidebackend_version/build_profile.observations/ the §8.5 ingest shape is left untouched — that table's identity fields are strictly binary-known-or-caller-supplied-via---git-sha-style flag, andmlx_naxdoesn't fit either slot as a forced column there; a bench script that wants it can already carry it inRunRecord.notes(free-form).docs/METRICS_DB.md(new column + why the propagation goes through a runtime setter),docs/FFI.md(points at the new run-identity use of the scan),docs/PERF_BASELINE.md(new top-of-file caveat: prefill numbers recorded ~2026-07-13 through the pin restoration on 2026-07-17 are suspect — 2-3.7x inflated, decode unaffected).#275's historical-row backfill — declined, not done
#275's title also asks to "annotate post-Jul-13 no-NAX prefill rows in runs.db." This PR does not touch
runs.db.eventsis explicitly append-only with noUPDATEpath (migration003_events_identity.sql's stated contract), and CLAUDE.md lists metrics-table mutation as an Ask-before item. Retroactively writingmlx_naxinto rows that never recorded it at insert time would fabricate identity the binary didn't know at the time. Documented instead of mutated — see the caveat sections above. The coordinator should confirm with the maintainer whether any other form of historical annotation (e.g. a documented exclusion range in downstream queries) is wanted; this PR takes no action on the DB itself.Closes #275 (the run-identity half; the historical-row half is intentionally left to the maintainer per the above).
Test plan
EventRecorder::recordwrites a non-NULL, self-consistentmlx_nax(crates/rmlx-metrics/src/events_tests.rs::record_populates_mlx_nax)schema_tests.rs::events_table_has_mlx_nax_column, existinginit_schema_idempotentgenerically covers ordering/idempotency sinceMIGRATIONSis a plain ordered slice)set_mlx_nax/first-writer-wins semantics (identity_tests.rs::mlx_nax_defaults_to_unknown_then_takes_the_first_set_value)crates/rmlx-mlx/tests/mlx_pin.rs::nax_capability_str_matrix)rmlx baseline --model gemma-4-e2b-it-mxfp8 --kv-quant none --prompt-tokens 4096 --max-ctx 8192 --recordagainst a scratchRMLX_HOME→SELECT DISTINCT mlx_nax FROM eventsreturnspresent(this dev box's pinned MLX), confirming the real end-to-end plumbing, not just the unit tests. Arch-independent metrics plumbing, so one real model is sufficient (not the ≥2-arch perf-proof rule, which is for perf/kernel claims).make ci: fmt/clippy/deny/audit/ci-metrics/gate-scripts all pass; fullcargo test --workspace(unit+integration+doc, every crate) ran to completion with 0 failures — the background invocation was killed by the harness mid doc-tests (external timeout, not a test failure) after every unit/integration test in the workspace had already passed, sodeny/audit/ci-metrics/the six gate scripts were re-run standalone afterward and are all green.🤖 Generated with Claude Code