Repository navigation
Use NEON packing for wide primitive array comparisons - #10843
Conversation
Use the existing fill-and-pack collector for i64, u64, and f64 array pairs on AArch64 with NEON. Keep this dispatch and its bounds proof inside the private primitive comparison kernel. Extend word-boundary coverage to wide types with sliced inputs and exercise floating-point total ordering through full words and a tail. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Merging this PR will regress 1 benchmark4 warnings, 1 note
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing 
Footnotes
|
| f: impl Fn(T, T) -> bool, | ||
| allocator: &BufferAllocatorRef, | ||
| ) -> BitBuffer { | ||
| #[cfg(all(target_arch = "aarch64", target_feature = "neon"))] |
There was a problem hiding this comment.
I don't think neon is an architecture though?
|
From Claude so take with a grain of salt:
DetailsNow the two lower-level kernels: the existing `collect_zip_bits` lane kernel and the multiversioned fill-and-pack loop.The short answer is that x86 can turn a vector compare into a bitmask in one instruction, at every element width. NEON can't. The lane kernel is near-optimal on x86 and expensive on ARM. This is reasoning from the code, not from reading the generated assembly, so treat it as a hypothesis. The two kernel shapes
Why x86 gains nothing
Why NEON gains
Two side notes
If you want to confirm the core claim cheaply, look at the aarch64 assembly for the i64 lane kernel. If it is scalar |
Related to #10676. This PR is independent of the RowFn stack.
Uses the existing fill-and-pack collector for
i64,u64, andf64array-pair comparisons on AArch64 with NEON. The dispatch and the unchecked slice reads stay inside the private primitive kernel, where equal lengths and the collector's index range establish the bounds proof. Constant comparisons, other primitive types, and validity handling retain their existing paths.Extends bitmap-boundary tests to wide types with differently sliced inputs, and checks floating-point total ordering through full words and a tail.
Validation: 236 comparison tests and all-target, all-feature Clippy with warnings denied pass on macOS ARM. Native ARM and AVX2 builds pass the 455-case probe (452 reference-array checks plus three unchanged Boolean controls) and 3,456 comparison edge checks plus four arithmetic error checks.
On Graviton3, the two counterbalanced full-suite sessions show the following elapsed-time reductions in the long array-pair cohorts:
i640.846 [0.843, 0.856]u640.860 [0.858, 0.864]f640.882 [0.880, 0.886]Each cohort contains 32 cases. Ratios are candidate/baseline, with a geometric mean across cases within a paired block and the median across 21 blocks. Intervals bootstrap those paired block ratios. Neither ARM session has a case crossing the 3% regression threshold with its interval above 1.
The AVX2 baseline and candidate
.textsections are byte-for-byte identical. Control timings still vary, so no x86 speedup is attributed to this change. These are public-path microbenchmarks, not query results. Release-profile and AVX-512 timing were not repeated for this ARM-only variant.Measurement protocol and reproduction
Baseline:
a59294e35e7109fac894a7a38a68f42db0c3c967. Candidate:f018aaeef739145e1bc50079a441c5d9fd53404a.Uses rustc 1.99.1 (LLVM 23.1.1), the repository's bench profile (16 codegen units, no LTO), MiMalloc, and CPU 2. Graviton3 uses
-C target-feature=+neon -C force-frame-pointers=yes. The AVX2 control uses-C target-feature=+avx2 -C force-frame-pointers=yeson AMD EPYC 7R13. Optimized LLVM IR and assembly were emitted during both builds, and the resulting executables were timed without instrumentation.Both labels run the same full 455-case suite in the same order. Each case has exactly 1,000 single-iteration samples. Each session has 21 baseline/baseline calibration blocks and 21 paired baseline/candidate blocks with alternating executable order. Separate arithmetic control binaries bracket the blocks. The two hosts are measured serially, and a second session confirms the result. The confirmation session reverses the starting order, so the combined sessions contain equal AB and BA counts. Preliminary fixed-order source comparisons were excluded, while valid identical-binary calibration was retained. The long-array cohorts each contain 32 cases, including partially valid inputs and lengths around the 16K boundary.
To reproduce, save the harness below as
vortex-array/benches/interaction_probe.rson each revision and add this target tovortex-array/Cargo.toml:Set
RUSTFLAGSto the host flags above. Build both revisions with those same flags, retain separate immutable executables, and alternate the commands below. UnsetVORTEX_PERF_CHECKfor timing.Exact benchmark harness
The measured harness SHA-256 is
0a7de7999381d237ad0471716107f7e1972a7c6b11c0936a7d3b7d8c15b6c37d.