Commit 5d5a276
Fix map_query_sql benchmark duplicate key error (#18427)
Fix map_query_sql benchmark duplicate key error
Description
The build_keys() function was generating 1000 random keys from range
0..9999, which could result in duplicate keys due to the birthday
paradox. The map() function requires unique keys, causing the benchmark
to fail with: Execution("map key must be unique, duplicate key found:
{key}")
This fix ensures all generated keys are unique by:
Using a HashSet to track seen keys
Only adding keys to the result if they haven't been seen before
Continuing to generate until exactly 1000 unique keys are produced
Fixes #18421
Which issue does this PR close?
Closes #18421
Rationale for this change
The benchmark was non-deterministic: it could pass or fail depending on
random key generation. With 1000 keys from a range of 9999 values,
collisions are likely (~50% chance), making the benchmark unreliable.
This change ensures uniqueness so the benchmark consistently succeeds
and accurately measures map function performance.
What changes are included in this PR?
Added use std::collections::HashSet; import
Modified build_keys() to:
Track generated keys using a HashSet
Only add keys if they are unique
Continue generating until exactly 1000 unique keys are produced
File changed: datafusion/core/benches/map_query_sql.rs
Code changes:
Added HashSet import at the top of the file
Replaced simple loop with uniqueness-checking logic in build_keys()
function
Are these changes tested?
The fix was verified by:
Logic review: the HashSet approach guarantees uniqueness
Code review: changes follow Rust best practices
No linter errors
The benchmark itself serves as the test — running cargo bench -p
datafusion --bench map_query_sql should now complete without errors.
Before this fix, the benchmark would fail with duplicate key errors in a
significant portion of runs.
Are there any user-facing changes?
No user-facing changes. This is an internal benchmark fix that ensures
the map_query_sql benchmark runs reliably. It does not affect the public
API or any runtime behavior of DataFusion.
---------
Co-authored-by: Jefffrey <[email protected]>1 parent 7fa6378 commit 5d5a276
1 file changed
+6
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
| |||
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
36 | | - | |
37 | | - | |
38 | | - | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
39 | 41 | | |
40 | | - | |
| 42 | + | |
41 | 43 | | |
42 | 44 | | |
43 | 45 | | |
| |||
0 commit comments