Skip to content

[BUG] TestLucene99AcceleratedHNSWVectorsFormat.testMergeTwoSegsWithASingleDocPerSeg fails with vector data mismatch after forceMerge(1) #2550

Description

@imotov

Describe the bug
testMergeTwoSegsWithASingleDocPerSeg fails deterministically with seed 599D21DD0964F401

Steps/Code to reproduce bug

cd java/cuvs-lucene
mvn test -Dtest=TestLucene99AcceleratedHNSWVectorsFormat#testMergeTwoSegsWithASingleDocPerSeg -Dtests.seed=599D21DD0964F401

Failure:

arrays first differed at element [0]; expected:<0.9042659> but was:<0.8326107>

Expected behavior
Test passes.

Environment details (please complete the following information):

  • originally found on arm64 PR build, but reproduced on amd64.

Additional context
Confirmed via a local diagnostic patch that 0.9042659 is f[0][0] (doc 0's indexed vector) and 0.8326107 is exactly f[1][0] (doc 1's indexed vector) — values.vectorValue(0) after merge returns doc 1's vector verbatim. This looks like a doc-ordinal swap.

getFloatVectorValues/vectorValue() are served entirely by the standard Lucene flat-vector format (not CuVS-specific graph code), so the bug is in how the flat storage gets merged. Lucene99AcceleratedHNSWVectorsWriter.mergeOneField() runs two independent merge iterations over the same MergeState/field — flatVectorsWriter.mergeOneField(fieldInfo, mergeState), then vectorBasedMerge(fieldInfo, mergeState) (which separately calls KnnVectorsWriter.MergedVectorValues.mergeFloatVectorValues in Utils.createListFromMergedVectors). If the underlying flat writer defers its actual merge to finish(), the second (CAGRA-build) iteration could run first and disturb shared per-segment reader/iterator state before the flat writer's own (later) pass executes - worth checking whether mergeState's vector readers are safely re-iterable across two independent merge passes for the same field.

There is a similar error in TestLucene99AcceleratedHNSWVectorsFormat.testTwoVectorFieldsPerDoc with seed EEBDAA2298537BF7.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

LucenebugSomething isn't working

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions