-
Notifications
You must be signed in to change notification settings - Fork 987
Open
Description
Bug
insertEmbedding in store.js uses INSERT OR REPLACE INTO vectors_vec, but sqlite-vec's vec0 virtual tables silently ignore the OR REPLACE conflict clause. This causes a SqliteError: UNIQUE constraint failed on vectors_vec primary key when re-embedding chunks that already exist in vectors_vec but not in content_vectors (e.g., after a crash mid-embed).
Reproduction
- Run
qmd embedon a collection - Kill the process mid-embed (or let it crash — e.g., via a TLS error in node-llama-cpp model resolution)
- Run
qmd embedagain - Chunks that were inserted into
vectors_vecbut notcontent_vectorsbefore the crash are re-selected bygetHashesForEmbedding(which checks onlycontent_vectors) - Re-inserting into
vectors_vecfails with UNIQUE constraint error becauseOR REPLACEis a no-op on vec0
Error output
⚠ Error embedding "file.md" chunk 18: SqliteError: UNIQUE constraint failed on vectors_vec primary key
Suggested fix
Two changes to insertEmbedding in store.js:
-
Insert into
content_vectorsfirst, thenvectors_vec. This way, if the process crashes after the first insert,getHashesForEmbeddingwon't re-select the hash (sincecontent_vectorsalready has the entry). -
Use
DELETE+INSERTinstead ofINSERT OR REPLACEfor thevectors_vectable, since vec0 doesn't honorOR REPLACE:
export function insertEmbedding(db, hash, seq, pos, embedding, model, embeddedAt) {
const hashSeq = `${hash}_${seq}`;
// Insert content_vectors first so getHashesForEmbedding won't re-select on crash
const insertContentVectorStmt = db.prepare(
`INSERT OR REPLACE INTO content_vectors (hash, seq, pos, model, embedded_at) VALUES (?, ?, ?, ?, ?)`
);
insertContentVectorStmt.run(hash, seq, pos, model, embeddedAt);
// vec0 virtual tables don't support OR REPLACE — use DELETE + INSERT instead
const deleteVecStmt = db.prepare(`DELETE FROM vectors_vec WHERE hash_seq = ?`);
const insertVecStmt = db.prepare(`INSERT INTO vectors_vec (hash_seq, embedding) VALUES (?, ?)`);
deleteVecStmt.run(hashSeq);
insertVecStmt.run(hashSeq, embedding);
}Environment
- qmd 1.0.7
- Node.js v24.13.1
- macOS (Apple Silicon)
- sqlite-vec via node-llama-cpp 3.16.2
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels