Skip to content

UNIQUE constraint error in insertEmbedding: vec0 ignores OR REPLACE #445

@expertpartners

Description

@expertpartners

Bug

insertEmbedding in store.js uses INSERT OR REPLACE INTO vectors_vec, but sqlite-vec's vec0 virtual tables silently ignore the OR REPLACE conflict clause. This causes a SqliteError: UNIQUE constraint failed on vectors_vec primary key when re-embedding chunks that already exist in vectors_vec but not in content_vectors (e.g., after a crash mid-embed).

Reproduction

  1. Run qmd embed on a collection
  2. Kill the process mid-embed (or let it crash — e.g., via a TLS error in node-llama-cpp model resolution)
  3. Run qmd embed again
  4. Chunks that were inserted into vectors_vec but not content_vectors before the crash are re-selected by getHashesForEmbedding (which checks only content_vectors)
  5. Re-inserting into vectors_vec fails with UNIQUE constraint error because OR REPLACE is a no-op on vec0

Error output

⚠ Error embedding "file.md" chunk 18: SqliteError: UNIQUE constraint failed on vectors_vec primary key

Suggested fix

Two changes to insertEmbedding in store.js:

  1. Insert into content_vectors first, then vectors_vec. This way, if the process crashes after the first insert, getHashesForEmbedding won't re-select the hash (since content_vectors already has the entry).

  2. Use DELETE + INSERT instead of INSERT OR REPLACE for the vectors_vec table, since vec0 doesn't honor OR REPLACE:

export function insertEmbedding(db, hash, seq, pos, embedding, model, embeddedAt) {
    const hashSeq = `${hash}_${seq}`;
    // Insert content_vectors first so getHashesForEmbedding won't re-select on crash
    const insertContentVectorStmt = db.prepare(
        `INSERT OR REPLACE INTO content_vectors (hash, seq, pos, model, embedded_at) VALUES (?, ?, ?, ?, ?)`
    );
    insertContentVectorStmt.run(hash, seq, pos, model, embeddedAt);
    // vec0 virtual tables don't support OR REPLACE — use DELETE + INSERT instead
    const deleteVecStmt = db.prepare(`DELETE FROM vectors_vec WHERE hash_seq = ?`);
    const insertVecStmt = db.prepare(`INSERT INTO vectors_vec (hash_seq, embedding) VALUES (?, ?)`);
    deleteVecStmt.run(hashSeq);
    insertVecStmt.run(hashSeq, embedding);
}

Environment

  • qmd 1.0.7
  • Node.js v24.13.1
  • macOS (Apple Silicon)
  • sqlite-vec via node-llama-cpp 3.16.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions