Embedding pegs all CPU cores on low-spec machines — make MEMEX_EMBED_THREADS default-capped (post-v0.1.2)

[sgwannabe]

Summary

On low-spec machines (e.g. an 8 GB M1 Air), embedding pegs every CPU core during indexing / watcher warm-up, causing UI lag. We have the knob to cap it, but it is opt-in and the released app does not ship it at all. This issue tracks turning it into a sane default in a future release.

Current state (verified)

• fastembed defaults ONNX intra-op threads to available_parallelism() (all cores).
• Our teammate contributed with_intra_threads upstream — feat: configurable ONNX Runtime intra-op thread count (with_intra_threads) Anush008/fastembed-rs#255, merged, shipped in fastembed 5.15.
• main wires it as MEMEX_EMBED_THREADS (commit a0e7453, indexer::embed_intra_threads()), but it is opt-in: unset → fastembed's all-core default is left unchanged. So the default behavior still pegs all cores.
• The released v0.1.2 binary pins fastembed 5.14 and predates a0e7453, so it has no cap at all — low-spec users have no workaround.

	thread-cap code	fastembed	low-spec default
v0.1.2 (released)	❌ absent	5.14	pegs all cores, no escape
main / source	✅ present (opt-in)	5.15	pegs all cores unless MEMEX_EMBED_THREADS set

Proposal

When MEMEX_EMBED_THREADS is unset, default to a sane cap instead of all cores — e.g. max(1, available_parallelism() - 1) (or a lower cap on small core counts) — while keeping the explicit env override. Validate the indexing-throughput trade-off before changing the default.

Decision / scope

• No release before the 2026-06-01 hackathon deadline. v0.1.2 is signed + notarized + on Homebrew and works; the release path has notarization flakiness (a v0.1.2 attempt failed before succeeding) and a manual Homebrew-cask sha bump, so re-cutting at the buzzer is not worth it.
• The lag is a temporary CPU spike during indexing/warm-up (BGE-small is light), not a permanent freeze.
• Address post-submission and ship the default-cap in the next release (v0.1.3).

References

• Upstream: feat: configurable ONNX Runtime intra-op thread count (with_intra_threads) Anush008/fastembed-rs#255
• Wiring: src-tauri/src/indexer.rs (embed_intra_threads, Embedder::new), src-tauri/src/watcher.rs
• Commit: a0e7453 (perf(embed): cap ONNX intra-op threads via MEMEX_EMBED_THREADS)

[ComBba]

Reviewed — facts all check out (PR #40 / a0e7453 merged to main as 5bbb886; embed_intra_threads() is opt-in; v0.1.2 predates fastembed 5.15 so it has no cap). Agree with deferring to v0.1.3 and not re-cutting a release before the deadline.

Three notes for whoever implements the default-cap, so we don't ship a cap that doesn't actually help:

1. max(1, available_parallelism() - 1) is too weak. On an 8-core M1 Air that's still 7 intra-op threads (~700% potential) — barely moves the needle on the UI-lag this issue targets. Worth considering cores/2 or min(4, cores - 1), with a lower floor on small core counts. (As the proposal already flags, validate against indexing throughput before committing to a number.)

2. It multiplies with the embed_pool Semaphore. The pool already caps concurrent embed jobs at num_cpus/2; total load ≈ (concurrent jobs) × (intra-op threads per job). A default of cores-1 combined with cores/2 concurrent jobs is heavy oversubscription. The default intra cap should be reasoned about together with the Semaphore (e.g. intra 2–4 × the existing concurrency cap), not in isolation.

3. Add an explicit escape hatch. embed_intra_threads() currently does filter(|&n| n >= 1), so MEMEX_EMBED_THREADS=0 is ignored. When a default cap lands, giving 0 the meaning "uncapped (all cores)" lets power users explicitly restore today's behavior.

No action needed now — just capturing these so the v0.1.3 implementation starts from the right design.