You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In any household with more than one speaker, GenieClaw has no way to tell users apart. SpeakerIdentityProvider::LocalBiometric is selectable in geniepod.toml and fully
documented in genie-common/src/config.rs, but voice/identity.rs contains only a
detection stub — enrollment, embedding extraction, and match scoring are never
implemented. The CLI commands genie-ctl speaker enroll <name>, speaker list, and speaker remove <name> exist in genie-ctl/src/main.rs but return stub errors.
This causes three user-visible failures:
Memory bleeds across household members.memory/policy.rs has person-scoped
read/write gates that only fire when identity resolves to a known member. Because
resolution always returns Unknown, personal preferences, private facts, and
health-adjacent memories from any speaker surface in any conversation. This is also
the root cause behind the write-side gap in [bug] memory_store writes person-scoped memories without verified identity context (#430 write-side) #454 — enforcing that gate is impossible
without a working identity layer underneath it.
Voice-triggered personalization is dead code.tools/dispatch.rs threads speaker
context into tool calls (timer labels, preference lookups, person-targeted HA actions).
With permanent Unknown identity, that path never fires.
genie-ctl speaker enroll is a broken user-facing contract. The command is
documented in the README and present in the CLI help text but exits with a stub error,
which is confusing and erodes trust in the tooling.
Proposed feature
When speaker_identity.provider = "local_biometric" is set, GenieClaw should:
Enroll household members by capturing N audio samples (min_enrollment_samples),
extracting speaker embeddings via a quantized on-device model (e.g. ECAPA-TDNN ONNX,
already assumed for Jetson in the config comments), and persisting voiceprints to a speaker_voiceprints table in the existing conversations.db.
Identify the speaker at runtime for each VAD-gated audio chunk before STT decodes
it. Emit SpeakerIdentity::Known { name, confidence } or SpeakerIdentity::Unknown
gated on a configurable similarity_threshold.
Propagate the resolved identity through voice_loop.rs → server.rs into memory
writes, recall, tool dispatch, and conversation turn tags so every subsystem sees a
consistent Option<SpeakerIdentity> per turn.
Expose enrollment and management via the already-stubbed genie-ctl speaker
subcommands.
Replace stub with enrollment DB + embedding extraction + match scoring
crates/genie-core/src/voice/stt.rs
Feed audio buffer to identity pipeline before STT decode
crates/genie-core/src/voice_loop.rs
Thread SpeakerIdentity result into conversation context
crates/genie-core/src/server.rs
Pass resolved identity to memory writes and tool dispatch
crates/genie-core/src/memory/mod.rs
Enforce person-scoped writes using verified identity
crates/genie-core/src/memory/policy.rs
Gate person-scoped reads on confirmed SpeakerIdentity::Known
crates/genie-core/src/tools/dispatch.rs
Thread speaker identity through tool calls that reference it
crates/genie-core/src/conversation.rs
Tag conversation turns with speaker name
crates/genie-common/src/config.rs
Add BiometricConfig sub-struct under SpeakerIdentityConfig
crates/genie-ctl/src/main.rs
Implement enroll / list / remove CLI commands
crates/genie-core/src/security/audit.rs
Emit audit events for identity resolution and enrollment
Alternatives considered
Fixed identity (provider = "fixed") — works for single-user households but
provides no differentiation and no fix for multi-user memory gating.
Telegram sender ID as identity — channel-specific, doesn't help for voice or HTTP
sessions, and bypasses the voice privacy boundary entirely.
Cloud speaker diarization — contradicts the local-only audio posture; all audio
must stay on-device.
Which milestone is this aimed at?
M3 — Home runtime boundary and skill safety
Privacy / security considerations
All audio and voiceprints stay on-device. The embedding model runs locally (ONNX on
Jetson); no audio, transcript, or biometric data leaves the box. Voiceprints are stored
in the existing conversations.db under the same file-permission boundary as the rest
of the runtime data. Enrollment events and identity resolution results are logged to security/audit.rs for operator review. The similarity_threshold config lets operators
tune the false-accept rate for their household.
Additional context
The audio pipeline (ALSA capture, Silero VAD, chunking) and the ONNX runtime are already
present on Jetson. The gap is purely the enrollment store, embedding extraction call, and
identity propagation through the conversation context — no new hardware or external
dependencies are required.
What problem does this solve?
In any household with more than one speaker, GenieClaw has no way to tell users apart.
SpeakerIdentityProvider::LocalBiometricis selectable ingeniepod.tomland fullydocumented in
genie-common/src/config.rs, butvoice/identity.rscontains only adetection stub — enrollment, embedding extraction, and match scoring are never
implemented. The CLI commands
genie-ctl speaker enroll <name>,speaker list, andspeaker remove <name>exist ingenie-ctl/src/main.rsbut return stub errors.This causes three user-visible failures:
Memory bleeds across household members.
memory/policy.rshas person-scopedread/write gates that only fire when identity resolves to a known member. Because
resolution always returns
Unknown, personal preferences, private facts, andhealth-adjacent memories from any speaker surface in any conversation. This is also
the root cause behind the write-side gap in [bug] memory_store writes person-scoped memories without verified identity context (#430 write-side) #454 — enforcing that gate is impossible
without a working identity layer underneath it.
Voice-triggered personalization is dead code.
tools/dispatch.rsthreads speakercontext into tool calls (timer labels, preference lookups, person-targeted HA actions).
With permanent
Unknownidentity, that path never fires.genie-ctl speaker enrollis a broken user-facing contract. The command isdocumented in the README and present in the CLI help text but exits with a stub error,
which is confusing and erodes trust in the tooling.
Proposed feature
When
speaker_identity.provider = "local_biometric"is set, GenieClaw should:Enroll household members by capturing N audio samples (
min_enrollment_samples),extracting speaker embeddings via a quantized on-device model (e.g. ECAPA-TDNN ONNX,
already assumed for Jetson in the config comments), and persisting voiceprints to a
speaker_voiceprintstable in the existingconversations.db.Identify the speaker at runtime for each VAD-gated audio chunk before STT decodes
it. Emit
SpeakerIdentity::Known { name, confidence }orSpeakerIdentity::Unknowngated on a configurable
similarity_threshold.Propagate the resolved identity through
voice_loop.rs→server.rsinto memorywrites, recall, tool dispatch, and conversation turn tags so every subsystem sees a
consistent
Option<SpeakerIdentity>per turn.Expose enrollment and management via the already-stubbed
genie-ctl speakersubcommands.
Config surface addition:
Estimated files touched (11):
crates/genie-core/src/voice/identity.rscrates/genie-core/src/voice/stt.rscrates/genie-core/src/voice_loop.rsSpeakerIdentityresult into conversation contextcrates/genie-core/src/server.rscrates/genie-core/src/memory/mod.rscrates/genie-core/src/memory/policy.rsSpeakerIdentity::Knowncrates/genie-core/src/tools/dispatch.rscrates/genie-core/src/conversation.rscrates/genie-common/src/config.rsBiometricConfigsub-struct underSpeakerIdentityConfigcrates/genie-ctl/src/main.rscrates/genie-core/src/security/audit.rsAlternatives considered
provider = "fixed") — works for single-user households butprovides no differentiation and no fix for multi-user memory gating.
sessions, and bypasses the voice privacy boundary entirely.
must stay on-device.
Which milestone is this aimed at?
M3 — Home runtime boundary and skill safety
Privacy / security considerations
All audio and voiceprints stay on-device. The embedding model runs locally (ONNX on
Jetson); no audio, transcript, or biometric data leaves the box. Voiceprints are stored
in the existing
conversations.dbunder the same file-permission boundary as the restof the runtime data. Enrollment events and identity resolution results are logged to
security/audit.rsfor operator review. Thesimilarity_thresholdconfig lets operatorstune the false-accept rate for their household.
Additional context
The audio pipeline (ALSA capture, Silero VAD, chunking) and the ONNX runtime are already
present on Jetson. The gap is purely the enrollment store, embedding extraction call, and
identity propagation through the conversation context — no new hardware or external
dependencies are required.