Skip to content

fix(memory): stop extract_facts panicking on non-ASCII input#635

Open
harrythomasben wants to merge 1 commit into
GeniePod:mainfrom
harrythomasben:fix/extract-facts-utf8-panic
Open

fix(memory): stop extract_facts panicking on non-ASCII input#635
harrythomasben wants to merge 1 commit into
GeniePod:mainfrom
harrythomasben:fix/extract-facts-utf8-panic

Conversation

@harrythomasben

Copy link
Copy Markdown

Summary

Fixes #634. memory::extract::extract_facts panics on non-ASCII input: the explicit-"remember" fast-path guards a byte-index slice with only a byte-length check, so any utterance whose 8th byte falls inside a multi-byte UTF-8 char slices on a non-char-boundary and crashes. extract_facts runs on raw chat, voice-transcript, and profile-document text, so the input is user-controlled — this takes down fact extraction for any non-Latin-script household.

Changes

  • Add a UTF-8-safe starts_with_ascii_ci(text, prefix) helper that uses str::get(..prefix.len()) (returns None on a non-char-boundary or too-short input) instead of a byte-length guard plus text[..n] slice.
  • Use it at the "remember" call-site guard in extract_facts (was trimmed.len() >= 8 && trimmed[..8].eq_ignore_ascii_case("remember")).
  • Use it for the twin byte slice in extract_remember (text[..PREFIX.len()]), which had the same unsound pattern.
  • Add tests/extract_facts_utf8_test.rs: regression tests for CJK / accented / emoji input (no panic), short input, and the unchanged ASCII "remember" path.

Behavior is unchanged for ASCII input; the only difference is that non-ASCII utterances no longer panic (they correctly fall through the "remember" fast-path).

Real Behavior Proof

  • I have built and run the affected code locally (or noted why I could not).
  • I have verified the change end-to-end on Jetson hardware.
  • I have NOT verified on Jetson hardware, and I explain the equivalent verification path or validation gap below.

Tested profile / hardware (check all that apply):

  • jetson
  • raspberry_pi
  • portable_sbc
  • laptop
  • mac
  • CI-only / docs-only
  • Not run locally

What I ran

On an x86_64 Linux laptop (Rust 1.96):

  • Reproduced the panic against the pre-fix logic with input "日本語hello":
    byte index 8 is not a char boundary; it is inside '語' (bytes 6..9 of string) (process aborted, exit 101).
  • cargo test -p genie-core --test extract_facts_utf8_test → 4 passed.
  • cargo test -p genie-core extract → 11 passed (existing extraction tests, no regression).
  • cargo clippy -p genie-core --tests → clean; cargo fmt applied.

This is a pure input-parsing panic with no hardware, voice, audio, or Home Assistant dependency — the deterministic cargo test repro exercises the full fix path, so Jetson verification is not applicable.

What I observed

After the fix, extract_facts("日本語hello") and the other non-ASCII inputs return normally (no "remember" fact, no panic), while extract_facts("remember to call mom") and "Remember that ..." still extract the fact unchanged.

Test plan

  1. cargo test -p genie-core --test extract_facts_utf8_test
  2. cargo test -p genie-core extract to confirm existing extraction behavior is unchanged.

The explicit-"remember" fast-path in extract_facts guarded a byte-index
slice (trimmed[..8]) with only a byte-length check (trimmed.len() >= 8).
len() counts bytes, so for a 14-byte non-ASCII utterance the guard passed
and trimmed[..8] sliced at byte 8 — inside a multi-byte UTF-8 char — and
panicked, taking down fact extraction for any non-Latin-script household.

Replace the unsound guard with a UTF-8-safe starts_with_ascii_ci helper
(str::get returns None on a non-boundary instead of panicking) and use it
at both the call-site guard and the twin slice in extract_remember.

Fixes GeniePod#634
@github-actions github-actions Bot added the bug Something isn't working label Jul 5, 2026

@matedev01 matedev01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — fixes non-ASCII panic in extract_facts. Needs rebase to resolve merge conflict (unrelated CHANGELOG clash). 826 lib, BFCL 96% after rebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] extract_facts panics on non-ASCII input (UTF-8 char-boundary slice in the "remember" fast-path)

2 participants