feat(cache): bypass tool cache when user asks for fresh data#515
Merged
Conversation
## Priority 1 — Smart home device feel The read-only tool cache (#505) is a big latency win for back-to-back follow-ups ("what's the weather?" → "will I need a coat?"). But it also serves stale data when the user *explicitly* asks for fresh: "what's the LATEST weather?" or "最新の天気を教えて" would still be answered from the 5-minute-old cache entry, silently defeating user intent. RefreshIntent scans the user utterance for narrow English / Japanese freshness markers ("latest", "refresh", "update", "right now", "current", "最新", "今の", "更新", "再読み込み", "リロード"). When matched, VoicePipeline.processUserInput clears the entire CachingToolExecutor before routing, so both fast-path and LLM paths hit the underlying tool delegate. The cache rebuilds naturally on the next identical query within the TTL. Keywords are kept narrow on purpose: false positives would silently defeat the latency win of the cache. "Weather" alone doesn't qualify, only explicit freshness markers. Word-boundary English regex prevents substring traps ("updatevehicle" won't trigger). Japanese stays substring-matched (no word boundaries) but the set is small enough to avoid incidental matches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Priority 1 — Smart home device feel
Summary
The read-only tool cache (#505) is a big latency win for back-to-back follow-ups ("what's the weather?" → "will I need a coat?"). But it also serves stale data when the user explicitly asks for fresh: "what's the latest weather?" or "最新の天気を教えて" would still be answered from the 5-minute-old cache entry, silently defeating user intent.
RefreshIntentscans the user utterance for narrow English / Japanese freshness markers ("latest","refresh","update","right now","current","最新","今の","更新","再読み込み","リロード"). When matched,VoicePipeline.processUserInputclears the entireCachingToolExecutorbefore routing, so both fast-path and LLM paths hit the underlying tool delegate. The cache rebuilds naturally on the next identical query.Narrow-by-design
False positives would silently defeat the latency win of the cache. "Weather" alone doesn't qualify — only explicit freshness markers. Word-boundary English regex prevents substring traps ("updatevehicle" won't trigger). Japanese stays substring-matched but the set is small enough to avoid incidental matches.
Test plan
RefreshIntentTestcovers: English keywords, Japanese keywords, neutral queries don't trigger, empty/blank, case-insensitivity, word-boundary guard../gradlew :app:testStandardDebugUnitTest— green./gradlew :app:assembleStandardDebug— green