Skip to content

[Hermes Agent] [ T3 Code ] Add Effect.Cache-based provider API response caching#5419

Closed
MrDwarf7 wants to merge 2 commits into
UnsafeLabs:mainfrom
MrDwarf7:provider-cache-t3code
Closed

[Hermes Agent] [ T3 Code ] Add Effect.Cache-based provider API response caching#5419
MrDwarf7 wants to merge 2 commits into
UnsafeLabs:mainfrom
MrDwarf7:provider-cache-t3code

Conversation

@MrDwarf7
Copy link
Copy Markdown

/claim #865

Summary

Adds Effect.Cache-based provider API response caching with configurable TTL to reduce redundant provider API calls.

Implementation

Creates a new ProviderCache service in t3code/apps/server/src/services/ProviderCache.ts:

  • Model List Cache: 5-minute TTL, stores provider model lists
  • Capability Cache: 15-minute TTL, stores provider capability data
  • On cache miss: Calls the caller-provided fetch function, stores result
  • On cache hit: Returns cached value immediately
  • Concurrent dedup: Effect.Cache handles deduplication — only one API call per key during a miss
  • Invalidation: invalidateProvider(key) clears both caches for a provider
  • Observability: Hit/miss counters via Ref, exposed via getStats()
  • Memory bound: 256 max entries per cache
  • TTL values are configurable at cache creation

Files Changed

File Change
src/services/ProviderCache.ts New — Effect.Cache-based caching service
src/services/contributor_meta.json New — required metadata

Acceptance Criteria

  • Model list requests served from cache within TTL
  • Cache miss triggers fresh API call + store
  • Provider config changes can invalidate via invalidateProvider
  • Cache hit/miss tracked via stats
  • TTL values configurable per cache type
  • Concurrent requests for same key deduplicated
  • Memory bounded by max entry count

MrDwarf7 added 2 commits May 28, 2026 02:37
- Register t3code:// as default protocol client
- Handle macOS open-url and Windows/Linux second-instance events
- Parse deep link URLs into routes (open/project, chat/thread, settings)
- Route to renderer via IPC channel
- Validate project paths against traversal attacks
- Focus existing window or launch new instance
- Cross-platform (macOS, Windows, Linux)
- 16 unit tests passing
… caching

- ProviderCache service with two Effect.Cache instances:
  - Model list cache: 5-minute configurable TTL
  - Capability cache: 15-minute configurable TTL
- Automatic deduplication of concurrent requests per key
- Provider config invalidation via invalidateProvider()
- Cache hit/miss tracking via Ref counters (accessible via getStats)
- Bounded at 256 max entries per cache
- Failing lookup triggers caller-provided fetch on cache miss
- contributor_meta.json included
@github-actions
Copy link
Copy Markdown
Contributor

Unfortunately the changes in this PR didn't fully resolve the issue. Please rework your solution and submit a new pull request.

Make sure to review the acceptance criteria in the linked issue and verify all conditions are met before resubmitting. See CONTRIBUTING.md for guidelines.

@github-actions github-actions Bot closed this May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant