OOM'd after long use (3 days);
Here's all the info we could collect:
- v0.5.5, 196K ctx, 20 banks, HBM, FORK_PARTIAL=0
- 3 days uptime → VMM budget (65.56 GiB) fully consumed by accumulated KV pages
- Trim-on-evict (v0.5.1 fix) helps but can't keep up over days of 76K+ token requests
- DS4_BATCH_VMM_BUDGET_MB cap could prevent this — we didn't have it set on the Balanced config
- Workload: agentic (Hermes), 50-76K prompts, DSpark k=2
OOM'd after long use (3 days);
Here's all the info we could collect: