Skip to content

Possible memory leak #11

Description

@styles01

OOM'd after long use (3 days);
Here's all the info we could collect:

  • v0.5.5, 196K ctx, 20 banks, HBM, FORK_PARTIAL=0
  • 3 days uptime → VMM budget (65.56 GiB) fully consumed by accumulated KV pages
  • Trim-on-evict (v0.5.1 fix) helps but can't keep up over days of 76K+ token requests
  • DS4_BATCH_VMM_BUDGET_MB cap could prevent this — we didn't have it set on the Balanced config
  • Workload: agentic (Hermes), 50-76K prompts, DSpark k=2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions