Skip to content

(WiP) feat: Add cost-based usage limits: UsageLimits.cost_limit_usd and BudgetGuard#1

Open
OchnikBartek wants to merge 3 commits into
mainfrom
feat/cost-limit-usd
Open

(WiP) feat: Add cost-based usage limits: UsageLimits.cost_limit_usd and BudgetGuard#1
OchnikBartek wants to merge 3 commits into
mainfrom
feat/cost-limit-usd

Conversation

@OchnikBartek

Copy link
Copy Markdown
Member

Summary

Adds cost-based usage limits in two complementary layers:

  • UsageLimits.cost_limit_usd — per-run cost cap, checked after each model response. Accepts int/str/Decimal (rejects float/bool). Accumulated on RunUsage.total_cost_usd via genai-prices; fail-open when a model is unpriced.
  • BudgetGuard capability — cumulative budget across runs over a rolling window, with a pluggable BudgetStore (InMemoryBudgetStore, SQLiteBudgetStore) and per-tenant key_fn.

Adds ModelResponse.cost_or_none(), docs (docs/budget.md, cost_limit_usd section in docs/agent.md, API reference), and tests.

What to review

  • Is the BudgetGuard API ergonomic for the client-API-key use case? (key_fn, per-tenant buckets)
  • Sensible defaults? (window_hours=24, retention_hours=168, fail_open=False)
  • Does cost_limit_usd as a per-run cap cover the need?
  • The provider_name == 'test' / function: short-circuit in cost_or_none — OK or smell?

Test plan

  • make lint — clean
  • make typecheck (pyright + mypy) — clean
  • make test — 70 new tests in tests/test_budget.py + tests/test_usage_limits.py, full suite green
  • make docs — builds clean

@OchnikBartek OchnikBartek requested a review from DEENUU1 June 1, 2026 09:44
@OchnikBartek OchnikBartek self-assigned this Jun 1, 2026
@OchnikBartek OchnikBartek added the enhancement New feature or request label Jun 1, 2026
@DEENUU1 DEENUU1 changed the title feat: Add cost-based usage limits: UsageLimits.cost_limit_usd and BudgetGuard (WiP) feat: Add cost-based usage limits: UsageLimits.cost_limit_usd and BudgetGuard Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size: L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant