Skip to content

Commit 6729c94

Browse files
committed
fix: strengthen async test assertion and fix README example
- Fix weak async cost_fn test: use distinct value (1500) vs estimate (1000) so test fails if cost_fn is ignored - Fix README streaming example: add missing variable definitions (max_tokens, openai_client, stream creation) so example is runnable - Update AUDIT.md: correct test count (64), add validation and IDEMPOTENCY_MISMATCH details - Update examples/README.md: streaming_usage.py description
1 parent 94170b9 commit 6729c94

4 files changed

Lines changed: 30 additions & 13 deletions

File tree

AUDIT.md

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,8 @@ Automated contract tests validate sample request/response payloads against the O
188188
## Streaming Convenience Module (added 2026-04-08)
189189

190190
**Module:** `runcycles/streaming.py`
191-
**Test file:** `tests/test_streaming.py` (57 tests, all passing)
191+
**Test file:** `tests/test_streaming.py` (64 tests, all passing)
192+
**Version:** 0.3.0
192193

193194
Added `StreamReservation` and `AsyncStreamReservation` context managers that automate the reserve → commit/release lifecycle for streaming use cases. This is a DX convenience layer — no protocol changes.
194195

@@ -197,8 +198,10 @@ Added `StreamReservation` and `AsyncStreamReservation` context managers that aut
197198
- **`StreamUsage`** — mutable accumulator for token counts and cost during streaming
198199
- **Client convenience methods:** `CyclesClient.stream_reservation()` and `AsyncCyclesClient.stream_reservation()` — thin factories that build Subject from config defaults
199200
- **Cost resolution:** explicit `usage.actual_cost` > `cost_fn(usage)` > estimate fallback
200-
- **Heartbeat:** automatic TTL extension, same interval formula as decorator lifecycle
201+
- **Heartbeat:** automatic TTL extension, same interval formula as decorator lifecycle (`max(ttl_ms / 2, 1000)` ms)
201202
- **Commit retry:** uses existing `CommitRetryEngine`/`AsyncCommitRetryEngine`
202-
- **Context propagation:** sets/clears `CyclesContext` via `ContextVar`, accessible via `get_cycles_context()`
203+
- **Context propagation:** sets/clears `CyclesContext` via `ContextVar`, accessible via `get_cycles_context()`; respects user-set `ctx.metrics` during streaming
204+
- **Spec validation:** `validate_ttl_ms()` (1000–86400000), `validate_grace_period_ms()` (0–60000), `validate_subject()` (at least one standard field) — matches lifecycle.py
205+
- **Error handling:** `RESERVATION_FINALIZED`, `RESERVATION_EXPIRED`, and `IDEMPOTENCY_MISMATCH` do not trigger release; other 4xx client errors do trigger release — matches lifecycle.py behavior exactly
203206

204-
Protocol conformance: No new endpoints or protocol changes. All reservation, commit, release, and extend calls use the same client methods and body formats as the decorator path. Verified by 57 unit tests covering success, deny, error, retry, heartbeat, cost resolution, and context propagation.
207+
Protocol conformance: No new endpoints or protocol changes. All reservation, commit, release, and extend calls use the same client methods and body formats as the decorator path. Verified by 64 unit tests covering success, deny, error, retry, heartbeat, cost resolution, context propagation, spec validation, and all commit error-code branches.

README.md

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -133,25 +133,38 @@ result = await call_llm("Hello")
133133
For streaming LLM responses, use the `stream_reservation()` context manager. It reserves budget on enter, auto-commits on successful exit, and auto-releases on exception:
134134
135135
```python
136+
from openai import OpenAI
136137
from runcycles import CyclesClient, CyclesConfig, Action, Amount, Unit
137138
138139
config = CyclesConfig(base_url="http://localhost:7878", api_key="your-api-key", tenant="acme")
139-
client = CyclesClient(config)
140+
cycles_client = CyclesClient(config)
141+
openai_client = OpenAI()
142+
max_tokens = 1024
140143
141-
with client.stream_reservation(
144+
with cycles_client.stream_reservation(
142145
action=Action(kind="llm.completion", name="gpt-4o"),
143146
estimate=Amount(unit=Unit.USD_MICROCENTS, amount=max_tokens * 1000),
144147
cost_fn=lambda u: u.tokens_input * 250 + u.tokens_output * 1000,
145148
) as reservation:
146-
# Caps available immediately
149+
# Caps available immediately after entering the context
147150
if reservation.caps and reservation.caps.max_tokens:
148151
max_tokens = min(max_tokens, reservation.caps.max_tokens)
149152
150-
for chunk in openai_stream:
153+
stream = openai_client.chat.completions.create(
154+
model="gpt-4o",
155+
messages=[{"role": "user", "content": "Hello"}],
156+
max_tokens=max_tokens,
157+
stream=True,
158+
stream_options={"include_usage": True},
159+
)
160+
161+
for chunk in stream:
162+
if chunk.choices and chunk.choices[0].delta.content:
163+
print(chunk.choices[0].delta.content, end="", flush=True)
151164
if chunk.usage:
152165
reservation.usage.tokens_input = chunk.usage.prompt_tokens
153166
reservation.usage.tokens_output = chunk.usage.completion_tokens
154-
# Committed automatically with actual cost from cost_fn
167+
# Committed automatically with actual cost computed by cost_fn
155168
```
156169
157170
Also available as `async with client.stream_reservation(...)` for async clients. See [streaming_usage.py](examples/streaming_usage.py) for a complete example.
@@ -402,7 +415,7 @@ The [`examples/`](examples/) directory contains runnable integration examples:
402415
| [async_usage.py](examples/async_usage.py) | Async client and async decorator |
403416
| [openai_integration.py](examples/openai_integration.py) | Guard OpenAI chat completions with budget checks |
404417
| [anthropic_integration.py](examples/anthropic_integration.py) | Guard Anthropic messages with per-tool budget tracking |
405-
| [streaming_usage.py](examples/streaming_usage.py) | Budget-managed streaming with token accumulation |
418+
| [streaming_usage.py](examples/streaming_usage.py) | `stream_reservation()` context manager with auto-commit |
406419
| [fastapi_integration.py](examples/fastapi_integration.py) | FastAPI middleware, dependency injection, per-tenant budgets |
407420
| [langchain_integration.py](examples/langchain_integration.py) | LangChain callback handler for budget-aware agents |
408421

examples/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ pip install runcycles
2828
| [async_usage.py](async_usage.py) | Async client and async decorator ||
2929
| [openai_integration.py](openai_integration.py) | Guard OpenAI chat completions with budget checks | `openai` |
3030
| [anthropic_integration.py](anthropic_integration.py) | Guard Anthropic messages with per-tool budget tracking | `anthropic` |
31-
| [streaming_usage.py](streaming_usage.py) | Budget-managed streaming with token accumulation | `openai` |
31+
| [streaming_usage.py](streaming_usage.py) | `stream_reservation()` context manager with auto-commit | `openai` |
3232
| [fastapi_integration.py](fastapi_integration.py) | FastAPI middleware, dependency injection, per-tenant budgets | `fastapi`, `uvicorn` |
3333
| [langchain_integration.py](langchain_integration.py) | LangChain callback handler for budget-aware agents | `langchain`, `langchain-openai` |
3434

tests/test_streaming.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -754,10 +754,11 @@ async def test_cost_fn_used(self) -> None:
754754
)
755755

756756
async with asr as reservation:
757-
reservation.usage.tokens_input = 200
757+
reservation.usage.tokens_input = 300
758758

759759
commit_body = mock.commit_reservation.call_args[0][1]
760-
assert commit_body["actual"]["amount"] == 1000
760+
# 300 * 5 = 1500, distinct from estimate (1000)
761+
assert commit_body["actual"]["amount"] == 1500
761762

762763
@pytest.mark.asyncio
763764
async def test_context_set_and_cleared(self) -> None:

0 commit comments

Comments
 (0)