Repository navigation
fix(server): populate OpenAI-compatible usage fields - #355
mywonderland808 wants to merge 1 commit into
Conversation
Fill prompt_tokens/total_tokens on approach and passthrough responses, and emit stream usage when stream_options.include_usage is set, so OpenAI-compatible clients get a complete usage object instead of zeros or missing fields.
|
|
|
Hi! Before signing the CLA, I’d like to clarify one point regarding AI-assisted contributions. This contribution was developed using Cursor (AI Agent) based on my own planning, architecture decisions, requirements, and instructions. The agent generated most of the implementation, which I subsequently reviewed and tested. Cursor's terms grant me rights to the generated output. Does this type of AI-assisted, agent-generated contribution comply with your Contributor License Agreement, particularly the representation that I am the sole author of the Contributions? I'd appreciate your clarification before I sign the CLA. Thanks! |
Summary
prompt_tokens/total_tokenson approach-synthesized and passthrough chat completions (and batch / legacy/v1/completions), instead of omitting them or leavingprompt_tokens: 0.choices: []whenstream_options.include_usageis set, matching OpenAI / existing dict-stream behavior.completion_tokenswhen known; estimate only when missing or zero with non-empty content. Includes focused unit and HTTP wiring tests.Test plan
python -m unittest tests.test_usage_reporting -vpython -m unittest tests.test_completions_endpoint tests.test_proxy_plugin -vusage.prompt_tokensandusage.total_tokensstream_options.include_usage: trueends with achoices: []usage chunk before[DONE]noneresponse missing upstreamprompt_tokensgets filled without clobbering complete upstream usage