Skip to content

fix: always populate mot.usage in HuggingFace backend #697

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jakelorocco merged 1 commit into from
Mar 18, 2026
+2 −10
Merged

fix: always populate mot.usage in HuggingFace backend #697

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 2 additions & 10 deletions mellea/backends/huggingface.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1133,26 +1133,18 @@ class used during generation, if any.
)

span = mot._meta.get("_telemetry_span")
from ..telemetry.metrics import is_metrics_enabled

metrics_enabled = is_metrics_enabled()

# Extract token counts only if needed
# Derive token counts from the output sequences (HF models have no usage object).
hf_output = mot._meta.get("hf_output")
n_prompt, n_completion = None, None
if (span is not None or metrics_enabled) and isinstance(
hf_output, GenerateDecoderOnlyOutput
):
# HuggingFace local models don't provide usage objects, but we can
# calculate token counts from sequences
if isinstance(hf_output, GenerateDecoderOnlyOutput):
try:
if input_ids is not None and hf_output.sequences is not None:
n_prompt = input_ids.shape[1]
n_completion = hf_output.sequences[0].shape[0] - n_prompt
except Exception:
pass

# Populate standardized usage field (convert to OpenAI format)
if n_prompt is not None and n_completion is not None:
mot.usage = {
"prompt_tokens": n_prompt,
Expand Down
Loading