chore(perplexity): Added all keys for usage metadata

[keenborder786]

• Description: Missing keys for usage metadata: According to the Perplexity API documentation, the following output token details were missing in ChatPerplexity — citation_tokens, num_search_queries, and reasoning_token — which are required for accurate cost calculation, as noted in the issue. These have now been added. Although OutputTokenDetails has restricted keys, all of them have been included for now.
• Issue: Add Perplexity cost tracking #31647

[codspeed-hq]

CodSpeed Performance Report

Merging #33480 will not alter performance

_{Comparing keenborder786:feat/preplexity_cost_tracking (0ae58fc) with master (ef85161)}

Summary

✅ 1 untouched
⏩ 33 skipped¹

Footnotes

1. 33 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[ccurme]

the linter is complaining here because UsageMetadata does not support extra keys.

We should follow the existing types where possible: https://python.langchain.com/api_reference/core/messages/langchain_core.messages.ai.UsageMetadata.html

reasoning_tokens has a dedicated slot reasoning under output_token_details.

citation_tokens we can also stash in output_token_details.

num_search_queries IMO can go in response_metadata since it isn't a token count but I don't feel strongly about that.

[keenborder786]

@ccurme please check now

[ccurme]

Thanks for this, could we add a test?

[ccurme]

(nit) might make sense to only populate the keys if they are present in token_usage.

[keenborder786]

@ccurme done

[ccurme]

Thanks. Are you able to recover any reasoning or citation tokens? I'm not seeing those fields in the output even though we get citations and search results. Here is my attempt:

from langchain_perplexity import ChatPerplexity

model = ChatPerplexity(model="sonar-reasoning")

response = model.invoke("What's the weather in San Francisco now?")

assert response.additional_kwargs["citations"]
assert response.additional_kwargs["search_results"]
response.usage_metadata

# {'input_tokens': 9,
#  'output_tokens': 1261,
#  'total_tokens': 1270,
#  'output_token_details': {'reasoning': 0, 'citation_tokens': 0}}

langchain-perplexity uses the openai SDK, are both citation and reasoning tokens available that way?

[keenborder786]

SDK, are both citation and reasoning tokens available that way?

Strange, since I am able to get them using sonar-reasoning, please see the following docs: https://docs.perplexity.ai/api-reference/chat-completions-post#response-usage

[keenborder786]

@ccurme is it looking good now?