fix(deps): update rust crate tiktoken-rs to 0.11.0

[renovate]

This PR contains the following updates:

Package	Type	Update	Change
tiktoken-rs	dependencies	minor	0.5.8 → 0.11.0

---

Release Notes

zurawiki/tiktoken-rs (tiktoken-rs)

v0.11.0

Compare Source

What's Changed

Features

• Add gpt-5.4 support, fix context sizes for o1-mini/chatgpt-4o/gpt-4.5 by @​zurawiki in #​156
• Add generic token type APIs (FromRank trait) by @​zurawiki in #​157
• Add gpt-5.2/5.3/codex model support by @​zurawiki in #​159
• Return Option from get_context_size, improve error messages, bump to 0.11.0 by @​zurawiki in #​160

Other Changes

• Use GitHub auto-generated release notes by @​zurawiki in #​158

Full Changelog: zurawiki/tiktoken-rs@v0.10.0...v0.11.0

v0.10.0

Compare Source

What's Changed

Features

• Use singleton BPE in num_tokens_from_messages and get_completion_max_tokens by @​zurawiki in #​140
• Count function_call tokens in num_tokens_from_messages by @​zurawiki in #​145
• Change decode() to accept &[Rank] and make decode_bytes() public by @​zurawiki in #​146
• Return &'static CoreBPE from get_bpe_from_model and get_bpe_from_tokenizer by @​zurawiki in #​147
• Rename misleading API functions, deprecate old names by @​zurawiki in #​154
• Add count-only token counting APIs by @​zurawiki in #​153

Bug Fixes

• Fix should-publish version check failing in CI by @​zurawiki in #​143
• Fix gpt-3.5 model version check in num_tokens_from_messages by @​zurawiki in #​144

Dependencies

• chore(deps): update extractions/setup-just action to v3 by @​renovate[bot] in #​127
• chore(deps): update actions/checkout action to v5 by @​renovate[bot] in #​126
• chore(deps): pin swatinem/rust-cache action to 98c8021 by @​renovate[bot] in #​125
• chore(deps): update actions/checkout v6, pin rust-toolchain, update create-gh-release v1.10.0 by @​zurawiki in #​142
• chore(deps): update extractions/setup-just action to v4 by @​renovate[bot] in #​150
• chore(deps): update swatinem/rust-cache action to v2.9.1 - autoclosed by @​renovate[bot] in #​149
• Sync regex patterns with upstream tiktoken, bump deps, port tests by @​zurawiki in #​151
• Bump async-openai to 0.34, add tool_calls/refusal token counting, update docs by @​zurawiki in #​152

Other Changes

• Fix release CI and harden workflow permissions by @​zurawiki in #​141
• Add release.yml for auto-generated release notes by @​zurawiki in #​148
• Release v0.10.0 by @​zurawiki in #​155

Full Changelog: zurawiki/tiktoken-rs@v0.9.1...v0.10.0

v0.9.1

What's Changed

• Support o-series models, gpt-5, and add o200k_harmony tokenizer for gpt-oss in #​123

Full Changelog: zurawiki/tiktoken-rs@v0.7.0...v0.9.1

v0.7.0

Compare Source

What's Changed

• Remove need for Arc<Mutex<>> on singletons by @​benbrandt in #​107
• Update to latest tiktoken + new model support by @​benbrandt in #​106
• Dependency and build/CI updates

New Contributors

• @​benbrandt made their first contribution in #​107

Full Changelog: zurawiki/tiktoken-rs@v0.6.0...v0.7.0

v0.6.0

Compare Source

Minor version release signifies a breaking change in 0.x

What's Changed

• Add support for chatgpt-4o-latest by @​Congyuwang in #​85
• Refactor internals to make future updates to tiktoken easier to merge
• Do not expose tiktoken internal modules and functions
• Update dependencies

New Contributors

• @​Congyuwang made their first contribution in #​85

Full Changelog: zurawiki/tiktoken-rs@v0.5.9...v0.6.0

---

Configuration

📅 Schedule: (in timezone America/Los_Angeles)

• Branch creation
  • "after 1am every 3 weeks on Saturday"
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.