Skip to content

chore(deps)(deps): update tokenizers requirement from 0.15 to 0.22#246

Merged
madmax983 merged 1 commit intotrunkfrom
dependabot/cargo/tokenizers-0.22
Apr 15, 2026
Merged

chore(deps)(deps): update tokenizers requirement from 0.15 to 0.22#246
madmax983 merged 1 commit intotrunkfrom
dependabot/cargo/tokenizers-0.22

Conversation

@dependabot
Copy link
Copy Markdown
Contributor

@dependabot dependabot Bot commented on behalf of github Jan 6, 2026 •
edited
Loading

Updates the requirements on tokenizers to permit the latest version.

Release notes

Sourced from tokenizers's releases.

Release v0.22.2

What's Changed

Okay mostly doing the release for these PR:

Basically good typing with at least ty, and a lot fast (from 4 to 8x faster) loading vocab with a lot of added tokens and GIL free !?

New Contributors

Full Changelog: huggingface/tokenizers@v0.22.1...v0.22.2

Commits

You can trigger a rebase of this PR by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Note
Automatic rebases have been disabled on this pull request as it has been open for over 30 days.

@dependabot @github
Copy link
Copy Markdown
Contributor Author

dependabot Bot commented on behalf of github Jan 6, 2026

Labels

The following labels could not be found: auto-merge-candidate, dependencies. Please create them before Dependabot can add them to a pull request.

Please fix the above issues or remove invalid values from dependabot.yml.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 6, 2026

🚀 Benchmark Results

Current State Benchmarks

�[1m�[92m     Locking�[0m 349 packages to latest Rust 1.92.0 compatible versions
�[1m�[92m  Downloaded�[0m proptest v1.9.0
�[1m�[92m   Compiling�[0m proptest v1.9.0
                        time:   [35.067 ns 35.189 ns 35.377 ns]
                        time:   [35.046 ns 35.126 ns 35.245 ns]
                        time:   [34.971 ns 35.006 ns 35.064 ns]
3_hop_traversal         time:   [9.0276 µs 9.0306 µs 9.0340 µs]
node_lookup             time:   [39.353 ns 39.414 ns 39.485 ns]
edge_lookup             time:   [42.335 ns 42.360 ns 42.389 ns]
labeled_traversal       time:   [108.51 ns 108.60 ns 108.72 ns]
node_creation           time:   [605.18 ns 623.33 ns 642.05 ns]
edge_creation           time:   [1.3873 µs 1.3925 µs 1.3991 µs]
out_degree              time:   [6.5266 ns 6.5306 ns 6.5365 ns]
in_degree               time:   [6.5287 ns 6.5346 ns 6.5426 ns]
find_neighbors          time:   [537.44 ns 537.68 ns 537.94 ns]
                        time:   [286.10 µs 286.50 µs 286.95 µs]

GallifreyDB Benchmarks

                        time:   [302.33 µs 305.68 µs 309.18 µs]
                        time:   [286.44 µs 290.10 µs 293.72 µs]
                        time:   [40.783 ns 40.811 ns 40.838 ns]
                        time:   [41.554 ns 41.574 ns 41.594 ns]
                        time:   [40.866 ns 40.893 ns 40.921 ns]
                        time:   [32.015 ns 32.081 ns 32.175 ns]
                        time:   [30.988 ns 31.028 ns 31.081 ns]
                        time:   [30.965 ns 30.992 ns 31.029 ns]
                        time:   [175.27 ns 175.76 ns 176.46 ns]
                        time:   [44.488 ns 44.498 ns 44.510 ns]
gallifreydb_out_degree  time:   [5.9060 ns 5.9083 ns 5.9109 ns]
gallifreydb_in_degree   time:   [5.9054 ns 5.9071 ns 5.9090 ns]
                        time:   [57.359 ns 57.394 ns 57.430 ns]
                        time:   [3.2119 µs 3.2141 µs 3.2173 µs]
                        time:   [2.8107 ms 2.8405 ms 2.8714 ms]
                        time:   [27.816 ms 28.052 ms 28.293 ms]
                        time:   [283.42 ms 285.35 ms 287.47 ms]

Benchmarks run on ubuntu-latest

@dependabot dependabot Bot force-pushed the dependabot/cargo/tokenizers-0.22 branch 2 times, most recently from 586db68 to 9f9ac9e Compare January 22, 2026 22:19
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 22, 2026

🚀 Benchmark Results

Benchmarks have been run for this PR.

📊 Performance Summary (Top 10)

Benchmark Base (trunk) New (PR) Change % Std Dev
target_3_hop/traverse_three_hops N/A 112.25 ns New ± 1.77 ns
target_3_hop/traverse_three_hops N/A -0.02 ns New ± 0.00 ns
target_batch_insertion/insert_1000_edges N/A 254.51 µs New ± 1.72 µs
target_batch_insertion/insert_1000_edges N/A 0.14 ns New ± 0.00 ns
target_single_hop/traverse_one_hop N/A 18.29 ns New ± 0.38 ns
target_single_hop/traverse_one_hop N/A -0.12 ns New ± 0.00 ns
target_time_travel/at_anchor N/A 505.33 ns New ± 7.41 ns
target_time_travel/with_5_deltas N/A 298.29 ns New ± 2.26 ns
target_time_travel/worst_case_9_deltas N/A 493.36 ns New ± 7.81 ns

Full benchmark results available in workflow artifacts

📊 View detailed results

@dependabot dependabot Bot force-pushed the dependabot/cargo/tokenizers-0.22 branch 2 times, most recently from dcad2df to 8f6e5d0 Compare January 23, 2026 03:12
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 23, 2026

🚀 Benchmark Results

Benchmarks have been run for this PR.

📊 Performance Summary (Top 10)

Benchmark Base (trunk) New (PR) Change % Std Dev
target_3_hop/traverse_three_hops N/A 112.09 ns New ± 0.96 ns
target_3_hop/traverse_three_hops N/A -0.00 ns New ± 0.00 ns
target_batch_insertion/insert_1000_edges N/A 252.79 µs New ± 801.59 µs
target_batch_insertion/insert_1000_edges N/A -0.01 ns New ± 0.00 ns
target_single_hop/traverse_one_hop N/A 18.36 ns New ± 0.63 ns
target_single_hop/traverse_one_hop N/A 0.00 ns New ± 0.00 ns
target_time_travel/at_anchor N/A 487.15 ns New ± 4.43 ns
target_time_travel/at_anchor N/A -0.04 ns New ± 0.00 ns
target_time_travel/with_5_deltas N/A 299.42 ns New ± 4.77 ns
target_time_travel/with_5_deltas N/A 0.00 ns New ± 0.00 ns

Full benchmark results available in workflow artifacts

📊 View detailed results

@dependabot dependabot Bot force-pushed the dependabot/cargo/tokenizers-0.22 branch 2 times, most recently from 4593a4a to a21eeb2 Compare January 26, 2026 00:01
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 26, 2026

🚀 Benchmark Results

Benchmarks have been run for this PR.

📊 Performance Summary (Top 10)

Benchmark Base (trunk) New (PR) Change % Std Dev
target_3_hop/traverse_three_hops N/A 116.14 ns New ± 0.77 ns
target_3_hop/traverse_three_hops N/A 0.03 ns New ± 0.00 ns
target_batch_insertion/insert_1000_edges N/A 264.06 µs New ± 2.94 µs
target_batch_insertion/insert_1000_edges N/A 0.04 ns New ± 0.00 ns
target_single_hop/traverse_one_hop N/A 18.78 ns New ± 0.11 ns
target_single_hop/traverse_one_hop N/A 0.03 ns New ± 0.00 ns
target_time_travel/at_anchor N/A 517.46 ns New ± 4.53 ns
target_time_travel/at_anchor N/A 0.02 ns New ± 0.00 ns
target_time_travel/with_5_deltas N/A 340.49 ns New ± 6.23 ns
target_time_travel/with_5_deltas N/A 0.14 ns New ± 0.00 ns

Full benchmark results available in workflow artifacts

📊 View detailed results

@dependabot
chore(deps)(deps): update tokenizers requirement from 0.15 to 0.22
e2e8f3d 
Updates the requirements on [tokenizers](https://github.com/huggingface/tokenizers) to permit the latest version.
- [Release notes](https://github.com/huggingface/tokenizers/releases)
- [Changelog](https://github.com/huggingface/tokenizers/blob/main/RELEASE.md)
- [Commits](huggingface/tokenizers@v0.15.0...v0.22.2)

---
updated-dependencies:
- dependency-name: tokenizers
  dependency-version: 0.22.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot Bot force-pushed the dependabot/cargo/tokenizers-0.22 branch from a21eeb2 to e2e8f3d Compare January 28, 2026 18:51
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 28, 2026

🚀 Benchmark Results

Benchmarks have been run for this PR.

📊 Performance Summary (Top 10)

Benchmark Base (trunk) New (PR) Change % Std Dev
target_3_hop/traverse_three_hops N/A 169.77 ns New ± 3.88 ns
target_3_hop/traverse_three_hops N/A 0.51 ns New ± 0.00 ns
target_batch_insertion/insert_1000_edges N/A 357.37 µs New ± 578.96 µs
target_batch_insertion/insert_1000_edges N/A 0.40 ns New ± 0.00 ns
target_single_hop/traverse_one_hop N/A 18.40 ns New ± 0.09 ns
target_single_hop/traverse_one_hop N/A 0.01 ns New ± 0.00 ns
target_time_travel/at_anchor N/A 200.73 ns New ± 1.08 ns
target_time_travel/at_anchor N/A -0.60 ns New ± 0.00 ns
target_time_travel/with_5_deltas N/A 193.69 ns New ± 1.07 ns
target_time_travel/with_5_deltas N/A -0.35 ns New ± 0.00 ns

Full benchmark results available in workflow artifacts

📊 View detailed results

@madmax983 madmax983 merged commit 5c040b6 into trunk Apr 15, 2026
15 of 16 checks passed
@madmax983 madmax983 deleted the dependabot/cargo/tokenizers-0.22 branch April 15, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@madmax983