Skip to content

Port sort key code from ICU4C (#2689) #6537

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 44 commits into from
Jun 10, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
e1a4140
Fix minor misspelling in documentation
rs-sac May 5, 2025
7190c34
Add new normalizer constructor that accepts borrowed data
rs-sac May 9, 2025
431424f
Factor out some common code for reuse by code to output sort keys
rs-sac May 5, 2025
bb47c1e
Port sort key code from ICU4C
rs-sac May 9, 2025
e34d3fe
Unit tests for sort keys
rs-sac May 5, 2025
9166cdc
Fix perf regression by inlining/macroizing
rs-sac May 14, 2025
f546757
Hide docs of DecomposingNormalizerBorrowed::new_with_data constructor
rs-sac May 14, 2025
983a27e
Remove terminator byte from final sort key
rs-sac May 14, 2025
7d423a0
Remove need_to_write callback
rs-sac May 14, 2025
c643d42
Demote asserts to debug asserts
rs-sac May 14, 2025
fc5db22
Fix incorrect type demotion
rs-sac May 15, 2025
3cefb9a
Fix typo in case level handling
rs-sac May 15, 2025
5f490fa
Pull trait's helpers into a private trait to prevent overriding
rs-sac May 15, 2025
a848433
Add SinkAdapter to eliminate copy of identical level bytes
rs-sac May 15, 2025
2fe8bab
Add write_sort_key_utf{8,16} variants and tests thereof
rs-sac May 15, 2025
41f70b2
Make write_sort_key_up_to_quaternary private
rs-sac May 15, 2025
6c013d5
More documentation, lots of warnings, and a doctest
rs-sac May 15, 2025
11ceef9
Remove large redundant copyright notice
rs-sac May 15, 2025
6d1231d
Rename Write trait to CollationKeySink
rs-sac May 15, 2025
e7e710f
Make CollationKeySink trait fallible
rs-sac May 15, 2025
8d763d2
Fix minor clippy complaints
rs-sac May 15, 2025
6643997
Resync diplomat allow list
rs-sac May 15, 2025
b83cece
Remove unnecessary iterators and factor sort key entry points
rs-sac May 16, 2025
f56a3e9
Remove write_to_zero
rs-sac May 16, 2025
f77fa88
Transcode UTF-16 to an on-stack buffer and optimize single chars
rs-sac May 16, 2025
2be0b21
Move write_byte back to main sink trait
rs-sac May 19, 2025
86c2fc7
Hoist UTF-16 transcoding array out of loop, delete duplicated encode
rs-sac May 19, 2025
4894ed9
Query compressible bytes data
rs-sac Jun 2, 2025
af7e4af
Move extern crate to crate root
rs-sac Jun 3, 2025
6498798
Remove SortKeyLevel's AsRef and AsMut
rs-sac Jun 3, 2025
fa322d1
Remove one mutable variable
rs-sac Jun 3, 2025
c45833a
Remove diplomat exclusion for doc(hidden) constructor
rs-sac Jun 3, 2025
51cff74
Fix another diplomat error from adjusting the config
rs-sac Jun 4, 2025
ce52c9f
Remove unnecessary type hints
rs-sac Jun 5, 2025
fdf47ed
Rename type parameter of SinkAdapter
rs-sac Jun 5, 2025
eaec863
Rename pub functions, adding a final "_to"
rs-sac Jun 5, 2025
03d8156
Switch from core::fmt::Error to core::convert::Infallible for error type
rs-sac Jun 5, 2025
3808148
impl CollationKeySink for [u8]
rs-sac Jun 5, 2025
1b78d07
impl CollationKeySink for VecDeque<u8>
rs-sac Jun 5, 2025
ffc70f0
Update diplomat
rs-sac Jun 5, 2025
ad1750c
Fix formatting
rs-sac Jun 6, 2025
1edb9ec
Twiddle diplomat config again
rs-sac Jun 6, 2025
b91dfc3
Update doc tests; compiler is lacking in criticality
rs-sac Jun 6, 2025
ef16c79
Store sink error from normalizer in adapter, remove CollationSink::error
rs-sac Jun 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion components/collator/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,14 @@ all-features = true
[dependencies]
displaydoc = { workspace = true }
icu_collections = { workspace = true }
icu_normalizer = { workspace = true }
icu_normalizer = { workspace = true, features = ["utf8_iter", "utf16_iter"] }
icu_locale_core = { workspace = true, features = ["alloc"] }
icu_properties = { workspace = true }
icu_provider = { workspace = true }
utf8_iter = { workspace = true }
utf16_iter = { workspace = true }
smallvec = { workspace = true, features = ["union", "const_generics", "const_new"] } # alloc
write16 = { workspace = true }
zerovec = { workspace = true }

databake = { workspace = true, optional = true, features = ["derive"] }
Expand Down
Loading