Skip to content

Migrate to new tekken crate - #1767

Open
jorge-menjivar wants to merge 1 commit into
dphnAI:mainfrom
jorge-menjivar:tekken-crate
Open

Migrate to new tekken crate#1767
jorge-menjivar wants to merge 1 commit into
dphnAI:mainfrom
jorge-menjivar:tekken-crate

Conversation

@jorge-menjivar

Copy link
Copy Markdown

I am the author of tekken-rs and recently renamed it to simply tekken to make it easier to import, find, and align with its library name. Since crates.io has no rename mechanism, this meant publishing a new crate; tekken-rs 0.1.2 is the final release under the old name.

Version 0.2.0 of tekken also adds:

  • Support for v15 of the tokenizer
  • Image tokenization support
  • Audio and image dependencies gated behind feature flags

Two things to note:

  • No code changes are needed: tekken-rs's library name was already tekken, so all existing use tekken::... imports will keep working
  • The existing default-features = false was a no-op on 0.1.1 (it had no feature flags, so the audio dependencies were unconditional), but on 0.2.0 it takes effect: since this project only uses tekken for text tokenization, hound, rubato, rustfft, and ndarray drop out of the dependency tree, and the new image feature is never pulled in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant