Skip to content

refactor: replace uniseg with uax29 and displaywidth - #237

Open
badger3478 wants to merge 1 commit into
charmbracelet:mainfrom
badger3478:replace-uniseg
Open

badger3478 wants to merge 1 commit into
charmbracelet:mainfrom
badger3478:replace-uniseg

Conversation

@badger3478

Copy link
Copy Markdown

What

Replaces ultraviolet's remaining github.com/rivo/uniseg uses with packages it already depends on:

  • decoder.go: uniseg.FirstGraphemeCluster(b, -1) becomes graphemes.FromBytes(b).First().
  • terminal_reader.go: the release grapheme is iterated with graphemes.FromString and Value(), in place of uniseg.NewGraphemes and Str().
  • buffer_test.go and examples/advanced/layout measure with displaywidth.

github.com/rivo/uniseg is no longer a requirement in go.mod.

Why

charmbracelet/x#606 moved ansi from uniseg to displaywidth, and charmbracelet/lipgloss#592 followed for maxRuneWidth. This finishes the move in ultraviolet, so it segments and measures text the same way ansi and lipgloss do.

Behaviour

uniseg v0.4.7 (February 2024) implements Unicode 15.0.0; clipperhouse/uax29/v2 and clipperhouse/displaywidth implement Unicode 17. They give the same answers for ASCII, Latin with combining marks, CJK, Hangul, box drawing, braille, arrows and most emoji. Where they differ, the difference follows the newer rules:

  • Devanagari conjuncts are one grapheme cluster (Unicode 15.1, GB9c): क्ष was two clusters.
  • A keycap sequence such as 1️⃣ measures 2 cells, as terminals draw it, where uniseg measured 1.

Over 200,000 random strings mixing those scripts, emoji and zero-width joiners, about 5% segment differently and about 2.5% measure differently, all from the newer Unicode data.

Testing

  • go test ./... and go vet ./... give the same results as on main, and go mod tidy -diff is clean.
  • The examples module builds; it still lists uniseg indirectly, through lipgloss.
  • Built together with the same change in lipgloss and bubbles (a go.work over the three), a downstream Bubble Tea app no longer links github.com/rivo/uniseg, and its TUI tests pass.

This PR stands on its own. I'll follow with the same change in lipgloss and bubbles; uniseg leaves a Bubble Tea app's module graph once all three are released.

  • I have read CONTRIBUTING.md.
  • I have created a discussion that was approved by a maintainer (for new features). Not a feature: a refactor.

Finish the move off github.com/rivo/uniseg that charmbracelet/x#606
began: the decoder takes a key's first grapheme cluster from
uax29/graphemes, and the terminal reader iterates a release grapheme with
it; the buffer test and the layout example measure with displaywidth.
uax29 was already a direct dependency, and uniseg is no longer one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant