chore(deps): Bump github.com/townsendmerino/aikit from 1.9.0 to 1.11.0

[dependabot]

Bumps github.com/townsendmerino/aikit from 1.9.0 to 1.11.0.

Release notes

Sourced from github.com/townsendmerino/aikit's releases.

v1.11.0 — the embedder-coverage release

aikit v1.11.0 — the embedder-coverage release

1.10.x was about hardening what already shipped. 1.11.0 is about reach. It takes aikit's set of embedding models certified against their HuggingFace references from two to eight — and, just as importantly, makes "which models are supported" a fact the code proves rather than a line in a README.

The unlock underneath the model count is a pure-Go SentencePiece/Unigram tokenizer (no cgo, no sentencepiece dependency), which opens the entire XLM-RoBERTa multilingual family, plus a mixture-of-experts FFN for the one MoE embedder. Everything is certified the same way: cosine 1.000000 against the reference, a hidden-state gate, and a break-it-first check that must go red when you perturb the pooling, the position offset, or the expert routing.

This is a minor release — no breaking changes. New API and new capability only; every behavior change widens what loads (things that used to fail now succeed).

---

Certified embedders (2 → 8)

Each row is gated by a parity test at cosine ≥ 0.9999 (all currently at 1.000000):

Model	Architecture	Pooling	Tokenizer	Dims	Truncatable
all-MiniLM-L6-v2	BERT	mean	WordPiece	384	—
CodeRankEmbed	nomic-bert (RoPE, SwiGLU)	cls	WordPiece	768	—
bge-small-en-v1.5	BERT	cls	WordPiece	384	—
nomic-embed-text-v1.5	nomic-bert (RoPE, SwiGLU)	mean	WordPiece	768	768→64
xlm-roberta-base	XLM-R	— (bare LM)	Unigram	768	—
multilingual-e5-base	XLM-R	mean	Unigram	768	—
bge-m3	XLM-R	cls	Unigram	1024	—
nomic-embed-text-v2-moe	nomic-bert + MoE (top-2/8)	mean	Unigram	768	768→256

Bold rows are new in this release. The generated docs/embedder-coverage.md is the source of truth.

---

Highlights

Pure-Go SentencePiece/Unigram tokenizer. A Viterbi-decoded Unigram model plus its normalizer and pre-tokenizer, all in Go — the piece that makes the multilingual family work end-to-end. No cgo boundary, no external tokenizer library.

Mixture-of-experts. Top-2-of-8 routing on alternating layers, certifying nomic-embed-text-v2-moe — the first MoE encoder in aikit.

encoder.MatryoshkaFloor(model) (min int, ok bool) — the one capability a serve layer must not guess. Truncating a Matryoshka-trained embedding to a shorter width is fine; truncating any other model returns a unit-length, entirely plausible vector that simply retrieves worse — a silent failure. This exports the per-model floor so a dimensions request can be refused instead of silently degraded. Only two of the eight certified models qualify, and the claim is measured, not asserted: at a quarter width, multilingual-e5-base drops paraphrase-pair recall 1.00 → 0.80 while genuine MRL models hold their floor.

Coverage you can't fake. docs/embedder-coverage.md is generated from a registry whose pooling/dimension claims are read back from the real checkpoints, behind a freshness gate — so the published table can't drift from the code, and a model can't be listed without a passing gate behind it.

Declared pooling, explicit loader variants. Pooling (CLS vs mean) is now read from 1_Pooling/config.json rather than assumed — the difference is silent and total when wrong. BERT loaders gained the XLM-R position-id offset (pad+1) and optional token_type embeddings.

---

Upgrading

go get github.com/townsendmerino/aikit@v1.11.0

Drop-in. The changed behaviors only accept more:

... (truncated)

Changelog

Sourced from github.com/townsendmerino/aikit's changelog.

[1.11.0] — 2026-07-22

The embedder-coverage release. 1.10.x hardened what was already here; this adds capability: a pure-Go SentencePiece/Unigram tokenizer, a mixture-of-experts FFN, declared pooling and explicit BERT loader variants — and on top of those, six newly certified embedders, taking the certified set from two to eight. Coverage is now a generated table checked against the real checkpoints rather than a claim, and the one property a serve layer cannot safely guess — whether a model's vectors may be truncated — is exported rather than documented.

Minor, not patch: new exported API and new capabilities, no breaking changes. The behavior changes below all widen acceptance (things that used to fail now succeed), so they are additive from a caller's perspective.

Added

• Pure-Go SentencePiece/Unigram tokenizer (embed). The Viterbi-decoded Unigram path plus its normalizer/pre-tokenizer, which is what unlocks the XLM-RoBERTa family end-to-end (embed/tokenize_unigram.go). No cgo, no sentencepiece dependency.
• Mixture-of-experts FFN (encoder). Top-2-of-8 routing on alternating layers, certifying nomic-ai/nomic-embed-text-v2-moe.
• encoder.MatryoshkaFloor(model) (min int, ok bool). The smallest width a model's embeddings may be truncated to, ok=false when it was not trained with Matryoshka Representation Learning — and for unknown models, deliberately. Slicing a non-MRL embedding returns a unit-length, entirely plausible vector that simply retrieves worse; TestEmbedderCoverage_matryoshka measures it (multilingual-e5-base 1.00 → 0.80 pair recall at a quarter width, while genuine MRL models hold their floor). Only two of the eight certified models qualify. Existed as documentation and a test-only registry before; nothing downstream could consult it.
• Six newly certified embedders, each with a hidden-state gate and a break-it-first check: BAAI/bge-small-en-v1.5, nomic-ai/nomic-embed-text-v1.5, FacebookAI/xlm-roberta-base (forward-only; bare LM), intfloat/multilingual-e5-base, BAAI/bge-m3, nomic-ai/nomic-embed-text-v2-moe.
• Generated docs/embedder-coverage.md. Built from the registry in encoder/coverage_test.go, whose pooling/dimension claims are read back from the real checkpoints when present, with a freshness gate so the table cannot drift from the code.
• Declared pooling as a per-model property (BERT and Nomic loaders), read from 1_Pooling/config.json instead of assumed — the difference between CLS and mean pooling is silent and total.
• Explicit BERT loader variants: position-id offset (XLM-R's pad+1) and optional token_type embeddings.

Changed

Each of these accepts input that previously failed; none removes or narrows behavior:

... (truncated)

Commits

• cfda968 fix(encoder): make the coverage freshness gate line-ending agnostic (Windows CI)
• b59e58c docs: set v1.11.0 CHANGELOG date to the release day
• 7fc33df docs: add why-short-variable-names — the naming philosophy behind the kit
• 3769b99 vision,linalg: finish the WeightMat unification — migrate the last consumer
• 7d5378a docs: add CHANGELOG section for v1.11.0
• d471b13 feat(encoder): export the Matryoshka floor so serve layers can refuse bad tru...
• 44b679f docs,encoder: turn the Matryoshka caveat into a measured column; close the ca...
• bd5b39b docs: generated embedder-coverage table, defended against rot three ways
• 70467a3 encoder: mixture-of-experts FFN — certify nomic-embed-text-v2-moe (Bucket C)
• 0820fb3 docs: record Phase 3 (Bucket B) done in goinfer; C's precondition now met
• Additional commits viewable in compare view