Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions .claude/skills/release/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ silently do the other thing.
[ ] 8 dart pub publish --dry-run → 0 warnings, every package
[ ] 12a website + README version pins bumped to the just-published versions
[ ] 12b new/changed public API + behavior documented (README + website) ← SAME PR
[ ] 12d skills/: `skills_review.sh <last-tag>` run, every flagged skill READ,
updated where the prose drifted, and `dart tool/check_skills.dart` green
[ ] 12c after merge: firebase-hosting-merge run == success (not just triggered)
```

Expand Down Expand Up @@ -615,6 +617,75 @@ Update each `^X.Y.Z` for the core packages (`flutter_gemma`, `flutter_gemma_lite
- **Breaking changes / migrations** → `migration.md`.
- **A bug class users hit** → `troubleshooting.md` (e.g. the #318 `maxTokens` vs `maxOutputTokens` confusion belongs here).

### 12d. Update the shipped agent skills — they are read by a MACHINE

`packages/flutter_gemma/skills/` holds eight `SKILL.md` files that ship inside
the core archive and are installed into users' coding agents by
`dart run skills@ get`. They are not a nice-to-have copy of the docs: an agent
follows them literally when writing code against this package.

That makes stale skills worse than stale docs. A human reading an outdated
README notices the mismatch; an agent does not — it writes confident, wrong code
against an API that moved, and the user blames the package.

**If this release changed public API or behaviour, the skills change with it.**
Map the change to the skill that covers it:

| Area | Skill |
|------|-------|
| registry, install, `ModelFileType`, `maxTokens`, sessions, chat, the `.litertlm` engine, backends, platform setup | `flutter-gemma-inference` (+ `references/platform-setup.md`) |
| function calling | `flutter-gemma-function-calling` |
| `.task`/`.bin`, MediaPipe web | `flutter-gemma-mediapipe` |
| ONNX / ORT-GenAI | `flutter-gemma-onnx` |
| the OS built-in model | `flutter-gemma-builtin-ai` |
| STT, TTS, `VoiceSession` | `flutter-gemma-speech` |
| embeddings, vector stores | `flutter-gemma-rag` |

**Do not go looking by hand.** Ask the diff which skills it puts in doubt:

```bash
bash tool/skills_review.sh <last-tag> # e.g. v1.8.0
```

For each skill it prints the symbols that skill NAMES and this release TOUCHED.
Run against the STT release it names `flutter-gemma-speech` with
`getActiveStt`, `language`, `SttModelType.whisper`; against the
`createChat`-tools fix it names the function-calling skill and leaves speech
alone.
That is the routing — a skill with hits gets opened, a skill without one gets
skipped with a clear conscience.

**Then open every flagged skill and read it against the change.** This is the
step, not the script. The script cannot tell whether the prose is still true;
it only says where to look.

Finally the mechanical gate:

```bash
dart tool/check_skills.dart # exit 0 required
```

It COMPILES the skills: every ```dart fence becomes a function body, every
inline `Type` and `Type.member` in the prose becomes a declaration, and
`dart analyze` runs over the result inside the example app, which depends on
every package. A misspelt parameter, a method that moved, a switch that is no
longer exhaustive — all fail. Read the count it prints, not just the exit code:
a run that extracted nothing exits 2 rather than reporting a pass.

It replaced a grep-based check that was green on four APIs that did not exist —
`gemma3` matched a model URL, `limit:` an unrelated argument. A text search
cannot tell "this name exists" from "this code is right".

**Why both.** `check_skills.dart` answers "does this code still compile" —
renames, deletions, signature changes. It stays green when a symbol survives and its MEANING moves,
which is the failure that actually happened here: `getActiveStt(language:)` went
from "the language this recognizer was built with" to "the default for its
transcriptions" with no rename anywhere. `skills_review.sh` is what puts that
change in front of your eyes; only reading closes it.

Skills live only in `flutter_gemma`, so a fix to any of them is one publish of
core. That is why they are all there rather than in the packages they describe.

### 12c. Deploy — it's automatic on merge to main

**You do NOT run a manual deploy.** `.github/workflows/firebase-hosting-merge.yml` auto-deploys to Firebase Hosting (`aichat-c0c27`, target `fluttergemma`, https://fluttergemma.dev → live channel) on every push to `main` that touches `website/**` or `packages/flutter_gemma/example/**`. So:
Expand Down
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ Core has NO pigeon (dropped at the 1.0 cut; its value types are hand-written in
- **LiteRT-LM**: native libs from `native-v0.16.0` GitHub Release (LiteRT-LM pin `924e79c9`, LiteRT pin `0ff28117`). Android tarball bundles the Qualcomm QNN dispatch stack and Windows tarball bundles Intel NPU dispatch (`LiteRtDispatch.dll` + OpenVino runtime + TBB) for `PreferredBackend.npu` (Qualcomm Snapdragon / Intel LunarLake/PantherLake) — both dispatch libs are **rebuilt from the pin every release**; carrying them forward is what silently broke NPU on both platforms (see the `build-native` skill). v0.16.0: fixes the Android OpenCL per-turn memory leak (LiteRT-LM #2699, #348/#402); v0.15.0 **broke the stream-callback ABI** (4-arg → 2-arg chunk object) with no compat path, handled by a runtime probe in `stream_proxy.c`. Windows discrete GPU works again — the crash was our own dead `litert_link_capi_so` Bazel define, not an upstream regression (#2957 retracted).
- **sqlite-vec**: `flutter_gemma_rag_sqlite` fetches the per-platform `vec0` loadable from the `native-sqlite-vec-v<X>` GitHub Release (`sqlite-vec-<target>.tar.gz` + `checksums_sqlite_vec.txt`), SHA256-verified by its `hook/build.dart`. `<X>` names the **upstream sqlite-vec release** the bytes were built from; a letter suffix (`0.1.9-a`) is only for RE-releasing changed bytes under an already-published number. The loadables are NOT committed — `native/sqlite_vec/prebuilt/` is a maintainer override produced by `build_local.sh`, gitignored and `.pubignore`d.
- **large_file_handler**: `^0.5.0` (core dep; 0.5.0 declares all 6 platforms — needed for pana platform support + the dart2wasm-clean web graph)
- **Current Version**: core `flutter_gemma` `1.8.1`, `flutter_gemma_rag_sqlite` `1.3.2`, `flutter_gemma_rag_qdrant` `1.3.1`; `flutter_gemma_litertlm` `1.6.3`, `flutter_gemma_mediapipe` `1.0.5`, `flutter_gemma_embeddings` `2.1.1`, `flutter_gemma_speech` `0.5.0`; `flutter_gemma_agent` `0.2.5`, `flutter_gemma_builtin_ai` `0.2.1`, `flutter_gemma_onnx` `0.3.3`; `genkit_flutter_gemma` `0.6.1`, `genkit_hybrid` `0.2.1`
- **Current Version**: core `flutter_gemma` `1.8.2`, `flutter_gemma_rag_sqlite` `1.3.2`, `flutter_gemma_rag_qdrant` `1.3.1`; `flutter_gemma_litertlm` `1.6.3`, `flutter_gemma_mediapipe` `1.0.5`, `flutter_gemma_embeddings` `2.1.1`, `flutter_gemma_speech` `0.5.0`; `flutter_gemma_agent` `0.2.5`, `flutter_gemma_builtin_ai` `0.2.1`, `flutter_gemma_onnx` `0.3.3`; `genkit_flutter_gemma` `0.6.1`, `genkit_hybrid` `0.2.1`
- **0.15.2**: embedding unified on LiteRT C API via Dart FFI on all native platforms (Android + iOS + Desktop). Drops `localagents-rag` JVM dep on Android and the separate TFLite C 0.12.7 tarball on Desktop; `TensorFlowLiteC` pod no longer needed on iOS. Single source of truth for `TaskType.prefix` in Dart, fixes cross-platform embedding drift (#264).

## Platform-Specific Setup
Expand Down Expand Up @@ -311,7 +311,7 @@ flutter analyze && dart format . && tool/test_all.sh
| `hook/build.dart` | Native Assets hook — fetches the per-platform `vec0` loadable extension |
| `web/rag/sqlite3.wasm` | custom `sqlite3.wasm` with `sqlite-vec`/`vec0` statically linked (app copies to its web root) |

**`packages/flutter_gemma_builtin_ai/` (OS built-in AI; Gemini Nano on Android, Apple Foundation Models on iOS/macOS; no web/desktop):**
**`packages/flutter_gemma_builtin_ai/` (OS built-in AI; Gemini Nano on Android and desktop Chrome via the Prompt API, Apple Foundation Models on iOS/macOS; no Windows/Linux):**

| File | Purpose |
|------|---------|
Expand All @@ -324,7 +324,7 @@ flutter analyze && dart format . && tool/test_all.sh
| `android/src/.../` | Android ML Kit GenAI (AICore) native layer; declares `minSdk 26` |
| `darwin/Classes/` (shared iOS+macOS source via `sharedDarwinSource: true`) | Apple Foundation Models native layer |

**`packages/flutter_gemma_onnx/` (ONNX Runtime — ORT-GenAI inference + plain-ORT embeddings; macOS arm64 only in v1, no web):**
**`packages/flutter_gemma_onnx/` (ONNX Runtime — ORT-GenAI inference + plain-ORT embeddings on macOS arm64 / Linux x64 / Windows x64 / Android arm64 / iOS arm64; web via Transformers.js + onnxruntime-web):**

| File | Purpose |
|------|---------|
Expand Down
17 changes: 16 additions & 1 deletion packages/flutter_gemma/.pubignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ web/*.js.map
!web/rag/
# And keep cache_api.js (hand-written, runtime needs window.cacheHas/Put/Get)
!web/cache_api.js
# And opfs_helper.js — defines window.flutterGemmaOPFS, which web streaming
# storage (WebStorageMode.streaming) binds to. Without this line it never
# reached pub, so the documented copy-from-the-package step had nothing to copy.
!web/opfs_helper.js

# Media files (for README, not needed in package)
docs/
Expand Down Expand Up @@ -133,4 +137,15 @@ example/web/test_vectorstore.html
# chromedriver binary — maintainer tool for running web integration tests
# (flutter drive -d chrome). ~16 MB, not part of the plugin. Without this
# it leaks into the pub package and bloats it (8 MB → archive).
chromedriver/
chromedriver/

# Agent skills MUST ship inside the archive — `dart run skills@ get` resolves
# the package on disk from package_config.json and reads `skills/` from there.
# Without this exception the blanket `**/*.md` above silently strips every
# SKILL.md and the skills reach nobody, with no error anywhere.
#
# It is LAST on purpose: the last matching rule wins, and rules such as
# `assets/`, `docs/` and `test/` would otherwise drop a skill subdirectory with
# those names — `assets/` is part of the Agent Skills layout.
!skills/
!skills/**
3 changes: 3 additions & 0 deletions packages/flutter_gemma/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
## 1.8.2
- Ship agent skills — `dart run skills@ get` teaches your AI assistant this package.

## 1.8.1
- Add `VectorStoreRepository.flush()`; custom implementations must declare it (#492).

Expand Down
35 changes: 27 additions & 8 deletions packages/flutter_gemma/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,17 +56,33 @@ There is an example of using:
- **🔐 Typed Download Errors:** Catch the public `DownloadException` sealed type (401/403/404/429/5xx) for gated HuggingFace models instead of substring-matching error strings
- **💾 Web Persistent Caching:** Models persist across browser restarts — Cache API for models <2GB, OPFS streaming for large ones (>2GB, e.g. Gemma 4 E4B) — no re-download on reload (Web only)

## What's new in 1.6.4
## Teach your AI assistant this package

- 📱 **iOS deployment floor lowered to 15.0** — core, built-in AI and embeddings build from iOS 15.0 (only `flutter_gemma_mediapipe` still needs 16.0). Every OS-26-only Foundation Models call is `#available`-guarded ([#441](https://github.com/DenisovAV/flutter_gemma/issues/441)).
`flutter_gemma` ships [agent skills](https://dart.dev/blog/skills-cli-1-0-bundle-and-distribute-ai-agent-skills-for-your-packages) — short instruction files your coding assistant reads so it uses this API correctly the first time:

## What's new in 1.6.3
```bash
dart run skills@ get --all
```

That scans your dependencies and installs every skill they bundle where your agent looks — Claude Code, Codex, Cursor, Antigravity, Cline, Copilot and OpenCode are supported. If it reports that it could not detect your agent, name it with `--agent claude` (or `codex`, `cursor`, …).

What they cover: registering an engine (core ships none), routing by the declared `ModelFileType` rather than the filename, and the two defaults that fail quietly — `maxTokens` is the context window and not the reply length, and `Message.isUser` defaults to `false`.

## What's new in 1.8.2

- 🤖 **Agent skills ship with the package** — `dart run skills@ get --all` installs seven skills that teach your coding assistant this API: inference (with platform setup), function calling, RAG, speech, MediaPipe, ONNX and built-in AI. Every code block in them is compiled against these packages before each release.

## What's new in 1.8.1

- 💾 **`VectorStoreRepository.flush()`** — a RAG index now survives the process; custom `VectorStoreRepository` implementations must declare it ([#492](https://github.com/DenisovAV/flutter_gemma/issues/492)).

## What's new in 1.8.0

- 📥 **flutter_gemma no longer claims `background_downloader`'s updates stream** — depending on this package used to make `FileDownloader().updates` unusable for your own downloads, because that stream takes a single subscription. Updates are now scoped to flutter_gemma's own task group ([#445](https://github.com/DenisovAV/flutter_gemma/issues/445)). Download priority is also corrected per platform.
- 🗣️ **Whisper output language per transcription** — `getActiveStt(language:)` sets the default and `transcribe(pcm, language:)` overrides it for one call, with no reload. **Breaking for custom `SpeechRecognizer` implementations**: `transcribe` gained `language:` and the type gained a `language` field ([#500](https://github.com/DenisovAV/flutter_gemma/issues/500)).

## What's new in 1.6.2
## What's new in 1.7.0

- 🌐 **ONNX on Web** — `flutter_gemma_onnx`'s `OnnxEngine` now generates text on Web via Transformers.js, with a fileless `ModelFileType.onnx` install (the model is a Hugging Face repo id, not a directory). `OnnxEmbeddingBackend` gained a web arm too, via onnxruntime-web. See [`flutter_gemma_onnx`](https://pub.dev/packages/flutter_gemma_onnx).
- 🤗 **One-call Hugging Face installs** — `fromHuggingFace(repo)` reads a repo's deployment manifest, picks the variant for the device and returns its tested runtime defaults; every engine carries its own resolver.

📖 Full docs & guides: **[fluttergemma.dev](https://fluttergemma.dev)**

Expand Down Expand Up @@ -1682,8 +1698,11 @@ await FlutterGemma.installEmbedder()
)
.install();

// 2. Initialize the vector store (one shard per database path)
await FlutterGemmaPlugin.instance.initializeVectorStore('rag_store');
// 2. Initialize the vector store (one shard per database path). On native pass
// an absolute path: a bare name resolves against the process working
// directory, which is not writable on Android or iOS. On web a name is enough.
final dir = await getApplicationDocumentsDirectory(); // package:path_provider
await FlutterGemmaPlugin.instance.initializeVectorStore('${dir.path}/rag_store');

// 3. Add documents — let the plugin compute embeddings for you
for (final doc in docs) {
Expand Down
2 changes: 1 addition & 1 deletion packages/flutter_gemma/example/pubspec.lock
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ packages:
path: ".."
relative: true
source: path
version: "1.8.1"
version: "1.8.2"
flutter_gemma_agent:
dependency: "direct main"
description:
Expand Down
2 changes: 1 addition & 1 deletion packages/flutter_gemma/ios/flutter_gemma.podspec
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
#
Pod::Spec.new do |s|
s.name = 'flutter_gemma'
s.version = '1.8.1'
s.version = '1.8.2'
s.summary = 'Flutter plugin for running Gemma and other LLMs locally on iOS.'
s.description = <<-DESC
Core runtime for running Gemma 4, Gemma3n, Gemma 3, FastVLM, Qwen3,
Expand Down
6 changes: 5 additions & 1 deletion packages/flutter_gemma/lib/core/api/flutter_gemma.dart
Original file line number Diff line number Diff line change
Expand Up @@ -1011,7 +1011,11 @@ class FlutterGemma {
/// throws a clear "add a RAG package" error.
///
/// ```dart
/// await FlutterGemma.rag.initialize('rag.db');
/// // Native: an absolute path in a writable directory. A bare name resolves
/// // against the process working directory, which is not writable on
/// // Android or iOS. Web: a bare name is fine.
/// final dir = await getApplicationDocumentsDirectory(); // path_provider
/// await FlutterGemma.rag.initialize('${dir.path}/rag.db');
/// await FlutterGemma.rag.addDocument(id: '1', content: 'hello');
/// final hits = await FlutterGemma.rag.searchSimilar(query: 'hi');
/// await FlutterGemma.rag.removeDocument(id: '1');
Expand Down
2 changes: 1 addition & 1 deletion packages/flutter_gemma/macos/flutter_gemma.podspec
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
#
Pod::Spec.new do |s|
s.name = 'flutter_gemma'
s.version = '1.8.1'
s.version = '1.8.2'
s.summary = 'Flutter Gemma - Run Gemma AI models locally on desktop'
s.description = <<-DESC
Flutter plugin for running Gemma AI models locally on macOS using LiteRT-LM.
Expand Down
2 changes: 1 addition & 1 deletion packages/flutter_gemma/pubspec.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: flutter_gemma
description: "Run Gemma and other LLMs on-device in Flutter (Android, iOS, Web, Desktop). Multimodal vision/audio, function calling, thinking mode, GPU, embeddings, RAG."
version: 1.8.1
version: 1.8.2
resolution: workspace
homepage: https://fluttergemma.dev
repository: https://github.com/DenisovAV/flutter_gemma
Expand Down
Loading
Loading