diff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.md index 8c8b6c43e..7abd8e9d3 100644 --- a/.claude/skills/release/SKILL.md +++ b/.claude/skills/release/SKILL.md @@ -34,6 +34,25 @@ silently do the other thing. 3. **Reproduce this Definition-of-Done in your reply and mark every item** (done / N/A + reason) before you publish. Do not publish off memory of the skill — walk it as a literal checklist against the actual repo state. +4. **The skills are fixed in the PR that changes the API, not at release time.** + `packages/flutter_gemma/skills/**` ships inside core and is read by other + people's coding agents, so a stale sentence there becomes confident, broken + code in someone else's app. On any PR touching `packages/*/lib/**`, a native + build file (`android/`, `ios/`, `darwin/`, `macos/`, `windows/`, `hook/`), a + pinned CDN version or a `pubspec.yaml` floor: + + ```bash + bash tool/skills_review.sh origin/main # which skills the diff puts in doubt + dart tool/check_skills.dart # compiles every block; exit 0 required + dart run skills_lint@0.5.1 # file-level rules; exit 0 required + ``` + + The two gates also run in CI (`skills` job), so a rename is caught without + you. What CI cannot catch is a symbol that survives while its MEANING moves — + `getActiveStt(language:)` went from "the language this recognizer was built + with" to "the default for its transcriptions" with no rename anywhere. That + is what reading the flagged skills is for. Step 12d is the release backstop, + not the first time this happens. ### Definition of Done (paste it; check 1a–12b before Step 10 publish; 12c is verified after merge) @@ -44,14 +63,20 @@ silently do the other thing. [ ] 5b manifest gate RUN and printed "N platform(s) compared" — N == number of tarballs [ ] 1e core public API changed? → upgrade-genkit (realign + version), else N/A [ ] 1f shared code duplicated across satellites patched everywhere (grep the pattern) -[ ] 1f-bis tool/check_macos_podfile_snippet.sh passes (5 copies of the macOS - post_install snippet identical) — RUN it, do not eyeball +[ ] 1f-bis tool/check_macos_podfile_snippet.sh passes (every copy of the macOS + post_install snippet byte-identical — 23 today: three example Podfiles, + the codelab step apps, README, desktop.md and the inference skill's + references/platform-setup.md) — RUN it, do not eyeball [ ] 1g each changed satellite's flutter_gemma: floor >= the core version it now needs [ ] 2 versions bumped: pubspec + podspec (if any) + CLAUDE.md Current-Version line [ ] 7 CHANGELOG: one short line per package, every published package [ ] 8 dart pub publish --dry-run → 0 warnings, every package [ ] 12a website + README version pins bumped to the just-published versions [ ] 12b new/changed public API + behavior documented (README + website) ← SAME PR +[ ] 12d skills/: `skills_review.sh ` run, every flagged skill READ, + updated where the prose drifted, `dart tool/check_skills.dart` green and + `dart run skills_lint@0.5.1` green — backstop: rule 4 means the PRs in + this release already did it [ ] 12c after merge: firebase-hosting-merge run == success (not just triggered) ``` @@ -615,6 +640,91 @@ Update each `^X.Y.Z` for the core packages (`flutter_gemma`, `flutter_gemma_lite - **Breaking changes / migrations** → `migration.md`. - **A bug class users hit** → `troubleshooting.md` (e.g. the #318 `maxTokens` vs `maxOutputTokens` confusion belongs here). +### 12d. Update the shipped agent skills — they are read by a MACHINE + +`packages/flutter_gemma/skills/` holds eight `SKILL.md` files that ship inside +the core archive and are installed into users' coding agents by +`dart run skills@ get --all`. They are not a nice-to-have copy of the docs: an agent +follows them literally when writing code against this package. + +That makes stale skills worse than stale docs. A human reading an outdated +README notices the mismatch; an agent does not — it writes confident, wrong code +against an API that moved, and the user blames the package. + +**If this release changed public API or behaviour, the skills change with it.** +Map the change to the skill that covers it: + +| Area | Skill | +|------|-------| +| registry, install, `ModelFileType`, `maxTokens`, sessions, chat, the `.litertlm` engine, backends, platform setup | `flutter-gemma-inference` (+ `references/platform-setup.md`) | +| function calling | `flutter-gemma-function-calling` | +| `.task`/`.bin`, MediaPipe web | `flutter-gemma-mediapipe` | +| ONNX / ORT-GenAI | `flutter-gemma-onnx` | +| the OS built-in model | `flutter-gemma-builtin-ai` | +| STT, TTS, `VoiceSession` | `flutter-gemma-speech` | +| embeddings, vector stores | `flutter-gemma-rag` | + +**Do not go looking by hand.** Ask the diff which skills it puts in doubt: + +```bash +bash tool/skills_review.sh # e.g. v1.8.0 +``` + +For each skill it prints the symbols that skill NAMES and this release TOUCHED. +Run against the STT release it names `flutter-gemma-speech` with +`getActiveStt`, `language`, `SttModelType.whisper`; against the +`createChat`-tools fix it names the function-calling skill and leaves speech +alone. +That is the routing — a skill with hits gets opened, a skill without one gets +skipped with a clear conscience. + +**Then open every flagged skill and read it against the change.** This is the +step, not the script. The script cannot tell whether the prose is still true; +it only says where to look. + +Finally the mechanical gate: + +```bash +dart tool/check_skills.dart # exit 0 required +``` + +It COMPILES the skills: every ```dart fence becomes a function body, every +inline `Type` and `Type.member` in the prose becomes a declaration, and +`dart analyze` runs over the result inside the example app, which depends on +every package. A misspelt parameter, a method that moved, a switch that is no +longer exhaustive — all fail. Read the count it prints, not just the exit code: +a run that extracted nothing exits 2 rather than reporting a pass. + +It replaced a grep-based check that was green on four APIs that did not exist — +`gemma3` matched a model URL, `limit:` an unrelated argument. A text search +cannot tell "this name exists" from "this code is right". + +And the file-level check, Google's linter for the Agent Skills format: + +```bash +dart run skills_lint@0.5.1 # exit 0 required; config in skills_lint.yaml +``` + +It checks what compilation cannot: frontmatter keys the spec allows (these +skills install into eight different agents, and the reference validator rejects +anything outside its allowlist), a `name` that matches its directory, the +1024-character description budget, and every relative link resolving — that last +one is off upstream by default and an error here, so a renamed +`references/platform-setup.md` fails instead of handing an agent a dead pointer. + +Both run in CI as the `skills` job (`.github/workflows/test.yml`), so a PR that +breaks either is red before it reaches this checklist. + +**Why both.** `check_skills.dart` answers "does this code still compile" — +renames, deletions, signature changes. It stays green when a symbol survives and its MEANING moves, +which is the failure that actually happened here: `getActiveStt(language:)` went +from "the language this recognizer was built with" to "the default for its +transcriptions" with no rename anywhere. `skills_review.sh` is what puts that +change in front of your eyes; only reading closes it. + +Skills live only in `flutter_gemma`, so a fix to any of them is one publish of +core. That is why they are all there rather than in the packages they describe. + ### 12c. Deploy — it's automatic on merge to main **You do NOT run a manual deploy.** `.github/workflows/firebase-hosting-merge.yml` auto-deploys to Firebase Hosting (`aichat-c0c27`, target `fluttergemma`, https://fluttergemma.dev → live channel) on every push to `main` that touches `website/**` or `packages/flutter_gemma/example/**`. So: diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 368bacb08..6ef346184 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -56,6 +56,41 @@ jobs: fail_ci_if_error: false continue-on-error: true + # The skills shipped inside flutter_gemma are read by other people's coding + # agents, so a wrong name there becomes confident, broken code in someone + # else's app. Two gates, both fail-closed, and neither ran in CI before: + # check_skills.dart compiles every ```dart fence and every inline type in + # the prose against the real packages — it analyses inside the example app + # because that is the package depending on all of them; + # skills_lint checks the files themselves (frontmatter, name, description + # budget, relative links), with the rules set in skills_lint.yaml. + skills: + runs-on: ubuntu-latest + timeout-minutes: 20 + + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Flutter + uses: subosito/flutter-action@v2 + with: + channel: 'stable' + cache: true + + - name: Install dependencies + run: flutter pub get + + - name: Resolve the example app + run: flutter pub get + working-directory: packages/flutter_gemma/example + + - name: Compile every code block in the skills + run: dart tool/check_skills.dart + + - name: Lint the skill files + run: dart run skills_lint@0.5.1 + build-example-android: runs-on: ubuntu-latest needs: analyze-and-test diff --git a/CLAUDE.md b/CLAUDE.md index 81e0d3771..59f0281b2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -150,7 +150,7 @@ Core has NO pigeon (dropped at the 1.0 cut; its value types are hand-written in - **LiteRT-LM**: native libs from `native-v0.16.0` GitHub Release (LiteRT-LM pin `924e79c9`, LiteRT pin `0ff28117`). Android tarball bundles the Qualcomm QNN dispatch stack and Windows tarball bundles Intel NPU dispatch (`LiteRtDispatch.dll` + OpenVino runtime + TBB) for `PreferredBackend.npu` (Qualcomm Snapdragon / Intel LunarLake/PantherLake) — both dispatch libs are **rebuilt from the pin every release**; carrying them forward is what silently broke NPU on both platforms (see the `build-native` skill). v0.16.0: fixes the Android OpenCL per-turn memory leak (LiteRT-LM #2699, #348/#402); v0.15.0 **broke the stream-callback ABI** (4-arg → 2-arg chunk object) with no compat path, handled by a runtime probe in `stream_proxy.c`. Windows discrete GPU works again — the crash was our own dead `litert_link_capi_so` Bazel define, not an upstream regression (#2957 retracted). - **sqlite-vec**: `flutter_gemma_rag_sqlite` fetches the per-platform `vec0` loadable from the `native-sqlite-vec-v` GitHub Release (`sqlite-vec-.tar.gz` + `checksums_sqlite_vec.txt`), SHA256-verified by its `hook/build.dart`. `` names the **upstream sqlite-vec release** the bytes were built from; a letter suffix (`0.1.9-a`) is only for RE-releasing changed bytes under an already-published number. The loadables are NOT committed — `native/sqlite_vec/prebuilt/` is a maintainer override produced by `build_local.sh`, gitignored and `.pubignore`d. - **large_file_handler**: `^0.5.0` (core dep; 0.5.0 declares all 6 platforms — needed for pana platform support + the dart2wasm-clean web graph) -- **Current Version**: core `flutter_gemma` `1.8.1`, `flutter_gemma_rag_sqlite` `1.3.2`, `flutter_gemma_rag_qdrant` `1.3.1`; `flutter_gemma_litertlm` `1.6.3`, `flutter_gemma_mediapipe` `1.0.5`, `flutter_gemma_embeddings` `2.1.1`, `flutter_gemma_speech` `0.5.0`; `flutter_gemma_agent` `0.2.5`, `flutter_gemma_builtin_ai` `0.2.1`, `flutter_gemma_onnx` `0.3.3`; `genkit_flutter_gemma` `0.6.1`, `genkit_hybrid` `0.2.1` +- **Current Version**: core `flutter_gemma` `1.8.2`, `flutter_gemma_rag_sqlite` `1.3.2`, `flutter_gemma_rag_qdrant` `1.3.1`; `flutter_gemma_litertlm` `1.6.3`, `flutter_gemma_mediapipe` `1.0.5`, `flutter_gemma_embeddings` `2.1.1`, `flutter_gemma_speech` `0.5.0`; `flutter_gemma_agent` `0.2.5`, `flutter_gemma_builtin_ai` `0.2.1`, `flutter_gemma_onnx` `0.3.3`; `genkit_flutter_gemma` `0.6.1`, `genkit_hybrid` `0.2.1` - **0.15.2**: embedding unified on LiteRT C API via Dart FFI on all native platforms (Android + iOS + Desktop). Drops `localagents-rag` JVM dep on Android and the separate TFLite C 0.12.7 tarball on Desktop; `TensorFlowLiteC` pod no longer needed on iOS. Single source of truth for `TaskType.prefix` in Dart, fixes cross-platform embedding drift (#264). ## Platform-Specific Setup @@ -311,7 +311,7 @@ flutter analyze && dart format . && tool/test_all.sh | `hook/build.dart` | Native Assets hook — fetches the per-platform `vec0` loadable extension | | `web/rag/sqlite3.wasm` | custom `sqlite3.wasm` with `sqlite-vec`/`vec0` statically linked (app copies to its web root) | -**`packages/flutter_gemma_builtin_ai/` (OS built-in AI; Gemini Nano on Android, Apple Foundation Models on iOS/macOS; no web/desktop):** +**`packages/flutter_gemma_builtin_ai/` (OS built-in AI; Gemini Nano on Android and desktop Chrome via the Prompt API, Apple Foundation Models on iOS/macOS; no Windows/Linux):** | File | Purpose | |------|---------| @@ -324,7 +324,7 @@ flutter analyze && dart format . && tool/test_all.sh | `android/src/.../` | Android ML Kit GenAI (AICore) native layer; declares `minSdk 26` | | `darwin/Classes/` (shared iOS+macOS source via `sharedDarwinSource: true`) | Apple Foundation Models native layer | -**`packages/flutter_gemma_onnx/` (ONNX Runtime — ORT-GenAI inference + plain-ORT embeddings; macOS arm64 only in v1, no web):** +**`packages/flutter_gemma_onnx/` (ONNX Runtime — ORT-GenAI inference + plain-ORT embeddings on macOS arm64 / Linux x64 / Windows x64 / Android arm64 / iOS arm64; web via Transformers.js + onnxruntime-web):** | File | Purpose | |------|---------| diff --git a/codelabs/getting-started-flutter-gemma/complete/android/app/build.gradle.kts b/codelabs/getting-started-flutter-gemma/complete/android/app/build.gradle.kts index 4a724f642..eca94ccec 100644 --- a/codelabs/getting-started-flutter-gemma/complete/android/app/build.gradle.kts +++ b/codelabs/getting-started-flutter-gemma/complete/android/app/build.gradle.kts @@ -19,7 +19,10 @@ android { applicationId = "dev.fluttergemma.quickstart" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/getting-started-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/getting-started-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift index 0a85e4cb2..0be86a9b4 100644 --- a/codelabs/getting-started-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/getting-started-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,11 +5,13 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) SharedPreferencesPlugin.register(with: registry.registrar(forPlugin: "SharedPreferencesPlugin")) diff --git a/codelabs/getting-started-flutter-gemma/complete/pubspec.yaml b/codelabs/getting-started-flutter-gemma/complete/pubspec.yaml index 324dd18bb..442185891 100644 --- a/codelabs/getting-started-flutter-gemma/complete/pubspec.yaml +++ b/codelabs/getting-started-flutter-gemma/complete/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 dev_dependencies: flutter_test: diff --git a/codelabs/getting-started-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc b/codelabs/getting-started-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/getting-started-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/getting-started-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/getting-started-flutter-gemma/complete/windows/flutter/generated_plugins.cmake b/codelabs/getting-started-flutter-gemma/complete/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/getting-started-flutter-gemma/complete/windows/flutter/generated_plugins.cmake +++ b/codelabs/getting-started-flutter-gemma/complete/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/getting-started-flutter-gemma/step_02_download/android/app/build.gradle.kts b/codelabs/getting-started-flutter-gemma/step_02_download/android/app/build.gradle.kts index 4a724f642..eca94ccec 100644 --- a/codelabs/getting-started-flutter-gemma/step_02_download/android/app/build.gradle.kts +++ b/codelabs/getting-started-flutter-gemma/step_02_download/android/app/build.gradle.kts @@ -19,7 +19,10 @@ android { applicationId = "dev.fluttergemma.quickstart" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/getting-started-flutter-gemma/step_02_download/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/getting-started-flutter-gemma/step_02_download/macos/Flutter/GeneratedPluginRegistrant.swift index 0a85e4cb2..0be86a9b4 100644 --- a/codelabs/getting-started-flutter-gemma/step_02_download/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/getting-started-flutter-gemma/step_02_download/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,11 +5,13 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) SharedPreferencesPlugin.register(with: registry.registrar(forPlugin: "SharedPreferencesPlugin")) diff --git a/codelabs/getting-started-flutter-gemma/step_02_download/pubspec.yaml b/codelabs/getting-started-flutter-gemma/step_02_download/pubspec.yaml index 6e7eae039..900145681 100644 --- a/codelabs/getting-started-flutter-gemma/step_02_download/pubspec.yaml +++ b/codelabs/getting-started-flutter-gemma/step_02_download/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 dev_dependencies: flutter_test: diff --git a/codelabs/getting-started-flutter-gemma/step_02_download/windows/flutter/generated_plugin_registrant.cc b/codelabs/getting-started-flutter-gemma/step_02_download/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/getting-started-flutter-gemma/step_02_download/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/getting-started-flutter-gemma/step_02_download/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/getting-started-flutter-gemma/step_02_download/windows/flutter/generated_plugins.cmake b/codelabs/getting-started-flutter-gemma/step_02_download/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/getting-started-flutter-gemma/step_02_download/windows/flutter/generated_plugins.cmake +++ b/codelabs/getting-started-flutter-gemma/step_02_download/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/getting-started-flutter-gemma/step_03_chat/android/app/build.gradle.kts b/codelabs/getting-started-flutter-gemma/step_03_chat/android/app/build.gradle.kts index 4a724f642..eca94ccec 100644 --- a/codelabs/getting-started-flutter-gemma/step_03_chat/android/app/build.gradle.kts +++ b/codelabs/getting-started-flutter-gemma/step_03_chat/android/app/build.gradle.kts @@ -19,7 +19,10 @@ android { applicationId = "dev.fluttergemma.quickstart" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/getting-started-flutter-gemma/step_03_chat/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/getting-started-flutter-gemma/step_03_chat/macos/Flutter/GeneratedPluginRegistrant.swift index 0a85e4cb2..0be86a9b4 100644 --- a/codelabs/getting-started-flutter-gemma/step_03_chat/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/getting-started-flutter-gemma/step_03_chat/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,11 +5,13 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) SharedPreferencesPlugin.register(with: registry.registrar(forPlugin: "SharedPreferencesPlugin")) diff --git a/codelabs/getting-started-flutter-gemma/step_03_chat/pubspec.yaml b/codelabs/getting-started-flutter-gemma/step_03_chat/pubspec.yaml index 6e7eae039..900145681 100644 --- a/codelabs/getting-started-flutter-gemma/step_03_chat/pubspec.yaml +++ b/codelabs/getting-started-flutter-gemma/step_03_chat/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 dev_dependencies: flutter_test: diff --git a/codelabs/getting-started-flutter-gemma/step_03_chat/windows/flutter/generated_plugin_registrant.cc b/codelabs/getting-started-flutter-gemma/step_03_chat/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/getting-started-flutter-gemma/step_03_chat/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/getting-started-flutter-gemma/step_03_chat/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/getting-started-flutter-gemma/step_03_chat/windows/flutter/generated_plugins.cmake b/codelabs/getting-started-flutter-gemma/step_03_chat/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/getting-started-flutter-gemma/step_03_chat/windows/flutter/generated_plugins.cmake +++ b/codelabs/getting-started-flutter-gemma/step_03_chat/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/getting-started-flutter-gemma/step_04_streaming/android/app/build.gradle.kts b/codelabs/getting-started-flutter-gemma/step_04_streaming/android/app/build.gradle.kts index 4a724f642..eca94ccec 100644 --- a/codelabs/getting-started-flutter-gemma/step_04_streaming/android/app/build.gradle.kts +++ b/codelabs/getting-started-flutter-gemma/step_04_streaming/android/app/build.gradle.kts @@ -19,7 +19,10 @@ android { applicationId = "dev.fluttergemma.quickstart" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/getting-started-flutter-gemma/step_04_streaming/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/getting-started-flutter-gemma/step_04_streaming/macos/Flutter/GeneratedPluginRegistrant.swift index 0a85e4cb2..0be86a9b4 100644 --- a/codelabs/getting-started-flutter-gemma/step_04_streaming/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/getting-started-flutter-gemma/step_04_streaming/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,11 +5,13 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) SharedPreferencesPlugin.register(with: registry.registrar(forPlugin: "SharedPreferencesPlugin")) diff --git a/codelabs/getting-started-flutter-gemma/step_04_streaming/pubspec.yaml b/codelabs/getting-started-flutter-gemma/step_04_streaming/pubspec.yaml index 6e7eae039..900145681 100644 --- a/codelabs/getting-started-flutter-gemma/step_04_streaming/pubspec.yaml +++ b/codelabs/getting-started-flutter-gemma/step_04_streaming/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 dev_dependencies: flutter_test: diff --git a/codelabs/getting-started-flutter-gemma/step_04_streaming/windows/flutter/generated_plugin_registrant.cc b/codelabs/getting-started-flutter-gemma/step_04_streaming/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/getting-started-flutter-gemma/step_04_streaming/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/getting-started-flutter-gemma/step_04_streaming/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/getting-started-flutter-gemma/step_04_streaming/windows/flutter/generated_plugins.cmake b/codelabs/getting-started-flutter-gemma/step_04_streaming/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/getting-started-flutter-gemma/step_04_streaming/windows/flutter/generated_plugins.cmake +++ b/codelabs/getting-started-flutter-gemma/step_04_streaming/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/hybrid-ai-flutter-genkit/complete/android/app/build.gradle.kts b/codelabs/hybrid-ai-flutter-genkit/complete/android/app/build.gradle.kts index d6b2a906d..f6f7c5207 100644 --- a/codelabs/hybrid-ai-flutter-genkit/complete/android/app/build.gradle.kts +++ b/codelabs/hybrid-ai-flutter-genkit/complete/android/app/build.gradle.kts @@ -23,7 +23,10 @@ android { applicationId = "dev.flutterberlin.workshop_flutter_gemma_hybrid_ai" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion versionCode = flutter.versionCode versionName = flutter.versionName diff --git a/codelabs/hybrid-ai-flutter-genkit/complete/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/hybrid-ai-flutter-genkit/complete/macos/Flutter/GeneratedPluginRegistrant.swift index 1a668cbd9..f50d0aebc 100644 --- a/codelabs/hybrid-ai-flutter-genkit/complete/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/hybrid-ai-flutter-genkit/complete/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,12 +5,14 @@ import FlutterMacOS import Foundation +import connectivity_plus import file_selector_macos import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FileSelectorPlugin.register(with: registry.registrar(forPlugin: "FileSelectorPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) diff --git a/codelabs/hybrid-ai-flutter-genkit/complete/pubspec.yaml b/codelabs/hybrid-ai-flutter-genkit/complete/pubspec.yaml index db9aa6061..92f51fa2d 100644 --- a/codelabs/hybrid-ai-flutter-genkit/complete/pubspec.yaml +++ b/codelabs/hybrid-ai-flutter-genkit/complete/pubspec.yaml @@ -18,10 +18,10 @@ dependencies: # On-device AI via genkit_flutter_gemma genkit_flutter_gemma: ^0.6.0 - flutter_gemma: ^1.7.0 + flutter_gemma: ^1.8.1 # LiteRT-LM engine (.litertlm inference) + LiteRT embedding backend — # flutter_gemma 1.x registers no engines by default; opt in here. - flutter_gemma_litertlm: ^1.6.1 + flutter_gemma_litertlm: ^1.6.3 # Hybrid on-device ↔ cloud routing genkit_hybrid: ^0.2.1 diff --git a/codelabs/hybrid-ai-flutter-genkit/complete/windows/flutter/generated_plugin_registrant.cc b/codelabs/hybrid-ai-flutter-genkit/complete/windows/flutter/generated_plugin_registrant.cc index bdc9b64b0..20486aa2f 100644 --- a/codelabs/hybrid-ai-flutter-genkit/complete/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/hybrid-ai-flutter-genkit/complete/windows/flutter/generated_plugin_registrant.cc @@ -6,10 +6,13 @@ #include "generated_plugin_registrant.h" +#include #include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FileSelectorWindowsRegisterWithRegistrar( registry->GetRegistrarForPlugin("FileSelectorWindows")); FlutterGemmaPluginRegisterWithRegistrar( diff --git a/codelabs/hybrid-ai-flutter-genkit/complete/windows/flutter/generated_plugins.cmake b/codelabs/hybrid-ai-flutter-genkit/complete/windows/flutter/generated_plugins.cmake index 812d3d4cc..265de1b53 100644 --- a/codelabs/hybrid-ai-flutter-genkit/complete/windows/flutter/generated_plugins.cmake +++ b/codelabs/hybrid-ai-flutter-genkit/complete/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus file_selector_windows flutter_gemma ) diff --git a/codelabs/hybrid-ai-flutter-genkit/step_00_starter/pubspec.yaml b/codelabs/hybrid-ai-flutter-genkit/step_00_starter/pubspec.yaml index c9d25393f..6b6ec4c77 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_00_starter/pubspec.yaml +++ b/codelabs/hybrid-ai-flutter-genkit/step_00_starter/pubspec.yaml @@ -4,8 +4,8 @@ publish_to: 'none' version: 1.0.0+1 environment: - sdk: '>=3.10.0 <4.0.0' - flutter: '>=3.27.0' + sdk: '>=3.12.0 <4.0.0' + flutter: '>=3.44.0' dependencies: flutter: @@ -13,12 +13,15 @@ dependencies: cupertino_icons: ^1.0.8 # Step 2: Uncomment to add Cloud AI - # genkit: ^0.13.0 - # genkit_google_genai: ^0.2.7 + # genkit: ^0.16.0 + # genkit_google_genai: ^0.3.1 - # Step 3: Uncomment to add On-device AI - # genkit_flutter_gemma: ^0.3.1 - # flutter_gemma: ^0.15.1 + # Step 3: Uncomment to add On-device AI (LiteRT-LM engine) + # genkit_flutter_gemma: ^0.6.0 + # flutter_gemma: ^1.8.1 + # flutter_gemma 1.x registers no engine by default — opt into LiteRT-LM + # (.litertlm inference) here. + # flutter_gemma_litertlm: ^1.6.3 dev_dependencies: flutter_test: diff --git a/codelabs/hybrid-ai-flutter-genkit/step_01_cloud_ai/pubspec.yaml b/codelabs/hybrid-ai-flutter-genkit/step_01_cloud_ai/pubspec.yaml index 3df9cd197..197903c68 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_01_cloud_ai/pubspec.yaml +++ b/codelabs/hybrid-ai-flutter-genkit/step_01_cloud_ai/pubspec.yaml @@ -4,8 +4,8 @@ publish_to: 'none' version: 1.0.0+1 environment: - sdk: '>=3.10.0 <4.0.0' - flutter: '>=3.27.0' + sdk: '>=3.12.0 <4.0.0' + flutter: '>=3.44.0' dependencies: flutter: diff --git a/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/android/app/build.gradle.kts b/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/android/app/build.gradle.kts index c456a54df..3884559fd 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/android/app/build.gradle.kts +++ b/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/android/app/build.gradle.kts @@ -23,7 +23,10 @@ android { applicationId = "dev.flutterberlin.workshop_flutter_gemma_hybrid_ai" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion versionCode = flutter.versionCode versionName = flutter.versionName diff --git a/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/macos/Flutter/GeneratedPluginRegistrant.swift index 0a85e4cb2..0be86a9b4 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,11 +5,13 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) SharedPreferencesPlugin.register(with: registry.registrar(forPlugin: "SharedPreferencesPlugin")) diff --git a/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/pubspec.yaml b/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/pubspec.yaml index d2b9dc22b..581664cc2 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/pubspec.yaml +++ b/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/pubspec.yaml @@ -4,8 +4,8 @@ publish_to: 'none' version: 1.0.0+1 environment: - sdk: '>=3.10.0 <4.0.0' - flutter: '>=3.27.0' + sdk: '>=3.12.0 <4.0.0' + flutter: '>=3.44.0' dependencies: flutter: @@ -18,10 +18,10 @@ dependencies: # Step 3: On-device AI via genkit_flutter_gemma genkit_flutter_gemma: ^0.6.0 - flutter_gemma: ^1.7.0 + flutter_gemma: ^1.8.1 # flutter_gemma 1.x registers no engine by default — opt into LiteRT-LM # (.litertlm inference) here. - flutter_gemma_litertlm: ^1.6.1 + flutter_gemma_litertlm: ^1.6.3 dev_dependencies: flutter_test: diff --git a/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/windows/flutter/generated_plugin_registrant.cc b/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/windows/flutter/generated_plugins.cmake b/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/windows/flutter/generated_plugins.cmake +++ b/codelabs/hybrid-ai-flutter-genkit/step_02_local_ai/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/android/app/build.gradle.kts b/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/android/app/build.gradle.kts index d6b2a906d..f6f7c5207 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/android/app/build.gradle.kts +++ b/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/android/app/build.gradle.kts @@ -23,7 +23,10 @@ android { applicationId = "dev.flutterberlin.workshop_flutter_gemma_hybrid_ai" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion versionCode = flutter.versionCode versionName = flutter.versionName diff --git a/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/macos/Flutter/GeneratedPluginRegistrant.swift index 0a85e4cb2..0be86a9b4 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,11 +5,13 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) SharedPreferencesPlugin.register(with: registry.registrar(forPlugin: "SharedPreferencesPlugin")) diff --git a/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/pubspec.yaml b/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/pubspec.yaml index 507c582a5..5a334fd42 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/pubspec.yaml +++ b/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/pubspec.yaml @@ -18,10 +18,10 @@ dependencies: # On-device AI via genkit_flutter_gemma genkit_flutter_gemma: ^0.6.0 - flutter_gemma: ^1.7.0 + flutter_gemma: ^1.8.1 # LiteRT-LM engine (.litertlm inference) + LiteRT embedding backend — # flutter_gemma 1.x registers no engines by default; opt in here. - flutter_gemma_litertlm: ^1.6.1 + flutter_gemma_litertlm: ^1.6.3 # Hybrid on-device ↔ cloud routing genkit_hybrid: ^0.2.1 diff --git a/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/windows/flutter/generated_plugin_registrant.cc b/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/windows/flutter/generated_plugins.cmake b/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/windows/flutter/generated_plugins.cmake +++ b/codelabs/hybrid-ai-flutter-genkit/step_03_hybrid/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/android/app/build.gradle.kts b/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/android/app/build.gradle.kts index d6b2a906d..f6f7c5207 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/android/app/build.gradle.kts +++ b/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/android/app/build.gradle.kts @@ -23,7 +23,10 @@ android { applicationId = "dev.flutterberlin.workshop_flutter_gemma_hybrid_ai" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion versionCode = flutter.versionCode versionName = flutter.versionName diff --git a/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/macos/Flutter/GeneratedPluginRegistrant.swift index 1a668cbd9..f50d0aebc 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,12 +5,14 @@ import FlutterMacOS import Foundation +import connectivity_plus import file_selector_macos import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FileSelectorPlugin.register(with: registry.registrar(forPlugin: "FileSelectorPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) diff --git a/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/pubspec.yaml b/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/pubspec.yaml index db9aa6061..92f51fa2d 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/pubspec.yaml +++ b/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/pubspec.yaml @@ -18,10 +18,10 @@ dependencies: # On-device AI via genkit_flutter_gemma genkit_flutter_gemma: ^0.6.0 - flutter_gemma: ^1.7.0 + flutter_gemma: ^1.8.1 # LiteRT-LM engine (.litertlm inference) + LiteRT embedding backend — # flutter_gemma 1.x registers no engines by default; opt in here. - flutter_gemma_litertlm: ^1.6.1 + flutter_gemma_litertlm: ^1.6.3 # Hybrid on-device ↔ cloud routing genkit_hybrid: ^0.2.1 diff --git a/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/windows/flutter/generated_plugin_registrant.cc b/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/windows/flutter/generated_plugin_registrant.cc index bdc9b64b0..20486aa2f 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/windows/flutter/generated_plugin_registrant.cc @@ -6,10 +6,13 @@ #include "generated_plugin_registrant.h" +#include #include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FileSelectorWindowsRegisterWithRegistrar( registry->GetRegistrarForPlugin("FileSelectorWindows")); FlutterGemmaPluginRegisterWithRegistrar( diff --git a/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/windows/flutter/generated_plugins.cmake b/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/windows/flutter/generated_plugins.cmake index 812d3d4cc..265de1b53 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/windows/flutter/generated_plugins.cmake +++ b/codelabs/hybrid-ai-flutter-genkit/step_04_smart_routing/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus file_selector_windows flutter_gemma ) diff --git a/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/android/app/build.gradle.kts b/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/android/app/build.gradle.kts index d6b2a906d..f6f7c5207 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/android/app/build.gradle.kts +++ b/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/android/app/build.gradle.kts @@ -23,7 +23,10 @@ android { applicationId = "dev.flutterberlin.workshop_flutter_gemma_hybrid_ai" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion versionCode = flutter.versionCode versionName = flutter.versionName diff --git a/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/macos/Flutter/GeneratedPluginRegistrant.swift index 1a668cbd9..f50d0aebc 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,12 +5,14 @@ import FlutterMacOS import Foundation +import connectivity_plus import file_selector_macos import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FileSelectorPlugin.register(with: registry.registrar(forPlugin: "FileSelectorPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) diff --git a/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/pubspec.yaml b/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/pubspec.yaml index db9aa6061..92f51fa2d 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/pubspec.yaml +++ b/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/pubspec.yaml @@ -18,10 +18,10 @@ dependencies: # On-device AI via genkit_flutter_gemma genkit_flutter_gemma: ^0.6.0 - flutter_gemma: ^1.7.0 + flutter_gemma: ^1.8.1 # LiteRT-LM engine (.litertlm inference) + LiteRT embedding backend — # flutter_gemma 1.x registers no engines by default; opt in here. - flutter_gemma_litertlm: ^1.6.1 + flutter_gemma_litertlm: ^1.6.3 # Hybrid on-device ↔ cloud routing genkit_hybrid: ^0.2.1 diff --git a/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/windows/flutter/generated_plugin_registrant.cc b/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/windows/flutter/generated_plugin_registrant.cc index bdc9b64b0..20486aa2f 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/windows/flutter/generated_plugin_registrant.cc @@ -6,10 +6,13 @@ #include "generated_plugin_registrant.h" +#include #include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FileSelectorWindowsRegisterWithRegistrar( registry->GetRegistrarForPlugin("FileSelectorWindows")); FlutterGemmaPluginRegisterWithRegistrar( diff --git a/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/windows/flutter/generated_plugins.cmake b/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/windows/flutter/generated_plugins.cmake index 812d3d4cc..265de1b53 100644 --- a/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/windows/flutter/generated_plugins.cmake +++ b/codelabs/hybrid-ai-flutter-genkit/step_05_embeddings/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus file_selector_windows flutter_gemma ) diff --git a/codelabs/inference-engines-flutter-gemma/complete/android/app/build.gradle.kts b/codelabs/inference-engines-flutter-gemma/complete/android/app/build.gradle.kts index e815b07fa..caa0c8c70 100644 --- a/codelabs/inference-engines-flutter-gemma/complete/android/app/build.gradle.kts +++ b/codelabs/inference-engines-flutter-gemma/complete/android/app/build.gradle.kts @@ -19,9 +19,11 @@ android { applicationId = "dev.fluttergemma.engines" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - // flutter_gemma_builtin_ai (ML Kit GenAI / AICore) declares minSdk 26; - // the manifest merger rejects an app below it. - minSdk = 26 + // flutter_gemma_builtin_ai (ML Kit GenAI / AICore) declares minSdk 26 and + // the manifest merger rejects an app below it; libLiteRtLm.so needs API 30+ + // Bionic (pthread_cond_clockwait, sem_clockwait) on top of that, so 30 is the + // floor for an app that registers both engines. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/inference-engines-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/inference-engines-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift index cfa44f150..b3bb4f9f5 100644 --- a/codelabs/inference-engines-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/inference-engines-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,12 +5,14 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import flutter_gemma_builtin_ai import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) FlutterGemmaBuiltInAiPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaBuiltInAiPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) diff --git a/codelabs/inference-engines-flutter-gemma/complete/pubspec.yaml b/codelabs/inference-engines-flutter-gemma/complete/pubspec.yaml index 8d9aa6136..6ed0517c0 100644 --- a/codelabs/inference-engines-flutter-gemma/complete/pubspec.yaml +++ b/codelabs/inference-engines-flutter-gemma/complete/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 # The OS built-in engine: Gemini Nano (Android, Chrome), Apple # Foundation Models (iOS, macOS). No file to download — the OS or the # browser owns the weights. Windows and Linux have no built-in arm. diff --git a/codelabs/inference-engines-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc b/codelabs/inference-engines-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/inference-engines-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/inference-engines-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/inference-engines-flutter-gemma/complete/windows/flutter/generated_plugins.cmake b/codelabs/inference-engines-flutter-gemma/complete/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/inference-engines-flutter-gemma/complete/windows/flutter/generated_plugins.cmake +++ b/codelabs/inference-engines-flutter-gemma/complete/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/inference-engines-flutter-gemma/step_01_starter/android/app/build.gradle.kts b/codelabs/inference-engines-flutter-gemma/step_01_starter/android/app/build.gradle.kts index effb20607..9477909e1 100644 --- a/codelabs/inference-engines-flutter-gemma/step_01_starter/android/app/build.gradle.kts +++ b/codelabs/inference-engines-flutter-gemma/step_01_starter/android/app/build.gradle.kts @@ -19,7 +19,10 @@ android { applicationId = "dev.fluttergemma.engines" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/inference-engines-flutter-gemma/step_01_starter/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/inference-engines-flutter-gemma/step_01_starter/macos/Flutter/GeneratedPluginRegistrant.swift index 0a85e4cb2..0be86a9b4 100644 --- a/codelabs/inference-engines-flutter-gemma/step_01_starter/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/inference-engines-flutter-gemma/step_01_starter/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,11 +5,13 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) SharedPreferencesPlugin.register(with: registry.registrar(forPlugin: "SharedPreferencesPlugin")) diff --git a/codelabs/inference-engines-flutter-gemma/step_01_starter/pubspec.yaml b/codelabs/inference-engines-flutter-gemma/step_01_starter/pubspec.yaml index 324dd18bb..442185891 100644 --- a/codelabs/inference-engines-flutter-gemma/step_01_starter/pubspec.yaml +++ b/codelabs/inference-engines-flutter-gemma/step_01_starter/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 dev_dependencies: flutter_test: diff --git a/codelabs/inference-engines-flutter-gemma/step_01_starter/windows/flutter/generated_plugin_registrant.cc b/codelabs/inference-engines-flutter-gemma/step_01_starter/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/inference-engines-flutter-gemma/step_01_starter/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/inference-engines-flutter-gemma/step_01_starter/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/inference-engines-flutter-gemma/step_01_starter/windows/flutter/generated_plugins.cmake b/codelabs/inference-engines-flutter-gemma/step_01_starter/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/inference-engines-flutter-gemma/step_01_starter/windows/flutter/generated_plugins.cmake +++ b/codelabs/inference-engines-flutter-gemma/step_01_starter/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/inference-engines-flutter-gemma/step_02_two_engines/android/app/build.gradle.kts b/codelabs/inference-engines-flutter-gemma/step_02_two_engines/android/app/build.gradle.kts index e815b07fa..caa0c8c70 100644 --- a/codelabs/inference-engines-flutter-gemma/step_02_two_engines/android/app/build.gradle.kts +++ b/codelabs/inference-engines-flutter-gemma/step_02_two_engines/android/app/build.gradle.kts @@ -19,9 +19,11 @@ android { applicationId = "dev.fluttergemma.engines" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - // flutter_gemma_builtin_ai (ML Kit GenAI / AICore) declares minSdk 26; - // the manifest merger rejects an app below it. - minSdk = 26 + // flutter_gemma_builtin_ai (ML Kit GenAI / AICore) declares minSdk 26 and + // the manifest merger rejects an app below it; libLiteRtLm.so needs API 30+ + // Bionic (pthread_cond_clockwait, sem_clockwait) on top of that, so 30 is the + // floor for an app that registers both engines. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/inference-engines-flutter-gemma/step_02_two_engines/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/inference-engines-flutter-gemma/step_02_two_engines/macos/Flutter/GeneratedPluginRegistrant.swift index cfa44f150..b3bb4f9f5 100644 --- a/codelabs/inference-engines-flutter-gemma/step_02_two_engines/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/inference-engines-flutter-gemma/step_02_two_engines/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,12 +5,14 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import flutter_gemma_builtin_ai import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) FlutterGemmaBuiltInAiPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaBuiltInAiPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) diff --git a/codelabs/inference-engines-flutter-gemma/step_02_two_engines/pubspec.yaml b/codelabs/inference-engines-flutter-gemma/step_02_two_engines/pubspec.yaml index 8d9aa6136..6ed0517c0 100644 --- a/codelabs/inference-engines-flutter-gemma/step_02_two_engines/pubspec.yaml +++ b/codelabs/inference-engines-flutter-gemma/step_02_two_engines/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 # The OS built-in engine: Gemini Nano (Android, Chrome), Apple # Foundation Models (iOS, macOS). No file to download — the OS or the # browser owns the weights. Windows and Linux have no built-in arm. diff --git a/codelabs/inference-engines-flutter-gemma/step_02_two_engines/windows/flutter/generated_plugin_registrant.cc b/codelabs/inference-engines-flutter-gemma/step_02_two_engines/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/inference-engines-flutter-gemma/step_02_two_engines/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/inference-engines-flutter-gemma/step_02_two_engines/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/inference-engines-flutter-gemma/step_02_two_engines/windows/flutter/generated_plugins.cmake b/codelabs/inference-engines-flutter-gemma/step_02_two_engines/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/inference-engines-flutter-gemma/step_02_two_engines/windows/flutter/generated_plugins.cmake +++ b/codelabs/inference-engines-flutter-gemma/step_02_two_engines/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/android/app/build.gradle.kts b/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/android/app/build.gradle.kts index e815b07fa..caa0c8c70 100644 --- a/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/android/app/build.gradle.kts +++ b/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/android/app/build.gradle.kts @@ -19,9 +19,11 @@ android { applicationId = "dev.fluttergemma.engines" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - // flutter_gemma_builtin_ai (ML Kit GenAI / AICore) declares minSdk 26; - // the manifest merger rejects an app below it. - minSdk = 26 + // flutter_gemma_builtin_ai (ML Kit GenAI / AICore) declares minSdk 26 and + // the manifest merger rejects an app below it; libLiteRtLm.so needs API 30+ + // Bionic (pthread_cond_clockwait, sem_clockwait) on top of that, so 30 is the + // floor for an app that registers both engines. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/macos/Flutter/GeneratedPluginRegistrant.swift index cfa44f150..b3bb4f9f5 100644 --- a/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,12 +5,14 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import flutter_gemma_builtin_ai import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) FlutterGemmaBuiltInAiPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaBuiltInAiPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) diff --git a/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/pubspec.yaml b/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/pubspec.yaml index 8d9aa6136..6ed0517c0 100644 --- a/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/pubspec.yaml +++ b/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 # The OS built-in engine: Gemini Nano (Android, Chrome), Apple # Foundation Models (iOS, macOS). No file to download — the OS or the # browser owns the weights. Windows and Linux have no built-in arm. diff --git a/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/windows/flutter/generated_plugin_registrant.cc b/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/windows/flutter/generated_plugins.cmake b/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/windows/flutter/generated_plugins.cmake +++ b/codelabs/inference-engines-flutter-gemma/step_03_pick_at_startup/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/multimodal-flutter-gemma/complete/android/app/build.gradle.kts b/codelabs/multimodal-flutter-gemma/complete/android/app/build.gradle.kts index 43915ad44..ad7acaad9 100644 --- a/codelabs/multimodal-flutter-gemma/complete/android/app/build.gradle.kts +++ b/codelabs/multimodal-flutter-gemma/complete/android/app/build.gradle.kts @@ -19,7 +19,10 @@ android { applicationId = "dev.fluttergemma.multimodal" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/multimodal-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/multimodal-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift index 5172c5423..c68ce0d54 100644 --- a/codelabs/multimodal-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/multimodal-flutter-gemma/complete/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,6 +5,7 @@ import FlutterMacOS import Foundation +import connectivity_plus import file_selector_macos import flutter_gemma import large_file_handler @@ -12,6 +13,7 @@ import record_macos import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FileSelectorPlugin.register(with: registry.registrar(forPlugin: "FileSelectorPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) diff --git a/codelabs/multimodal-flutter-gemma/complete/pubspec.yaml b/codelabs/multimodal-flutter-gemma/complete/pubspec.yaml index 5da20b5ab..af075fe9b 100644 --- a/codelabs/multimodal-flutter-gemma/complete/pubspec.yaml +++ b/codelabs/multimodal-flutter-gemma/complete/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 # Picks a photo on all six platforms: a gallery on Android and iOS, a file # dialog (via file_selector) on macOS, Windows, Linux and the web. diff --git a/codelabs/multimodal-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc b/codelabs/multimodal-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc index e1be57539..78c359b53 100644 --- a/codelabs/multimodal-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/multimodal-flutter-gemma/complete/windows/flutter/generated_plugin_registrant.cc @@ -6,11 +6,14 @@ #include "generated_plugin_registrant.h" +#include #include #include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FileSelectorWindowsRegisterWithRegistrar( registry->GetRegistrarForPlugin("FileSelectorWindows")); FlutterGemmaPluginRegisterWithRegistrar( diff --git a/codelabs/multimodal-flutter-gemma/complete/windows/flutter/generated_plugins.cmake b/codelabs/multimodal-flutter-gemma/complete/windows/flutter/generated_plugins.cmake index 9d7135d2f..a12621d05 100644 --- a/codelabs/multimodal-flutter-gemma/complete/windows/flutter/generated_plugins.cmake +++ b/codelabs/multimodal-flutter-gemma/complete/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus file_selector_windows flutter_gemma record_windows diff --git a/codelabs/multimodal-flutter-gemma/step_01_starter/android/app/build.gradle.kts b/codelabs/multimodal-flutter-gemma/step_01_starter/android/app/build.gradle.kts index 43915ad44..ad7acaad9 100644 --- a/codelabs/multimodal-flutter-gemma/step_01_starter/android/app/build.gradle.kts +++ b/codelabs/multimodal-flutter-gemma/step_01_starter/android/app/build.gradle.kts @@ -19,7 +19,10 @@ android { applicationId = "dev.fluttergemma.multimodal" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/multimodal-flutter-gemma/step_01_starter/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/multimodal-flutter-gemma/step_01_starter/macos/Flutter/GeneratedPluginRegistrant.swift index 0a85e4cb2..0be86a9b4 100644 --- a/codelabs/multimodal-flutter-gemma/step_01_starter/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/multimodal-flutter-gemma/step_01_starter/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,11 +5,13 @@ import FlutterMacOS import Foundation +import connectivity_plus import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) SharedPreferencesPlugin.register(with: registry.registrar(forPlugin: "SharedPreferencesPlugin")) diff --git a/codelabs/multimodal-flutter-gemma/step_01_starter/pubspec.yaml b/codelabs/multimodal-flutter-gemma/step_01_starter/pubspec.yaml index 324dd18bb..442185891 100644 --- a/codelabs/multimodal-flutter-gemma/step_01_starter/pubspec.yaml +++ b/codelabs/multimodal-flutter-gemma/step_01_starter/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 dev_dependencies: flutter_test: diff --git a/codelabs/multimodal-flutter-gemma/step_01_starter/windows/flutter/generated_plugin_registrant.cc b/codelabs/multimodal-flutter-gemma/step_01_starter/windows/flutter/generated_plugin_registrant.cc index 0d9da1814..38c2332a8 100644 --- a/codelabs/multimodal-flutter-gemma/step_01_starter/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/multimodal-flutter-gemma/step_01_starter/windows/flutter/generated_plugin_registrant.cc @@ -6,9 +6,12 @@ #include "generated_plugin_registrant.h" +#include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FlutterGemmaPluginRegisterWithRegistrar( registry->GetRegistrarForPlugin("FlutterGemmaPlugin")); } diff --git a/codelabs/multimodal-flutter-gemma/step_01_starter/windows/flutter/generated_plugins.cmake b/codelabs/multimodal-flutter-gemma/step_01_starter/windows/flutter/generated_plugins.cmake index 71fc21b26..8dcbac209 100644 --- a/codelabs/multimodal-flutter-gemma/step_01_starter/windows/flutter/generated_plugins.cmake +++ b/codelabs/multimodal-flutter-gemma/step_01_starter/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus flutter_gemma ) diff --git a/codelabs/multimodal-flutter-gemma/step_02_vision/android/app/build.gradle.kts b/codelabs/multimodal-flutter-gemma/step_02_vision/android/app/build.gradle.kts index 43915ad44..ad7acaad9 100644 --- a/codelabs/multimodal-flutter-gemma/step_02_vision/android/app/build.gradle.kts +++ b/codelabs/multimodal-flutter-gemma/step_02_vision/android/app/build.gradle.kts @@ -19,7 +19,10 @@ android { applicationId = "dev.fluttergemma.multimodal" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/multimodal-flutter-gemma/step_02_vision/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/multimodal-flutter-gemma/step_02_vision/macos/Flutter/GeneratedPluginRegistrant.swift index 1a668cbd9..f50d0aebc 100644 --- a/codelabs/multimodal-flutter-gemma/step_02_vision/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/multimodal-flutter-gemma/step_02_vision/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,12 +5,14 @@ import FlutterMacOS import Foundation +import connectivity_plus import file_selector_macos import flutter_gemma import large_file_handler import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FileSelectorPlugin.register(with: registry.registrar(forPlugin: "FileSelectorPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) diff --git a/codelabs/multimodal-flutter-gemma/step_02_vision/pubspec.yaml b/codelabs/multimodal-flutter-gemma/step_02_vision/pubspec.yaml index 2d6d15848..b9c96fc01 100644 --- a/codelabs/multimodal-flutter-gemma/step_02_vision/pubspec.yaml +++ b/codelabs/multimodal-flutter-gemma/step_02_vision/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 # Picks a photo on all six platforms: a gallery on Android and iOS, a file # dialog (via file_selector) on macOS, Windows, Linux and the web. diff --git a/codelabs/multimodal-flutter-gemma/step_02_vision/windows/flutter/generated_plugin_registrant.cc b/codelabs/multimodal-flutter-gemma/step_02_vision/windows/flutter/generated_plugin_registrant.cc index bdc9b64b0..20486aa2f 100644 --- a/codelabs/multimodal-flutter-gemma/step_02_vision/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/multimodal-flutter-gemma/step_02_vision/windows/flutter/generated_plugin_registrant.cc @@ -6,10 +6,13 @@ #include "generated_plugin_registrant.h" +#include #include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FileSelectorWindowsRegisterWithRegistrar( registry->GetRegistrarForPlugin("FileSelectorWindows")); FlutterGemmaPluginRegisterWithRegistrar( diff --git a/codelabs/multimodal-flutter-gemma/step_02_vision/windows/flutter/generated_plugins.cmake b/codelabs/multimodal-flutter-gemma/step_02_vision/windows/flutter/generated_plugins.cmake index 812d3d4cc..265de1b53 100644 --- a/codelabs/multimodal-flutter-gemma/step_02_vision/windows/flutter/generated_plugins.cmake +++ b/codelabs/multimodal-flutter-gemma/step_02_vision/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus file_selector_windows flutter_gemma ) diff --git a/codelabs/multimodal-flutter-gemma/step_03_audio/android/app/build.gradle.kts b/codelabs/multimodal-flutter-gemma/step_03_audio/android/app/build.gradle.kts index 43915ad44..ad7acaad9 100644 --- a/codelabs/multimodal-flutter-gemma/step_03_audio/android/app/build.gradle.kts +++ b/codelabs/multimodal-flutter-gemma/step_03_audio/android/app/build.gradle.kts @@ -19,7 +19,10 @@ android { applicationId = "dev.fluttergemma.multimodal" // You can update the following values to match your application needs. // For more information, see: https://flutter.dev/to/review-gradle-config. - minSdk = flutter.minSdkVersion + // libLiteRtLm.so needs API 30+ Bionic (pthread_cond_clockwait, + // sem_clockwait). Below 30 the app installs and then fails at the first + // model load with a dlopen error. + minSdk = 30 targetSdk = flutter.targetSdkVersion // Uses the version code from pubspec.yaml. When using split APKs, 1000 * ABI_VERSION // is added automatically by Flutter. (https://developer.android.com/studio/build/configure-apk-splits#configure-APK-versions) diff --git a/codelabs/multimodal-flutter-gemma/step_03_audio/macos/Flutter/GeneratedPluginRegistrant.swift b/codelabs/multimodal-flutter-gemma/step_03_audio/macos/Flutter/GeneratedPluginRegistrant.swift index 5172c5423..c68ce0d54 100644 --- a/codelabs/multimodal-flutter-gemma/step_03_audio/macos/Flutter/GeneratedPluginRegistrant.swift +++ b/codelabs/multimodal-flutter-gemma/step_03_audio/macos/Flutter/GeneratedPluginRegistrant.swift @@ -5,6 +5,7 @@ import FlutterMacOS import Foundation +import connectivity_plus import file_selector_macos import flutter_gemma import large_file_handler @@ -12,6 +13,7 @@ import record_macos import shared_preferences_foundation func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) { + ConnectivityPlusPlugin.register(with: registry.registrar(forPlugin: "ConnectivityPlusPlugin")) FileSelectorPlugin.register(with: registry.registrar(forPlugin: "FileSelectorPlugin")) FlutterGemmaPlugin.register(with: registry.registrar(forPlugin: "FlutterGemmaPlugin")) LargeFileHandlerPlugin.register(with: registry.registrar(forPlugin: "LargeFileHandlerPlugin")) diff --git a/codelabs/multimodal-flutter-gemma/step_03_audio/pubspec.yaml b/codelabs/multimodal-flutter-gemma/step_03_audio/pubspec.yaml index 5da20b5ab..af075fe9b 100644 --- a/codelabs/multimodal-flutter-gemma/step_03_audio/pubspec.yaml +++ b/codelabs/multimodal-flutter-gemma/step_03_audio/pubspec.yaml @@ -36,10 +36,10 @@ dependencies: cupertino_icons: ^1.0.8 # The engine-agnostic core: registry, install/runtime API, chat. - flutter_gemma: ^1.7.1 + flutter_gemma: ^1.8.1 # The .litertlm inference engine. Engines are opt-in — core # registers none, so this package must be added explicitly. - flutter_gemma_litertlm: ^1.6.2 + flutter_gemma_litertlm: ^1.6.3 # Picks a photo on all six platforms: a gallery on Android and iOS, a file # dialog (via file_selector) on macOS, Windows, Linux and the web. diff --git a/codelabs/multimodal-flutter-gemma/step_03_audio/windows/flutter/generated_plugin_registrant.cc b/codelabs/multimodal-flutter-gemma/step_03_audio/windows/flutter/generated_plugin_registrant.cc index e1be57539..78c359b53 100644 --- a/codelabs/multimodal-flutter-gemma/step_03_audio/windows/flutter/generated_plugin_registrant.cc +++ b/codelabs/multimodal-flutter-gemma/step_03_audio/windows/flutter/generated_plugin_registrant.cc @@ -6,11 +6,14 @@ #include "generated_plugin_registrant.h" +#include #include #include #include void RegisterPlugins(flutter::PluginRegistry* registry) { + ConnectivityPlusWindowsPluginRegisterWithRegistrar( + registry->GetRegistrarForPlugin("ConnectivityPlusWindowsPlugin")); FileSelectorWindowsRegisterWithRegistrar( registry->GetRegistrarForPlugin("FileSelectorWindows")); FlutterGemmaPluginRegisterWithRegistrar( diff --git a/codelabs/multimodal-flutter-gemma/step_03_audio/windows/flutter/generated_plugins.cmake b/codelabs/multimodal-flutter-gemma/step_03_audio/windows/flutter/generated_plugins.cmake index 9d7135d2f..a12621d05 100644 --- a/codelabs/multimodal-flutter-gemma/step_03_audio/windows/flutter/generated_plugins.cmake +++ b/codelabs/multimodal-flutter-gemma/step_03_audio/windows/flutter/generated_plugins.cmake @@ -3,6 +3,7 @@ # list(APPEND FLUTTER_PLUGIN_LIST + connectivity_plus file_selector_windows flutter_gemma record_windows diff --git a/packages/flutter_gemma/.pubignore b/packages/flutter_gemma/.pubignore index fa78459e5..d87cd76cf 100644 --- a/packages/flutter_gemma/.pubignore +++ b/packages/flutter_gemma/.pubignore @@ -33,6 +33,10 @@ web/*.js.map !web/rag/ # And keep cache_api.js (hand-written, runtime needs window.cacheHas/Put/Get) !web/cache_api.js +# And opfs_helper.js — defines window.flutterGemmaOPFS, which web streaming +# storage (WebStorageMode.streaming) binds to. Without this line it never +# reached pub, so the documented copy-from-the-package step had nothing to copy. +!web/opfs_helper.js # Media files (for README, not needed in package) docs/ @@ -133,4 +137,15 @@ example/web/test_vectorstore.html # chromedriver binary — maintainer tool for running web integration tests # (flutter drive -d chrome). ~16 MB, not part of the plugin. Without this # it leaks into the pub package and bloats it (8 MB → archive). -chromedriver/ \ No newline at end of file +chromedriver/ + +# Agent skills MUST ship inside the archive — `dart run skills@ get` resolves +# the package on disk from package_config.json and reads `skills/` from there. +# Without this exception the blanket `**/*.md` above silently strips every +# SKILL.md and the skills reach nobody, with no error anywhere. +# +# It is LAST on purpose: the last matching rule wins, and rules such as +# `assets/`, `docs/` and `test/` would otherwise drop a skill subdirectory with +# those names — `assets/` is part of the Agent Skills layout. +!skills/ +!skills/** diff --git a/packages/flutter_gemma/CHANGELOG.md b/packages/flutter_gemma/CHANGELOG.md index bfafaeddd..59184d5b9 100644 --- a/packages/flutter_gemma/CHANGELOG.md +++ b/packages/flutter_gemma/CHANGELOG.md @@ -1,3 +1,6 @@ +## 1.8.2 +- Ship agent skills — `dart run skills@ get --all` teaches your AI assistant this package. + ## 1.8.1 - Add `VectorStoreRepository.flush()`; custom implementations must declare it (#492). diff --git a/packages/flutter_gemma/DESKTOP_SUPPORT.md b/packages/flutter_gemma/DESKTOP_SUPPORT.md index d480af3da..5c7490f40 100644 --- a/packages/flutter_gemma/DESKTOP_SUPPORT.md +++ b/packages/flutter_gemma/DESKTOP_SUPPORT.md @@ -60,7 +60,7 @@ loading sequence differs per platform (handled in `litert_lm_client.dart`). > > Desktop accepts only LiteRT-LM `.litertlm` files. MediaPipe `.bin` / `.task` > models used on web won't load on desktop. See -> [AI Edge Model Garden](https://ai.google.dev/edge/litert/models) for compatible models. +> [litert-community on Hugging Face](https://huggingface.co/litert-community) for compatible models. --- diff --git a/packages/flutter_gemma/README.md b/packages/flutter_gemma/README.md index d539e71c3..d9b673e56 100644 --- a/packages/flutter_gemma/README.md +++ b/packages/flutter_gemma/README.md @@ -56,17 +56,33 @@ There is an example of using: - **🔐 Typed Download Errors:** Catch the public `DownloadException` sealed type (401/403/404/429/5xx) for gated HuggingFace models instead of substring-matching error strings - **💾 Web Persistent Caching:** Models persist across browser restarts — Cache API for models <2GB, OPFS streaming for large ones (>2GB, e.g. Gemma 4 E4B) — no re-download on reload (Web only) -## What's new in 1.6.4 +## Teach your AI assistant this package -- 📱 **iOS deployment floor lowered to 15.0** — core, built-in AI and embeddings build from iOS 15.0 (only `flutter_gemma_mediapipe` still needs 16.0). Every OS-26-only Foundation Models call is `#available`-guarded ([#441](https://github.com/DenisovAV/flutter_gemma/issues/441)). +`flutter_gemma` ships [agent skills](https://dart.dev/blog/skills-cli-1-0-bundle-and-distribute-ai-agent-skills-for-your-packages) — short instruction files your coding assistant reads so it uses this API correctly the first time: -## What's new in 1.6.3 +```bash +dart run skills@ get --all +``` + +That scans your dependencies and installs every skill they bundle where your agent looks — Claude Code, Codex, Cursor, Antigravity, Cline, Copilot and OpenCode are supported. If it reports that it could not detect your agent, name it with `--agent claude` (or `codex`, `cursor`, …). + +What they cover: registering an engine (core ships none), routing by the declared `ModelFileType` rather than the filename, and the two defaults that fail quietly — `maxTokens` is the context window and not the reply length, and `Message.isUser` defaults to `false`. + +## What's new in 1.8.2 + +- 🤖 **Agent skills ship with the package** — `dart run skills@ get --all` installs seven skills that teach your coding assistant this API: inference (with platform setup), function calling, RAG, speech, MediaPipe, ONNX and built-in AI. Every code block in them is compiled against these packages before each release. + +## What's new in 1.8.1 + +- 💾 **`VectorStoreRepository.flush()`** — a RAG index now survives the process; custom `VectorStoreRepository` implementations must declare it ([#492](https://github.com/DenisovAV/flutter_gemma/issues/492)). + +## What's new in 1.8.0 -- 📥 **flutter_gemma no longer claims `background_downloader`'s updates stream** — depending on this package used to make `FileDownloader().updates` unusable for your own downloads, because that stream takes a single subscription. Updates are now scoped to flutter_gemma's own task group ([#445](https://github.com/DenisovAV/flutter_gemma/issues/445)). Download priority is also corrected per platform. +- 🗣️ **Whisper output language per transcription** — `getActiveStt(language:)` sets the default and `transcribe(pcm, language:)` overrides it for one call, with no reload. **Breaking for custom `SpeechRecognizer` implementations**: `transcribe` gained `language:` and the type gained a `language` field ([#500](https://github.com/DenisovAV/flutter_gemma/issues/500)). -## What's new in 1.6.2 +## What's new in 1.7.0 -- 🌐 **ONNX on Web** — `flutter_gemma_onnx`'s `OnnxEngine` now generates text on Web via Transformers.js, with a fileless `ModelFileType.onnx` install (the model is a Hugging Face repo id, not a directory). `OnnxEmbeddingBackend` gained a web arm too, via onnxruntime-web. See [`flutter_gemma_onnx`](https://pub.dev/packages/flutter_gemma_onnx). +- 🤗 **One-call Hugging Face installs** — `fromHuggingFace(repo)` reads a repo's deployment manifest, picks the variant for the device and returns its tested runtime defaults; every engine carries its own resolver. 📖 Full docs & guides: **[fluttergemma.dev](https://fluttergemma.dev)** @@ -1682,8 +1698,11 @@ await FlutterGemma.installEmbedder() ) .install(); -// 2. Initialize the vector store (one shard per database path) -await FlutterGemmaPlugin.instance.initializeVectorStore('rag_store'); +// 2. Initialize the vector store (one shard per database path). On native pass +// an absolute path: a bare name resolves against the process working +// directory, which is not writable on Android or iOS. On web a name is enough. +final dir = await getApplicationDocumentsDirectory(); // package:path_provider +await FlutterGemmaPlugin.instance.initializeVectorStore('${dir.path}/rag_store'); // 3. Add documents — let the plugin compute embeddings for you for (final doc in docs) { diff --git a/packages/flutter_gemma/example/pubspec.lock b/packages/flutter_gemma/example/pubspec.lock index f034fe913..e6234ffb7 100644 --- a/packages/flutter_gemma/example/pubspec.lock +++ b/packages/flutter_gemma/example/pubspec.lock @@ -241,7 +241,7 @@ packages: path: ".." relative: true source: path - version: "1.8.1" + version: "1.8.2" flutter_gemma_agent: dependency: "direct main" description: diff --git a/packages/flutter_gemma/ios/flutter_gemma.podspec b/packages/flutter_gemma/ios/flutter_gemma.podspec index 01dfa64d5..51eaba207 100644 --- a/packages/flutter_gemma/ios/flutter_gemma.podspec +++ b/packages/flutter_gemma/ios/flutter_gemma.podspec @@ -4,7 +4,7 @@ # Pod::Spec.new do |s| s.name = 'flutter_gemma' - s.version = '1.8.1' + s.version = '1.8.2' s.summary = 'Flutter plugin for running Gemma and other LLMs locally on iOS.' s.description = <<-DESC Core runtime for running Gemma 4, Gemma3n, Gemma 3, FastVLM, Qwen3, diff --git a/packages/flutter_gemma/lib/core/api/flutter_gemma.dart b/packages/flutter_gemma/lib/core/api/flutter_gemma.dart index 676a07875..5f881747a 100644 --- a/packages/flutter_gemma/lib/core/api/flutter_gemma.dart +++ b/packages/flutter_gemma/lib/core/api/flutter_gemma.dart @@ -1011,7 +1011,11 @@ class FlutterGemma { /// throws a clear "add a RAG package" error. /// /// ```dart - /// await FlutterGemma.rag.initialize('rag.db'); + /// // Native: an absolute path in a writable directory. A bare name resolves + /// // against the process working directory, which is not writable on + /// // Android or iOS. Web: a bare name is fine. + /// final dir = await getApplicationDocumentsDirectory(); // path_provider + /// await FlutterGemma.rag.initialize('${dir.path}/rag.db'); /// await FlutterGemma.rag.addDocument(id: '1', content: 'hello'); /// final hits = await FlutterGemma.rag.searchSimilar(query: 'hi'); /// await FlutterGemma.rag.removeDocument(id: '1'); diff --git a/packages/flutter_gemma/macos/flutter_gemma.podspec b/packages/flutter_gemma/macos/flutter_gemma.podspec index a745e660d..fed89b41d 100644 --- a/packages/flutter_gemma/macos/flutter_gemma.podspec +++ b/packages/flutter_gemma/macos/flutter_gemma.podspec @@ -4,7 +4,7 @@ # Pod::Spec.new do |s| s.name = 'flutter_gemma' - s.version = '1.8.1' + s.version = '1.8.2' s.summary = 'Flutter Gemma - Run Gemma AI models locally on desktop' s.description = <<-DESC Flutter plugin for running Gemma AI models locally on macOS using LiteRT-LM. diff --git a/packages/flutter_gemma/pubspec.yaml b/packages/flutter_gemma/pubspec.yaml index 304ad1965..9b5b214fe 100644 --- a/packages/flutter_gemma/pubspec.yaml +++ b/packages/flutter_gemma/pubspec.yaml @@ -1,6 +1,6 @@ name: flutter_gemma description: "Run Gemma and other LLMs on-device in Flutter (Android, iOS, Web, Desktop). Multimodal vision/audio, function calling, thinking mode, GPU, embeddings, RAG." -version: 1.8.1 +version: 1.8.2 resolution: workspace homepage: https://fluttergemma.dev repository: https://github.com/DenisovAV/flutter_gemma diff --git a/packages/flutter_gemma/skills/flutter-gemma-builtin-ai/SKILL.md b/packages/flutter_gemma/skills/flutter-gemma-builtin-ai/SKILL.md new file mode 100644 index 000000000..2094e4648 --- /dev/null +++ b/packages/flutter_gemma/skills/flutter-gemma-builtin-ai/SKILL.md @@ -0,0 +1,120 @@ +--- +name: flutter-gemma-builtin-ai +description: Use when running the device's own model with flutter_gemma_builtin_ai — Gemini Nano on Android or in desktop Chrome, Phi-4-mini in Microsoft Edge, Apple Foundation Models on iPhone, iPad and Mac — with nothing to download or bundle, or when falling back to a downloaded model where it is missing. Also use when BuiltInAiUnavailableException or a TimeoutException is thrown, availability reports "downloadable", web throws NotAllowedError about a user gesture, the Android build fails the manifest merge on minSdk, or the model is missing in Chrome. For models the app downloads itself, use flutter-gemma-inference. +--- + +# The built-in OS model + +## Rules + +1. The OS owns the weights, but the model is still installed — as an identity: `fileType: ModelFileType.builtIn` with `.fromBundled(...)`. The app downloads nothing. +2. Call `BuiltInAi.ensureReady()` before `getActiveModel()`, from a user action: the first call downloads the model and can take minutes, and on web the browser refuses to start that download without a user gesture. Call it straight from the tap handler, with no slow `await` in front of it. It throws `TimeoutException` after `timeout` — 10 minutes by default. +3. On web a missing gesture is **not** distinguishable by type: `ensureReady` rewraps it as `BuiltInAiUnavailableException` with `unavailableOther`, and only `.message` carries the browser's "NotAllowedError: Requires a user gesture". Read the message before concluding the device cannot do it — otherwise the fallback below downloads gigabytes for nothing. +4. Catch `BuiltInAiUnavailableException` and fall back to a downloadable model. +5. Android apps need `minSdk 26`, or the manifest merge fails. +6. There is no Windows or Linux support — and `BuiltInAi.availability()` does not report that, it throws a Flutter PlatformException (from package:flutter/services.dart) there. Guard by platform before calling it, as the setup below does. + +## Setup with a fallback + +```sh +flutter pub add flutter_gemma flutter_gemma_builtin_ai flutter_gemma_litertlm +``` + +```dart +import 'dart:async'; + +import 'package:flutter/foundation.dart'; +import 'package:flutter_gemma/flutter_gemma.dart'; +import 'package:flutter_gemma_builtin_ai/flutter_gemma_builtin_ai.dart'; +import 'package:flutter_gemma_litertlm/flutter_gemma_litertlm.dart'; + +await FlutterGemma.initialize( + inferenceEngines: [BuiltInAiEngine(), LiteRtLmEngine()], +); + +final spec = kIsWeb || defaultTargetPlatform == TargetPlatform.android + ? BuiltInAiModels.geminiNano + : defaultTargetPlatform == TargetPlatform.iOS || + defaultTargetPlatform == TargetPlatform.macOS + ? BuiltInAiModels.appleFoundationModels + : null; // Windows and Linux have no built-in model + +Future downloadGemma() async { + await FlutterGemma.installModel( + modelType: ModelType.gemma4, + fileType: ModelFileType.litertlm, + ).fromNetwork( + // 2.6 GB — ask first. On web: gemma-4-E2B-it-web.litertlm + 'https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/gemma-4-E2B-it.litertlm', + ).install(); + return FlutterGemma.getActiveModel(maxTokens: 1024); +} + +InferenceModel model; +if (spec == null) { + model = await downloadGemma(); +} else { + try { + await FlutterGemma.installModel( + modelType: ModelType.general, + fileType: ModelFileType.builtIn, + ).fromBundled(spec.name).install(); + // onProgress reports real percentages on web only: ML Kit gives no byte + // total on Android, and Apple downloads nothing — ensureReady just waits. + await BuiltInAi.ensureReady(onProgress: (int percent) => print('$percent%')); + model = await FlutterGemma.getActiveModel(maxTokens: 4096); + } on BuiltInAiUnavailableException { + model = await downloadGemma(); + } on TimeoutException { + model = await downloadGemma(); + } +} +``` + +The latest install is the one `getActiveModel` loads, so the fallback replaces the built-in model. Sessions and chats then work as in the flutter-gemma-inference skill. + +## Checking before offering the feature + +```dart +final availability = await BuiltInAi.availability(); +final usable = availability == BuiltInAiAvailability.available || + availability == BuiltInAiAvailability.downloadable || + availability == BuiltInAiAvailability.downloading; +``` + +`downloadable` and `downloading` mean "not yet" — `ensureReady` finishes the job. The `unavailable*` states describe the device now, not forever: + +| State | Meaning | +| --- | --- | +| `BuiltInAiAvailability.unavailableDeviceUnsupported` | the hardware cannot run it — and on web, that the browser exposes no Prompt API at all, including desktop Chrome with the flag off | +| `BuiltInAiAvailability.unavailableOsTooOld` | an OS update would enable it; on Apple the floor is OS 26 | +| `BuiltInAiAvailability.unavailableDisabled` | the user can turn it on in system settings (Apple Intelligence on Apple devices) | +| `BuiltInAiAvailability.unavailableOther` | anything else: a probe that timed out after 20 seconds, or Chrome's reasonless "unavailable", which in practice is its disk and VRAM floor — worth asking again later | + +## Platforms + +| Platform | Model | Needs | +| --- | --- | --- | +| Android | Gemini Nano (AICore) | Pixel 9+, Galaxy S25+; `minSdk 26` | +| iOS, macOS | Apple Foundation Models | iOS 26+ / macOS 26+ on an iPhone 15 Pro or newer, or an Apple Silicon Mac, with Apple Intelligence on. The package itself builds from iOS 15 / macOS 10.15, so no deployment-target bump | +| Web | Gemini Nano (Chrome Prompt API) | desktop Chrome — not mobile browsers, Firefox or Safari | +| Web | Phi-4-mini (Edge Prompt API) | Microsoft Edge with the Prompt API flag on; Edge Dev 154–155 exposes the API but cannot run the model | + +Images work on Android only, one per message, and only when asked for: `getActiveModel(maxTokens: 4096, supportImage: true)` **and** `createChat(supportImage: true)`. With the default `supportImage: false` the image is dropped with no warning. On Apple every image fails on every OS version — a Flutter PlatformException with code "IMAGE_UNSUPPORTED_OS" — the package builds against the OS 26 SDK, which has no attachment API — so do not build an image path there. The web model is text-only. + +## Web + +There is no script to add: the Prompt API is part of the browser. It has to be enabled. + +- Production — register the origin for the Prompt API origin trial and add the token to `web/index.html`: + +```html + +``` + +- Local development — enable `chrome://flags/#prompt-api-for-gemini-nano` and restart Chrome. +- Microsoft Edge — enable `edge://flags` → "Prompt API for on-device language model" and restart. Both browsers use `BuiltInAiModels.geminiNano`: the spec names the API, and the browser picks the model. + +## Trade-offs + +No choice of weights, no LoRA, and capabilities that vary by OS version. The right pick when zero download and zero disk matter more than choosing the model. diff --git a/packages/flutter_gemma/skills/flutter-gemma-function-calling/SKILL.md b/packages/flutter_gemma/skills/flutter-gemma-function-calling/SKILL.md new file mode 100644 index 000000000..d335fd7b0 --- /dev/null +++ b/packages/flutter_gemma/skills/flutter-gemma-function-calling/SKILL.md @@ -0,0 +1,118 @@ +--- +name: flutter-gemma-function-calling +description: Use when adding function calling (tool calling) to a flutter_gemma chat — letting an on-device model call the app's own Dart functions, declaring Tool objects, handling FunctionCallResponse or ParallelFunctionCallResponse, returning results with Message.toolResponse, or running the built-in tool loop generateChatResponseWithTools. Also use when the model describes an action in prose instead of calling the tool, raw tool-call markers appear in the reply text, or a switch over ModelResponse fails to compile. For plain chat, use flutter-gemma-inference. Not for flutter_gemma_agent, which gives the on-device model SKILL.md skills of its own. +--- + +# Function calling with flutter_gemma + +Packages, engine and model install are in the flutter-gemma-inference skill. This one starts from a loaded `InferenceModel`. + +## Rules + +1. `createChat` needs `tools` and `supportsFunctionCalls: true`. Without the flag no call is parsed and only a debug warning is logged — and on Gemma 4 with `.litertlm` the declarations still reach the SDK, so the model answers with raw tool-call JSON inside the text stream. +2. Pass `modelType` on web and on ONNX. `createChat` on native `.litertlm`, on MediaPipe Android and iOS, and on built-in AI uses the installed model's type when it is left out; the web engines and ONNX fall back to `ModelType.gemmaIt`, and another model's calls then arrive as raw text. `openChat` always falls back — pass it there on every platform. +3. Switch over all four `ModelResponse` subtypes. It is sealed — a switch that leaves out `ThinkingResponse` does not compile. +4. Return tool results as data, errors included. Never throw from a tool. +5. Prefer `generateChatResponseWithTools` to a hand-written loop. +6. Use a tool-capable model: Gemma 4, Gemma 3n, Gemma 3 1B, FunctionGemma, Phi-4 Mini, Qwen 2.5, Qwen3, DeepSeek R1. Gemma 3 270M and SmolLM cannot call tools. + +## Declare a tool + +`parameters` is a JSON Schema object. The model matches the user's intent against `description`, so write it as an action and describe every parameter. + +```dart +import 'package:flutter_gemma/flutter_gemma.dart'; + +const changeColor = Tool( + name: 'change_color', + description: 'Change the app background colour.', + parameters: { + 'type': 'object', + 'properties': { + 'color': {'type': 'string', 'description': 'A colour name, e.g. red.'}, + }, + 'required': ['color'], + }, +); +``` + +## Open the chat + +```dart +final InferenceChat chat = await model.createChat( + tools: myTools, + supportsFunctionCalls: true, + modelType: ModelType.gemma4, +); +``` + +## The built-in loop + +It calls the handler for each tool call, feeds the result back, and continues until the model answers in text or `maxToolTurns` is reached. + +```dart +await chat.addQueryChunk(Message(text: prompt, isUser: true)); +final answer = StringBuffer(); +await for (final r in chat.generateChatResponseWithTools( + onToolCall: (FunctionCallResponse call) => runTool(call.name, call.args), + maxToolTurns: 8, + onMaxToolTurns: () => print('stopped after 8 tool turns'), +)) { + if (r is TextResponse) answer.write(r.token); +} +``` + +`onToolCall` receives a `FunctionCallResponse` with `name` and `args`, and returns the map the model reads back. The stream carries `TextResponse` and `ThinkingResponse`. Reaching `maxToolTurns` ends the stream without an error — `onMaxToolTurns` is the only signal. An exception from `onToolCall` is reported to the model, then rethrown on the stream. + +## Handling calls yourself + +```dart +final response = await chat.generateChatResponse(); +switch (response) { + case FunctionCallResponse(:final name, :final args): + final result = await runTool(name, args); + await chat.addQueryChunk( + Message.toolResponse(toolName: name, response: result), + ); + final followUp = await chat.generateChatResponse(); + case ParallelFunctionCallResponse(:final calls): + for (final call in calls) { + final result = await runTool(call.name, call.args); + await chat.addQueryChunk( + Message.toolResponse(toolName: call.name, response: result), + ); + } + final afterAll = await chat.generateChatResponse(); + case TextResponse(:final token): + print(token); // the model chose to answer directly — a valid outcome + case ThinkingResponse(): + break; +} +``` + +## Errors are results + +```dart +await chat.addQueryChunk( + Message.toolResponse( + toolName: 'change_color', + response: {'error': 'unknown colour: mauvish'}, + ), +); +``` + +The model can recover from an error it can read. An exception thrown out of a tool ends the turn instead. + +## Traps + +**Model answers in prose** +- Symptom: "I would change the colour to red" instead of a call. +- Fix: check `supportsFunctionCalls: true`, pass `modelType` on web and ONNX, then check the model is tool-capable. + +**Raw markers in the text** +- Symptom: `<|tool_call>` or `` appears in `TextResponse` tokens. +- Fix: `modelType` does not match the installed model. + +## Web + +Function calling works on the `.litertlm` web engine. Pass `modelType`, and close each chat before creating the next — the web engine holds one session at a time. diff --git a/packages/flutter_gemma/skills/flutter-gemma-inference/SKILL.md b/packages/flutter_gemma/skills/flutter-gemma-inference/SKILL.md new file mode 100644 index 000000000..c45cbf720 --- /dev/null +++ b/packages/flutter_gemma/skills/flutter-gemma-inference/SKILL.md @@ -0,0 +1,257 @@ +--- +name: flutter-gemma-inference +description: Use when adding on-device LLM inference to a Flutter app with flutter_gemma — offline chat, running Gemma, Qwen or Phi locally, installing a model from Hugging Face (gated repos included), streaming replies, a system prompt, thinking or reasoning output, image or audio prompts, picking a CPU, GPU or NPU backend, stopping generation — or setting up the recommended .litertlm engine (ModelFileType.litertlm) on Android, iOS, macOS, Windows, Linux or web, including the Android minSdk and internet permission, the Apple entitlements and Podfile, and the web index.html script tags. Also use when a reply comes back empty, the model answers identically every time, maxTokens does not shorten replies, FlutterGemma is an undefined name, getActiveModel throws "No inference engine can handle this model", a session throws "Session is closed", or .litertlm fails to load on Android. For .task or .bin models (ModelFileType.task or ModelFileType.binary), use flutter-gemma-mediapipe. +--- + +# Running a model with flutter_gemma + +## Rules + +1. Depend on `flutter_gemma` and an engine package, and import both. Engine packages do not re-export core. +2. Register the engine in `FlutterGemma.initialize(inferenceEngines: [...])`. Core ships none. +3. Declare `fileType` on `installModel`. It defaults to `ModelFileType.task`, and the declaration — never the file name — picks the engine. +4. `maxTokens` is the context window. Cap the reply with `maxOutputTokens` on the session or chat. +5. Pass `isUser: true` on every user `Message`. +6. Close a session or chat when its conversation ends. Keep the model while the feature is in use, and close it when the app no longer needs it. +7. Keep Hugging Face tokens out of source: read them with `String.fromEnvironment`. That keeps a token out of git, not out of the app — it is compiled into the binary, and on web into `main.dart.js`. A shipped app should download from a repo that needs no token. +8. On Android, set `minSdk 30` for anything built on `.litertlm` — inference, embeddings, speech. +9. Read [references/platform-setup.md](references/platform-setup.md) before the first build on a platform: without those entries the model fails to load or the app is killed for memory. + +## Setup — the recommended engine (.litertlm) + +```sh +flutter pub add flutter_gemma flutter_gemma_litertlm +``` + +```dart +import 'package:flutter_gemma/flutter_gemma.dart'; +import 'package:flutter_gemma_litertlm/flutter_gemma_litertlm.dart'; + +await FlutterGemma.initialize(inferenceEngines: [LiteRtLmEngine()]); + +await FlutterGemma.installModel( + modelType: ModelType.gemma4, + fileType: ModelFileType.litertlm, +) + .fromNetwork( + 'https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/gemma-4-E2B-it.litertlm', + ) + .withProgress((int percent) => print('downloading: $percent%')) + .install(); + +final InferenceModel model = await FlutterGemma.getActiveModel(maxTokens: 1024); +``` + +Gemma 4 E2B is 2.6 GB and needs no token. On web use `gemma-4-E2B-it-web.litertlm` from the same repo (2.0 GB). + +`install()` skips the download when the file is already on disk, so calling it at every launch is safe. The latest install becomes the model `getActiveModel` loads. + +A gated repo needs a token, given once: + +```dart +const hfToken = String.fromEnvironment('HUGGINGFACE_TOKEN'); + +await FlutterGemma.initialize( + inferenceEngines: [LiteRtLmEngine()], + huggingFaceToken: hfToken.isEmpty ? null : hfToken, +); +``` + +Build with `--dart-define=HUGGINGFACE_TOKEN=hf_...`. + +When a Hugging Face repo publishes a deployment manifest, one call picks the variant and its tested runtime settings. The engine carries its own resolver, so registering `LiteRtLmEngine` is enough: + +```dart +final install = await FlutterGemma.installModel( + modelType: ModelType.general, + fileType: ModelFileType.litertlm, +).fromHuggingFace('litert-community/LFM2.5-230M').install(); + +final model = await FlutterGemma.getActiveModel(defaults: install.runtime); +``` + +Other sources on the same builder: `.fromAsset(path)` for a model bundled in the app, `.fromFile(path)` for one already on disk, `.fromBundled(name)` for a platform-bundled resource. + +`modelType` tells flutter_gemma how the model writes tool calls and reasoning, and on some engines it also picks the prompt format. Gemma 3 and Gemma 3n are `ModelType.gemmaIt` — there is no `gemma3`. The full set: `ModelType.general`, `ModelType.gemmaIt`, `ModelType.gemma4`, `ModelType.deepSeek`, `ModelType.qwen`, `ModelType.qwen3`, `ModelType.llama`, `ModelType.hammer`, `ModelType.functionGemma`, `ModelType.phi`. A wrong type still generates text; tool calls and reasoning then arrive as raw text. + +## Traps + +**Core not imported** +- Symptom: `Undefined name 'FlutterGemma'`, `Undefined class 'InferenceModel'`, with only the engine package imported. +- Fix: `import 'package:flutter_gemma/flutter_gemma.dart';` as well. + +**No engine registered** +- Symptom: `StateError: No inference engine can handle this model (ModelFileType.litertlm). Add the engine package to pubspec.yaml and pass it in inferenceEngines: of FlutterGemma.initialize(...)` +- Fix: add the engine package and register its provider — or fix `fileType` if the wrong engine is registered. + +**`maxTokens` used as a reply length** + +```dart +// WRONG — asks for a 100-token context, not a 100-token reply +final model = await FlutterGemma.getActiveModel(maxTokens: 100); +``` + +- Symptom: replies are as long as ever. On native `.litertlm` the value is raised to 1024, the smallest context those models support, and only a debug-mode log says so. The web `.litertlm` engine does not take the value at all; on MediaPipe it is the real limit. +- Fix: + +```dart +final model = await FlutterGemma.getActiveModel(maxTokens: 1024); +final session = await model.createSession(maxOutputTokens: 100); +``` + +Use 4096 or more with images or audio — one image costs hundreds of tokens. + +**`isUser` left out** +- Symptom: an empty response, no error. `Message.isUser` defaults to `false`, so the prompt is read as the model's own turn. +- Fix: `Message(text: prompt, isUser: true)`. + +**Same reply every time** +- Symptom: identical output for identical input. `createSession` and `createChat` default to `topK: 1`, which is greedy decoding. +- Fix: pass `topK` (e.g. 40) and a `temperature`. Set them on the first session after `getActiveModel` — on `.litertlm` the first session's sampler settings can stay in effect for later ones. + +**`Session is closed`** +- Symptom: `StateError: Session is closed` from a session or chat that is still in use. +- Cause: `createSession` and `createChat` fill one slot per model; creating another closes the one before. +- Fix: one conversation at a time, or `openSession` / `openChat` for several (below). On the web `.litertlm` engine a second `createSession` hands back the session that is already open, history and all, rather than a fresh one — close the current chat before creating the next. + +## Generate + +```dart +final InferenceModelSession session = await model.createSession( + temperature: 0.8, + topK: 40, + maxOutputTokens: 256, +); +try { + await session.addQueryChunk(Message(text: prompt, isUser: true)); + final String reply = await session.getResponse(); +} finally { + await session.close(); +} +``` + +Streaming: + +```dart +final reply = StringBuffer(); +await session.addQueryChunk(Message(text: prompt, isUser: true)); +await for (final token in session.getResponseAsync()) { + reply.write(token); // update the UI here +} +``` + +To stop early, call `await session.stopGeneration()` — `chat.stopGeneration()` on a chat. Cancelling the stream subscription detaches Dart but does not stop native decoding on every engine. + +## Multi-turn chat + +```dart +final InferenceChat chat = await model.createChat( + systemInstruction: 'You are a concise assistant.', + temperature: 0.8, + topK: 40, + maxOutputTokens: 512, +); +try { + await chat.addQueryChunk(Message(text: prompt, isUser: true)); + final reply = StringBuffer(); + await for (final r in chat.generateChatResponseAsync()) { + switch (r) { + case TextResponse(:final token): + reply.write(token); + case ThinkingResponse() || FunctionCallResponse() || ParallelFunctionCallResponse(): + break; + } + } +} finally { + await chat.close(); +} +``` + +The chat keeps the history: add the next user message and generate again. `generateChatResponse()` returns the whole reply as one sealed `ModelResponse` — `TextResponse`, `FunctionCallResponse`, `ParallelFunctionCallResponse` or `ThinkingResponse` — and a `switch` over it must cover all four. + +## Two conversations at once + +`createSession` and `createChat` fill a single slot on the model, so a second one closes the first. For concurrent conversations use `openSession` / `openChat`, and close each one. + +They live only on the base class, so — unlike `createChat` — they inherit nothing from the installed model: pass `modelType:` (and `supportImage:` if the chat sends images) explicitly, or the chat runs as `ModelType.gemmaIt` with images off. They work on `.litertlm` (native and web) and on MediaPipe Android and iOS; everywhere else they throw `UnsupportedError`. + +```dart +final summariser = await model.openChat(modelType: ModelType.gemma4); +final assistant = await model.openChat(modelType: ModelType.gemma4); +try { + await summariser.addQueryChunk(Message(text: chunk, isUser: true)); + await assistant.addQueryChunk(Message(text: question, isUser: true)); +} finally { + await summariser.close(); + await assistant.close(); +} +``` + +## Thinking models + +Gemma 4, Qwen3 and DeepSeek R1 can emit reasoning. Pass `isThinking: true` to `createChat`. Reasoning arrives as `ThinkingResponse` only from `generateChatResponseAsync()`; `generateChatResponse()` strips it. On web Gemma 4 has no thinking; Qwen3 and DeepSeek R1 reasoning is still separated out of the text. + +```dart +final chat = await model.createChat(isThinking: true, modelType: ModelType.qwen3); +final answer = StringBuffer(); +await chat.addQueryChunk(Message(text: question, isUser: true)); +await for (final r in chat.generateChatResponseAsync()) { + switch (r) { + case ThinkingResponse(:final content): + print('reasoning: $content'); + case TextResponse(:final token): + answer.write(token); + case FunctionCallResponse() || ParallelFunctionCallResponse(): + break; + } +} +await chat.close(); +``` + +## Images + +```dart +final model = await FlutterGemma.getActiveModel(maxTokens: 4096, supportImage: true); +final chat = await model.createChat(supportImage: true); +await chat.addQueryChunk( + Message(text: 'What is in this photo?', isUser: true, imageBytes: bytes), +); +``` + +## Audio + +```dart +final model = await FlutterGemma.getActiveModel(maxTokens: 4096, supportAudio: true); +final chat = await model.createChat(supportAudio: true); +await chat.addQueryChunk( + Message(text: 'What is said in this recording?', isUser: true, audioBytes: bytes), +); +``` + +`audioBytes` is a whole WAV file — 16 kHz mono, header included. The speech package is the opposite: `transcribe` takes raw PCM with no header. Audio input needs Gemma 4 or Gemma 3n, on Android, iOS or desktop; the `.litertlm` web engine takes no audio. + +## The model is a singleton + +`getActiveModel` returns one model per process. Calling it again with different runtime arguments rebuilds it and closes the previous one — a handle still held stops working. Load it once, then create and close sessions per conversation. + +## Backends + +```dart +final model = await FlutterGemma.getActiveModel( + maxTokens: 1024, + preferredBackend: PreferredBackend.gpu, +); +print(model.activeBackend); // what actually loaded +``` + +| `preferredBackend` | Tried in order | +| --- | --- | +| `null` or `gpu` | GPU, then CPU | +| `npu` | NPU, GPU, CPU | +| `cpu` | CPU only | + +Read `activeBackend` rather than assuming the requested one loaded; the web `.litertlm` engine reports `null`. `PreferredBackend.npu` needs a Snapdragon (Android) or Intel Lunar/Panther Lake (Windows) and a model compiled for that NPU; `PreferredBackend.cpu` never falls back. The iOS Simulator is CPU-only. On web, MediaPipe is GPU-only. + +## Platform setup + +Android needs `minSdk 30` and the internet permission in release builds, and ships `arm64-v8a` only. iOS needs Podfile or Xcode settings and memory entitlements; macOS needs entitlements and a Podfile build phase; web needs script tags in `web/index.html`. Read [references/platform-setup.md](references/platform-setup.md) before building for any of them — without those entries the model fails to load or the app runs out of memory. diff --git a/packages/flutter_gemma/skills/flutter-gemma-inference/references/platform-setup.md b/packages/flutter_gemma/skills/flutter-gemma-inference/references/platform-setup.md new file mode 100644 index 000000000..3a4c6dcb4 --- /dev/null +++ b/packages/flutter_gemma/skills/flutter-gemma-inference/references/platform-setup.md @@ -0,0 +1,226 @@ +# Platform setup for flutter_gemma + +Entries each platform needs before a model will load. Without them the app +builds and then fails at model load, or is killed for memory. + +- [Android](#android) +- [iOS](#ios) +- [macOS](#macos) +- [Windows and Linux](#windows-and-linux) +- [Web](#web) + +## Android + +`android/app/build.gradle.kts` (or `build.gradle`): + +``` +android { + defaultConfig { + minSdk = 30 + } +} +``` + +`minSdk 30` covers everything built on `.litertlm`: inference, embeddings and +speech. On API 29 the native library fails to load at runtime — the build does +not catch it. MediaPipe `.task` models run on lower API levels. + +`android/app/src/main/AndroidManifest.xml` needs the internet permission to +download a model. Flutter's template declares it only for debug and profile +builds, so without this line the release build cannot download: + +```xml + +``` + +Only `arm64-v8a` is shipped for `.litertlm`. The OpenCL manifest entries the GPU +backend needs are merged in by the plugin; nothing to add. + +## iOS + +Minimum iOS 15.0 — 16.0 if the app includes `flutter_gemma_mediapipe`. + +With CocoaPods, in `ios/Podfile`, declared once: + +```ruby +platform :ios, '15.0' # '16.0' if the app includes flutter_gemma_mediapipe +use_frameworks! :linkage => :static +``` + +With Swift Package Manager — the default since Flutter 3.44 — there is no +Podfile. Set **iOS Deployment Target** on the Runner target in Xcode instead, or +the build fails with `requires minimum platform version 15.0`. +`flutter_gemma_mediapipe` has no `Package.swift`, so an app using it gets a +Podfile as well; set the platform there too. + +In Xcode, under **Signing & Capabilities**, add **Extended Virtual Addressing** +and **Increased Memory Limit**. That writes these keys to +`ios/Runner/Runner.entitlements` and links the file to the target — a file +edited by hand but not linked does nothing. Without them large models are +killed for memory: + +```xml +com.apple.developer.kernel.extended-virtual-addressing + +com.apple.developer.kernel.increased-memory-limit + +``` + +The iOS Simulator cannot run GPU inference; use CPU there, or a real device. + +## macOS + +Add to both `macos/Runner/DebugProfile.entitlements` and +`macos/Runner/Release.entitlements`: + +```xml +com.apple.security.cs.disable-library-validation + +com.apple.security.network.client + +com.apple.developer.kernel.extended-virtual-addressing + +com.apple.developer.kernel.increased-memory-limit + +``` + +`disable-library-validation` lets the app load the bundled native frameworks; +`network.client` lets it download the model; the two kernel keys keep a large +model from being killed for memory, exactly as on iOS. Add them to both files — the debug +and release builds read different ones. + +`.litertlm` on macOS also needs a build phase that copies the LiteRT-LM +companion libraries into the app: the package deliberately keeps them out of +Native Assets, so nothing else puts them in the bundle. + +With CocoaPods, paste this into `macos/Podfile`, replacing any existing +`post_install` block, then run `pod install`. + +**A Swift Package Manager app has no `macos/Podfile` to paste into.** SPM is the +default since Flutter 3.44, and an app whose plugins all ship a `Package.swift` — +core does, and `flutter_gemma_litertlm` is not a plugin at all — never gets one +generated. Either turn SPM off for the project +(`flutter config --no-enable-swift-package-manager`, then +`flutter build macos --config-only`, which writes the Podfile), or add the same +step by hand in Xcode: a Run Script phase on the Runner target named +`[flutter_gemma] Setup LiteRT-LM macOS`, carrying the `shell_script`, input path +and output path from the block below. + +```ruby +post_install do |installer| + installer.pods_project.targets.each do |target| + flutter_additional_macos_build_settings(target) + end + + # flutter_gemma: stage the upstream Apple companion dylibs into the built + # .app. `hook/build.dart` deliberately skips them from Native Assets on macOS + # (#247 — Google ships them without `-Wl,-headerpad_max_install_names`, so the + # JIT bundling path cannot rewrite their install_name), which leaves this + # build phase to stage them. + # + # The phase only LOCATES and RUNS a script; the staging logic itself lives in + # flutter_gemma_litertlm and is delivered next to the dylibs it stages. That + # is deliberate: this block is frozen into your Xcode project, and a copy of + # the logic frozen there cannot be fixed by upgrading the package. + installer.aggregate_targets.each do |aggregate_target| + aggregate_target.user_targets.each do |user_target| + phase_name = '[flutter_gemma] Setup LiteRT-LM macOS' + + # Only the app target embeds the Frameworks/ this phase patches. + # RunnerTests inherits Runner's framework search paths and has no + # Contents/Frameworks of its own — having the phase there creates a + # cross-target dependency on Runner's framework output that Xcode reports + # as "Cycle inside Flutter Assemble" (#300). Remove any stale copy from + # non-app targets and skip them. + unless user_target.name == 'Runner' + user_target.build_phases + .select { |p| p.respond_to?(:name) && p.name == phase_name } + .each { |p| user_target.build_phases.delete(p) } + next + end + + existing = user_target.shell_script_build_phases.find { |p| p.name == phase_name } + phase = existing || user_target.new_shell_script_build_phase(phase_name) + # The embedded LiteRtLm binary is an INPUT so the phase re-runs whenever + # Flutter's always-out-of-date `embed` phase re-copies the raw, unpatched + # binary over the patched one. Without it Xcode caches the phase after the + # first build and the second incremental build ships an unpatched + # LiteRtLm that fails dlopen at runtime (#368). + phase.input_paths = [ + '$(BUILT_PRODUCTS_DIR)/$(PRODUCT_NAME).app/Contents/Frameworks/LiteRtLm.framework/Versions/A/LiteRtLm', + ] + # A declared output lets Xcode order the phase in its dependency graph + # instead of treating it as "runs every build with no outputs" — the other + # half of the cycle warning (#300). The script touches this file. + phase.output_paths = ['$(DERIVED_FILE_DIR)/flutter_gemma_litertlm_macos.stamp'] + phase.shell_script = <<~SHELL + set -e + STAGER="${HOME}/Library/Caches/flutter_gemma/native/macos_arm64/stage_macos_companions.sh" + if [ ! -f "${STAGER}" ]; then + echo "[flutter_gemma] ERROR: ${STAGER} not found." >&2 + echo " flutter_gemma_litertlm 1.6.2+ installs it there from its build hook." >&2 + echo " Upgrade the package, then: flutter clean && flutter pub get" >&2 + exit 1 + fi + sh "${STAGER}" "${BUILT_PRODUCTS_DIR}/${PRODUCT_NAME}.app/Contents/Frameworks" + mkdir -p "$(dirname "${SCRIPT_OUTPUT_FILE_0}")" + touch "${SCRIPT_OUTPUT_FILE_0}" + SHELL + end + end +end +``` + +Without it the build succeeds and the model fails to load at runtime. + +## Windows and Linux + +Nothing to add to the project. The native libraries — including the Windows GPU +shader compiler and NPU runtime — are bundled at build time. + +- Windows: end users need the Microsoft Visual C++ Redistributable 2019 or later. +- Linux: building needs `clang cmake ninja-build libgtk-3-dev lld`. GPU needs the + vendor Vulkan driver; Mesa's `llvmpipe` software fallback cannot run Gemma 4. + +## Web + +All script tags go in `web/index.html` ``, before Flutter boots. + +`.litertlm` engine: + +```html + +``` + +Model storage helpers. Copy `cache_api.js` and `opfs_helper.js` from the +`flutter_gemma` package's `web/` directory into the app's `web/`, then: + +```html + + +``` + +Find the package directory with +`grep -A1 '"name": "flutter_gemma"' .dart_tool/package_config.json`. + +Storage mode, set in `FlutterGemma.initialize(webStorageMode: ...)`: + +| `WebStorageMode` | Use for | +| --- | --- | +| `cacheApi` (default) | models under about 2 GB | +| `streaming` | larger models — streams through OPFS | +| `none` | no persistence; downloads every launch | + +The `.litertlm` web engine loads the web build of a model — +`gemma-4-E2B-it-web.litertlm` (2.0 GB, so use `streaming`), not +`gemma-4-E2B-it.litertlm`. It is text-only: no images, audio or LoRA, and no +Gemma 4 thinking. + +A `--dart-define` token is compiled into `main.dart.js`, where every visitor can +read it. Serve web users a model from a repo that needs no token. diff --git a/packages/flutter_gemma/skills/flutter-gemma-mediapipe/SKILL.md b/packages/flutter_gemma/skills/flutter-gemma-mediapipe/SKILL.md new file mode 100644 index 000000000..edbbbbe26 --- /dev/null +++ b/packages/flutter_gemma/skills/flutter-gemma-mediapipe/SKILL.md @@ -0,0 +1,112 @@ +--- +name: flutter-gemma-mediapipe +description: Use when running .task or .bin models (MediaPipe GenAI, ModelFileType.task or ModelFileType.binary) with flutter_gemma_mediapipe on Android, iOS or web. Also use when CocoaPods rejects the iOS platform version, images are ignored in a MediaPipe chat, or maxOutputTokens has no effect. MediaPipe has no macOS, Windows or Linux support — use a .litertlm model there (flutter-gemma-inference). +--- + +# The MediaPipe engine + +## Rules + +1. Depend on `flutter_gemma` and `flutter_gemma_mediapipe`, and import both. The engine package does not re-export core. +2. Declare `fileType: ModelFileType.task` for `.task` files and `ModelFileType.binary` for `.bin` files. +3. An app that includes this package needs iOS 16.0. +4. There is no desktop support. +5. `maxTokens` is the real context limit — small values are not raised as they are on `.litertlm`. `maxOutputTokens` is ignored; stop generation with `session.stopGeneration()`. +6. On Android and iOS, `createChat` inherits the model's audio support but **not** its image support — pass `supportImage: true` to the chat as well, or the image is dropped. + +## Setup + +```sh +flutter pub add flutter_gemma flutter_gemma_mediapipe +``` + +```dart +import 'package:flutter_gemma/flutter_gemma.dart'; +import 'package:flutter_gemma_mediapipe/flutter_gemma_mediapipe.dart'; + +await FlutterGemma.initialize(inferenceEngines: [MediaPipeEngine()]); + +await FlutterGemma.installModel( + modelType: ModelType.gemmaIt, + fileType: ModelFileType.task, +).fromNetwork(url).install(); + +final InferenceModel model = await FlutterGemma.getActiveModel(maxTokens: 1024); +``` + +Sessions, chats, streaming and the common traps work as in the flutter-gemma-inference skill — except that `openSession` / `openChat` (its concurrent-conversation pattern) work on Android and iOS only; on web they throw `UnsupportedError`. + +## iOS + +`ios/Podfile`, declared once: + +```ruby +platform :ios, '16.0' +use_frameworks! :linkage => :static +``` + +With Swift Package Manager, also set **iOS Deployment Target** to 16.0 on the Runner target in Xcode. This package ships no Swift package manifest, so the app gets an `ios/Podfile` either way. + +Core and the other engines build from iOS 15.0. If the app does not use `.task` models, leave this package out and stay on 15. + +Large models also need **Extended Virtual Addressing** and **Increased Memory Limit**, added in Xcode under **Signing & Capabilities**, or the app is killed for memory. + +## Images and audio + +```dart +final model = await FlutterGemma.getActiveModel(maxTokens: 4096, supportImage: true); +final chat = await model.createChat(supportImage: true); +await chat.addQueryChunk( + Message(text: 'Describe this image.', isUser: true, imageBytes: bytes), +); +``` + +On Android and iOS, a chat without `supportImage: true` drops the image and the model answers the text alone. On web the chat follows the model: an image sent to a model loaded without `supportImage: true` throws `ArgumentError`. + +Audio input works on Android and iOS with a model that takes audio, such as Gemma 3n. + +## Bounding output + +```dart +final session = await model.createSession(); +await session.addQueryChunk(Message(text: prompt, isUser: true)); +final reply = StringBuffer(); +var produced = 0; +await for (final token in session.getResponseAsync()) { + reply.write(token); + if (++produced >= 200) { + await session.stopGeneration(); + break; + } +} +await session.close(); +``` + +## Web + +Add to `web/index.html` ``, before Flutter boots: + +```html + + + +``` + +Pin the version — an unpinned import takes whatever was published last. Copy `cache_api.js` and `opfs_helper.js` from the `flutter_gemma` package's `web/` directory into the app's `web/`; find it with `grep -A1 '"name": "flutter_gemma"' .dart_tool/package_config.json`. + +Web is GPU-only. Models over about 2 GB need OPFS streaming storage: + +```dart +await FlutterGemma.initialize( + webStorageMode: WebStorageMode.streaming, + inferenceEngines: [MediaPipeEngine()], +); +``` + +## Android + +Text inference runs on `arm64-v8a`, `x86_64` and `armeabi-v7a`. The release build needs `` in `android/app/src/main/AndroidManifest.xml` to download a model — Flutter's template declares it only for debug and profile builds. diff --git a/packages/flutter_gemma/skills/flutter-gemma-onnx/SKILL.md b/packages/flutter_gemma/skills/flutter-gemma-onnx/SKILL.md new file mode 100644 index 000000000..a2f308ee3 --- /dev/null +++ b/packages/flutter_gemma/skills/flutter-gemma-onnx/SKILL.md @@ -0,0 +1,88 @@ +--- +name: flutter-gemma-onnx +description: Use when running ONNX models with flutter_gemma_onnx (ModelFileType.onnx) — ORT-GenAI text generation (e.g. Phi-3.5-mini) or ONNX embeddings — on macOS arm64, Linux x64, Windows x64, Android arm64, iOS arm64, or in the browser through Transformers.js. Also use when an ONNX install is routed to the wrong engine, genai_config.json is missing, or getActiveModel throws "No inference engine can handle this model" on another platform. For .litertlm models use flutter-gemma-inference; for .task, flutter-gemma-mediapipe. +--- + +# The ONNX engine + +## Rules + +1. Depend on `flutter_gemma` and `flutter_gemma_onnx`, and import both. The engine package does not re-export core. +2. Declare `fileType: ModelFileType.onnx`. Without it the install defaults to `task` and never reaches `OnnxEngine`. +3. An ORT-GenAI model is a directory — `genai_config.json`, the `.onnx` graph, its weights and a tokenizer. Install it with `fromHuggingFace(repo)`, which downloads the whole folder, or point `fromFile` at a local `genai_config.json`. A single-file download or a Flutter asset cannot produce it. +4. Native generation runs on macOS arm64, Linux x64, Windows x64, Android arm64 and iOS arm64. On any other native host no engine accepts the model and `getActiveModel` throws `No inference engine can handle this model`. Web is a separate arm with its own rules (below). +5. Android needs `minSdk 24` — the build hook fails the build below it. Phi-3.5-mini peaks near 3.7 GB of RAM, so target 8 GB devices. +6. Text only: no images, no audio, no LoRA. + +## Setup + +```sh +flutter pub add flutter_gemma flutter_gemma_onnx +``` + +```dart +import 'package:flutter_gemma/flutter_gemma.dart'; +import 'package:flutter_gemma_onnx/flutter_gemma_onnx.dart'; + +await FlutterGemma.initialize( + inferenceEngines: [OnnxEngine()], + embeddingBackends: [OnnxEmbeddingBackend()], +); + +await FlutterGemma.installModel( + modelType: ModelType.phi, + fileType: ModelFileType.onnx, +).fromHuggingFace('microsoft/Phi-3.5-mini-instruct-onnx').install(); + +final InferenceModel model = await FlutterGemma.getActiveModel(maxTokens: 4096); +``` + +A repo with several execution-provider folders resolves to its CPU/mobile folder automatically — the bundled runtime is CPU-only. + +A bundle shipped with the app: + +```dart +await FlutterGemma.installModel( + modelType: ModelType.phi, + fileType: ModelFileType.onnx, +).fromFile('$path/genai_config.json').install(); +``` + +Sessions, chats and streaming work as in the flutter-gemma-inference skill, except `openSession` / `openChat`, which throw `UnsupportedError` here — one conversation at a time. Pass `modelType` to `createChat` for function calling; ONNX falls back to `ModelType.gemmaIt`. + +## Web + +On web `OnnxEngine` runs the model through Transformers.js. Install it by Hugging Face repo URL; the browser downloads and caches the files on first use: + +```dart +await FlutterGemma.installModel( + modelType: ModelType.qwen, + fileType: ModelFileType.onnx, +).fromNetwork('https://huggingface.co/onnx-community/Qwen2.5-0.5B-Instruct').install(); +``` + +The repo must be in Transformers.js layout, as the `onnx-community` ones are. ORT-GenAI repos such as `microsoft/Phi-3.5-mini-instruct-onnx` do not run in the browser. `fromFile` and `fromAsset` install on web without complaint and then throw `UnsupportedError` from the first `createSession` — on web the model identity has to be a repo, not a file. `fromBundled('')` serves one from the app's own origin. `PreferredBackend.cpu` forces WASM; anything else tries WebGPU first. + +Add to `web/index.html` ``, before Flutter boots — the first script for generation, the second for embeddings: + +```html + + +``` + +## Embeddings + +`OnnxEmbeddingBackend` handles single-file `.onnx` or `.ort` embedding models, installed with `FlutterGemma.installEmbedder()` like any other — see the flutter-gemma-rag skill for the indexing flow. diff --git a/packages/flutter_gemma/skills/flutter-gemma-rag/SKILL.md b/packages/flutter_gemma/skills/flutter-gemma-rag/SKILL.md new file mode 100644 index 000000000..38c440d9a --- /dev/null +++ b/packages/flutter_gemma/skills/flutter-gemma-rag/SKILL.md @@ -0,0 +1,135 @@ +--- +name: flutter-gemma-rag +description: Use when adding RAG, semantic search or text embeddings to a flutter_gemma app — searching the user's documents on-device, an embedding model plus a vector store (flutter_gemma_rag_sqlite or flutter_gemma_rag_qdrant). Also use when a metadata filter returns unfiltered results, retrieval quality is poor, addDocument throws about a missing embedding model, the vector store fails to open on a phone, or it throws UnimplementedError on web. +--- + +# On-device RAG with flutter_gemma + +## Rules + +1. Use the `FlutterGemma.rag` facade: `initialize`, `addDocument`, `searchSimilar`. It embeds documents and queries with the correct task types. +2. Declare every field used in a filter in `filterSchema:` at `initialize`. A condition on an undeclared field is dropped, never rejected: with no schema at all the search comes back completely unfiltered, and a filter that mixes declared and undeclared fields narrows only by the declared ones. +3. On native, give `rag.initialize` an absolute path in a writable directory. A bare name resolves against the process working directory, which is not writable on Android or iOS. +4. Activate an embedding model with `getActiveEmbedder()` before `addDocument`. +5. `LiteRtEmbeddingBackend` comes from `flutter_gemma_litertlm`, not `flutter_gemma_embeddings`. +6. On web use `WebSqliteVectorStore`. `SqliteVectorStore` constructs there without complaint and throws `UnimplementedError` from the first call — `configure()` is a silent no-op, so the mistake shows up as a failed search, not a failed setup. `flutter_gemma_rag_qdrant` is native-only. +7. Android needs `minSdk 30` — the LiteRT embedding runtime, not the vector store. The rest of the build setup is the flutter-gemma-inference skill's [platform setup](../flutter-gemma-inference/references/platform-setup.md). + +## Setup + +```sh +flutter pub add flutter_gemma flutter_gemma_litertlm flutter_gemma_rag_sqlite path_provider +``` + +```dart +import 'package:flutter/foundation.dart'; +import 'package:flutter_gemma/flutter_gemma.dart'; +import 'package:flutter_gemma_litertlm/flutter_gemma_litertlm.dart'; +import 'package:flutter_gemma_rag_sqlite/flutter_gemma_rag_sqlite.dart'; +import 'package:path_provider/path_provider.dart'; + +const hfToken = String.fromEnvironment('HUGGINGFACE_TOKEN'); +const embeddingGemma = + 'https://huggingface.co/litert-community/embeddinggemma-300m/resolve/main'; + +await FlutterGemma.initialize( + embeddingBackends: [LiteRtEmbeddingBackend()], + vectorStore: kIsWeb ? WebSqliteVectorStore() : SqliteVectorStore(), + filterSchema: const FilterSchema(fields: [ + FilterField(name: 'lang', type: FilterFieldType.string), + FilterField(name: 'year', type: FilterFieldType.number), + ]), + huggingFaceToken: hfToken.isEmpty ? null : hfToken, +); + +await FlutterGemma.installEmbedder() + .modelFromNetwork('$embeddingGemma/embeddinggemma-300M_seq512_mixed-precision.tflite') + .tokenizerFromNetwork('$embeddingGemma/sentencepiece.model') + .install(); +final EmbeddingModel embedder = await FlutterGemma.getActiveEmbedder(); + +await FlutterGemma.rag.initialize( + kIsWeb ? 'rag.db' : '${(await getApplicationDocumentsDirectory()).path}/rag.db', +); +``` + +EmbeddingGemma is a gated repo: the token's Hugging Face account must have accepted the Gemma licence, and the token ships inside the app — on web inside `main.dart.js`. `seq512` in the file name is the input window in tokens; `seq256`, `seq1024` and `seq2048` variants sit in the same repo. + +`rag.initialize` takes a database file for sqlite and a directory for qdrant. On native it persists across launches at that path. Add `inferenceEngines:` from the flutter-gemma-inference skill when the app also generates answers from the results. + +## Index and search + +```dart +import 'dart:convert'; + +await FlutterGemma.rag.addDocument( + id: 'doc-1', + content: chunk, + metadata: jsonEncode({'lang': 'en', 'year': 2024}), +); + +final List hits = await FlutterGemma.rag.searchSimilar( + query: question, + topK: 5, + filter: const Filter( + must: [FieldEquals(key: 'lang', value: 'en')], + mustNot: [FieldRange(key: 'year', lte: 2010)], + ), +); +for (final hit in hits) { + print('${hit.id} ${hit.similarity.toStringAsFixed(2)} ${hit.content}'); +} +``` + +`searchSimilar` takes the question as text and embeds it itself. Each `RetrievalResult` has `id`, `content`, `similarity` and `metadata`. Filter operators: `FieldEquals`, `FieldRange` (`gte`, `lte`), `FieldMatchAny`, combined with `must`, `should` and `mustNot`. + +`addDocument` with an existing `id` replaces that document. `FlutterGemma.rag.removeDocument(id:)` deletes one; `FlutterGemma.rag.clear()` empties the store. + +## Traps + +**Filter has no effect** +- Symptom: results ignore the filter; no error. +- Fix: declare the field in `filterSchema`. With `flutter_gemma_rag_sqlite`, names must match `^[A-Za-z][A-Za-z0-9_]*$` and cannot be `id`, `embedding`, `content`, `metadata`, `distance` or `k`. Its `vec0` table also caps declared metadata columns at 16; nothing checks that at `initialize`, so a 17th field surfaces when the table is created, on the first `addDocument`. + +**Poor retrieval after embedding by hand** +- Query and document embeddings are trained asymmetrically. `generateEmbedding` defaults to `TaskType.retrievalQuery`, so text embedded for indexing without a task type gets the query prefix. +- Fix: pass `TaskType.retrievalDocument` when indexing by hand: + +```dart +final vector = await embedder.generateEmbedding( + chunk, + taskType: TaskType.retrievalDocument, +); +await FlutterGemma.rag.addDocumentWithEmbedding( + id: 'doc-2', + content: chunk, + embedding: vector, +); +``` + +**`addDocument` throws** +- Symptom: `No embedding model is active. addDocument(content:) and searchSimilar(query:) auto-embed text, which requires an embedding model.` +- Fix: install an embedder and call `FlutterGemma.getActiveEmbedder()` first. + +**Store fails to open on a phone** +- Cause: a bare name such as `'rag.db'` passed to `rag.initialize` on Android or iOS. +- Fix: an absolute path under `getApplicationDocumentsDirectory()`, as in Setup. + +## Backend + +LiteRT embeddings always run on CPU. `LiteRtEmbeddingBackend` hardcodes it and ignores `getActiveEmbedder(preferredBackend:)` entirely — passing `PreferredBackend.gpu` there changes nothing. That is deliberate: the GPU delegate compiles and then returns all-zero vectors for EmbeddingGemma. + +## Web + +- Copy `web/rag/sqlite3.wasm` from the `flutter_gemma_rag_sqlite` package into the app as `web/rag/sqlite3.wasm`. +- Web embeddings need four module files side by side in the app's `web/`: `litert_embeddings.js` and `sentencepiece.js` from `flutter_gemma_embeddings/web/`, plus `litert.js` and `tensorflow.js` from `flutter_gemma_litertlm/web/` — the first one imports the other three by relative path, so three files alone give a 404 and an embedder that never initialises. +- They also need the LiteRT WASM runtime at `web/wasm/`, which no package ships: build it once from the core package (`cd /web/rag && npm install && npm run build`) and copy `dist/wasm` into the app's `web/`. The Dart side loads it from `/wasm/` and nowhere else. +- In `web/index.html`, before Flutter boots: `` first — it is not a module, and the embedding runtime calls its cache helpers during init — then ``. + +Find a package's directory with `grep -A1 '"name": "flutter_gemma_rag_sqlite"' .dart_tool/package_config.json`. + +## Chunking + +Splitting documents, chunk size and overlap are the app's to decide. Keep each chunk within the embedding model's window — 512 tokens for `seq512`; a longer one is truncated without an error. + +For ONNX embedding models see the flutter-gemma-onnx skill. diff --git a/packages/flutter_gemma/skills/flutter-gemma-speech/SKILL.md b/packages/flutter_gemma/skills/flutter-gemma-speech/SKILL.md new file mode 100644 index 000000000..e849f9017 --- /dev/null +++ b/packages/flutter_gemma/skills/flutter-gemma-speech/SKILL.md @@ -0,0 +1,184 @@ +--- +name: flutter-gemma-speech +description: Use when adding speech to a flutter_gemma app — speech-to-text (transcribe a voice note, dictation, Whisper, moonshine, Parakeet), text-to-speech (Matcha, Qwen3-TTS, Inflect), or a push-to-talk voice assistant with VoiceSession. Also use when transcripts come back in English for non-English audio, a WAV file has to become 16 kHz PCM, synthesized audio plays at the wrong pitch, or getActiveTts throws a StateError about the language. For audio sent straight to Gemma in a chat, use flutter-gemma-inference. +--- + +# Speech with flutter_gemma_speech + +## Rules + +1. Depend on `flutter_gemma` and `flutter_gemma_speech`, and import both. The speech package does not re-export core. +2. `transcribe` takes raw PCM — 16 kHz, mono, 16-bit little-endian, as a `Uint8List` — and returns the text. Not a WAV file, not 44.1 or 48 kHz: nothing resamples or converts it. +3. Play synthesized audio at `synth.sampleRate`. It differs per model. +4. Only Whisper has a selectable output language. moonshine-tiny and Parakeet are English-only, and passing a language to them throws `ArgumentError`. +5. STT language: set a default with `getActiveStt(language:)` or override one call with `transcribe(pcm, language:)`. Nothing reloads. +6. TTS language: `close()` the synthesizer first. Asking a live synthesizer for another language throws `StateError`. +7. Android needs `minSdk 30`. There is no web support — the web backends throw `UnsupportedError`. +8. Close recognizers and synthesizers. + +## Setup + +```sh +flutter pub add flutter_gemma flutter_gemma_speech +``` + +```dart +import 'package:flutter_gemma/flutter_gemma.dart'; +import 'package:flutter_gemma_speech/flutter_gemma_speech.dart'; + +await FlutterGemma.initialize( + sttBackends: [LiteRtSttBackend()], + ttsBackends: [LiteRtTtsBackend()], +); +``` + +Speech runs on the same native libraries as the `.litertlm` engine. Its build setup — Android `minSdk 30`, the Apple entries — is in the flutter-gemma-inference skill's [platform setup](../flutter-gemma-inference/references/platform-setup.md), installed alongside this one. + +## Speech-to-text + +An STT model is two files — the model and its tokenizer — usually from different repos. `install()` skips files already on disk. + +```dart +await FlutterGemma.installStt() + .modelFromNetwork('https://huggingface.co/litert-community/whisper-tiny/resolve/main/whisper_tiny_30s_f32.tflite') + .tokenizerFromNetwork('https://huggingface.co/openai/whisper-tiny/resolve/main/tokenizer.json') + .ofType(SttModelType.whisper) + .install(); + +final SpeechRecognizer recognizer = await FlutterGemma.getActiveStt(language: 'de'); +try { + final String german = await recognizer.transcribe(germanPcm); + final String french = await recognizer.transcribe(frenchPcm, language: 'fr'); +} finally { + await recognizer.close(); +} +``` + +| `SttModelType` | Languages | Window | +| --- | --- | --- | +| `SttModelType.moonshine` | English | 5 s | +| `SttModelType.whisper` | 99, selectable, default `'en'` | 30 s | +| `SttModelType.parakeet` | English | 5 s; 2.35 GB, so desktop in practice — nothing refuses it on a phone | + +**Audio longer than the window is silently truncated**, not rejected: it is zero-padded when shorter and cut when longer, so a 40-second clip on Whisper returns the first 30 seconds with no error. Split long recordings yourself. + +Whisper tiny is weak outside English. Whisper base int8 is the next size up in the catalog; install it the same way from `https://huggingface.co/litert-community/whisper-base/resolve/main/whisper_base_30s_i8.tflite` with the tokenizer `https://huggingface.co/openai/whisper-base/resolve/main/tokenizer.json`. + +## Getting 16 kHz mono PCM + +The package has no resampler and no WAV reader. Record in the right format from the start — with the `record` package: + +```dart +import 'package:record/record.dart'; + +const config = RecordConfig( + encoder: AudioEncoder.wav, + sampleRate: 16000, + numChannels: 1, +); +``` + +That produces a WAV file. Its header is not always 44 bytes — take the samples from the `data` chunk: + +```dart +import 'dart:typed_data'; + +/// The samples of a 16 kHz mono 16-bit WAV file, without its header. +Uint8List pcmFromWav(Uint8List wav) { + final view = ByteData.sublistView(wav); + var offset = 12; // after 'RIFF', the size and 'WAVE' + while (offset + 8 <= wav.length) { + final id = String.fromCharCodes(wav, offset, offset + 4); + final size = view.getUint32(offset + 4, Endian.little); + final start = offset + 8; + if (id == 'data') { + final end = start + size > wav.length ? wav.length : start + size; + return Uint8List.sublistView(wav, start, end); + } + offset = start + size + (size & 1); // chunks are padded to an even size + } + throw const FormatException('WAV file has no data chunk'); +} +``` + +A file recorded at another rate or channel count — 44.1 kHz stereo, say — has to be converted first: average the channels to mono, then resample with a low-pass filter. Dropping samples instead aliases and costs accuracy. + +## Traps + +**Transcript comes back in English** +- Symptom: German audio, fluent English text, no error. +- Cause: Whisper's language token decides the output language, not what it understands — with `'en'` it translates. moonshine only ever produces English. +- Fix: use Whisper and pass `language:`. + +**A language is rejected** +- Whisper codes are bare and lowercase: `'de'`, not `'de-DE'`, `'DE'` or `'german'`. Malformed codes throw `ArgumentError` from `getActiveStt`. +- A well-formed code the installed checkpoint lacks (e.g. `'zz'`) throws `ArgumentError` from `transcribe`. + +## Text-to-speech + +```dart +await FlutterGemma.installTts() + .fromNetwork('https://huggingface.co/litert-community/Matcha-TTS/resolve/main/') + .ofType(TtsModelType.matcha) + .install(); + +final SpeechSynthesizer synth = await FlutterGemma.getActiveTts(); +try { + final audio = await synth.synthesize('Hello world.'); // 16-bit PCM + final rate = synth.sampleRate; // 22050 for Matcha +} finally { + await synth.close(); +} +``` + +| `TtsModelType` | Languages | +| --- | --- | +| `TtsModelType.matcha` | English, fixed by the installed bundle — it ignores `language:` | +| `TtsModelType.qwen3` | `chinese`, `english`, `german`, `italian`, `portuguese`, `spanish`, `japanese`, `korean`, `french`, `russian`, or `auto` | +| `TtsModelType.inflect` | English | + +`TtsModelType.supertonic` and `TtsModelType.kokoro` are in the enum but throw `UnimplementedError` — do not use them. + +Switching the Qwen3 language — full lowercase names, not ISO codes, and only with the Qwen3 bundle installed (Matcha still throws the same `StateError` but the language changes nothing): + +```dart +final english = await FlutterGemma.getActiveTts(language: 'english'); +await english.close(); +final german = await FlutterGemma.getActiveTts(language: 'german'); +``` + +Without the `close()`, the second call throws `StateError: Active TTS synthesizer was created for language 'english'; call close() before requesting 'german'.` + +## Voice assistant + +`VoiceSession` runs one push-to-talk turn: transcribe, generate, speak, with barge-in. It uses the recognizer's current language. + +Wrap the loop in `try`/`catch`: a failed stage — transcribe, generate or synthesize — arrives as a **stream error**, not as an event. `VoiceErrorEvent` is reserved in this release and never emitted; the `case` is only there because the switch must be exhaustive. To barge in, call `await voice.interrupt()` — cancelling the subscription is not a portable stop. + +```dart +final reply = StringBuffer(); +final voice = VoiceSession.fromChat( + recognizer: await FlutterGemma.getActiveStt(language: 'de'), + chat: chat, + synthesizer: await FlutterGemma.getActiveTts(), +); + +await for (final event in voice.runTurn(pcm16kMono)) { + switch (event) { + case VoiceTranscriptEvent(:final text): + print('heard: $text'); + case VoiceReplyTextEvent(:final chunk): + reply.write(chunk); + case VoiceReplyAudioEvent(:final sampleRate): + print('audio at $sampleRate Hz'); + case VoiceTurnInterruptedEvent(): + print('interrupted — stop the player'); + case VoiceTurnCompleteEvent(): + print('done'); + case VoiceErrorEvent(:final error): + print('failed: $error'); + } +} +``` + +`chat` is an `InferenceChat` from the flutter-gemma-inference skill. A chat created with tools also needs `onToolCall:` — without it `fromChat` throws. diff --git a/packages/flutter_gemma_agent/README.md b/packages/flutter_gemma_agent/README.md index 685cf17a7..a4864fe9e 100644 --- a/packages/flutter_gemma_agent/README.md +++ b/packages/flutter_gemma_agent/README.md @@ -70,6 +70,11 @@ WebView), verified on hardware. On web the skill runs in a sandboxed `