feat: ship agent skills for consumers of the package - #505
Open
DenisovAV wants to merge 8 commits into
Open
Conversation
The Dart team's `skills` CLI (labs.dart.dev, 1.0.1) lets a package carry
instruction files that a user's coding assistant installs with
`dart run skills@ get`. Three skills, aimed at the mistakes that actually
generate issues here:
flutter-gemma-setup core registers no engine; installModel
defaults fileType to `task`, so a
`.litertlm` file is routed to MediaPipe
unless declared
flutter-gemma-inference maxTokens is the CONTEXT WINDOW, not the
reply length (#318 — below 1024 it crashes
`.litertlm`, it does not truncate); use
maxOutputTokens to cap output;
Message.isUser defaults to false and a
missing `isUser: true` returns an empty
string with no error
flutter-gemma-speech-stt 16 kHz mono PCM in; the Whisper output
language is a property of a transcription,
not of the loaded model
Our own `.claude/skills/` (release, build-native, review-pr, ci-status,
upgrade-genkit) stay where they are — those are maintainer tools, and
shipping instructions for rebuilding our dylibs to every consumer would be
noise.
The distribution half is the part that would have failed silently.
`packages/flutter_gemma/.pubignore` carries a blanket `**/*.md` with only
README and CHANGELOG exempted, so every SKILL.md was stripped from the
archive. `dart run skills@ get` resolves the package on disk from
package_config.json and reads `skills/` from there, so the skills would
have reached nobody, with no error at any point. Measured before and after:
`dart pub publish --dry-run | grep -c SKILL.md` was 0 for core and 1 for
speech (which has no such rule); it is 2 and 1 now.
Verified end to end, not by reading the spec: ran `dart run skills@ get
--all --agent claude` from the example app and all three installed, then
`skills@ remove --all` to clean up.
Versions bumped because SKILL.md ships inside the archive — without a
release the files exist only in the repo. Both READMEs now say the command
exists, since a mechanism nobody knows about is not a mechanism.
Reworked after discussion. Three changes from the first attempt.
Split by TASK, not by API area. A single request — "add on-device chat"
— needs the engine registration, the fileType declaration, the maxTokens
floor and the isUser default all at once. Separate `setup` and
`inference` skills risked the agent loading one and writing half-correct
code, so they are one skill now.
All eight live in `flutter_gemma`, not in the packages they describe.
Per-package placement is what the naming convention is built for, but it
buys one release path per skill: a typo in the litertlm skill would mean
publishing litertlm. One home means one publish, and the CLI is
interactive, so a MediaPipe-only user is offered the litertlm skill and
simply declines it.
Coverage now spans the engines rather than stopping at core:
flutter-gemma-inference registry, fileType routing, maxTokens vs
maxOutputTokens, isUser, lifecycle
flutter-gemma-tools function calling — supportsFunctionCalls
AND modelType are both required
flutter-gemma-litertlm kv_cache 1024 floor, minSdk 30, backends
flutter-gemma-mediapipe iOS 16 floor, web script tags, no desktop
flutter-gemma-onnx the model is a DIRECTORY, five host arches
flutter-gemma-builtin-ai no file to install, availability probing
flutter-gemma-speech STT + TTS + VoiceSession
flutter-gemma-rag TaskType asymmetry, CPU-only embedding
`flutter_gemma_speech` drops out of this release entirely — its skill
moved to core, so its version and README are back to what main has.
Verified by running the CLI from the example app, not by reading the
spec: all eight install, then `skills@ remove --all` cleans up. Archive
carries exactly eight SKILL.md.
Skills are read by a machine. A stale README makes a human squint; a stale SKILL.md makes their agent write confident code against an API that moved, and nothing in the build notices — markdown keeps analyze, test and format green. Two additions. `tool/check_skills.sh` extracts every API symbol the skills name and fails if one is gone from packages/*/lib/. It reads three shapes, because between them they cover where names actually appear: inline backticks, named arguments inside ```dart fences, and dotted members inside them. The first shape alone is not enough — it misses maxOutputTokens entirely, which the skills mention only inside code blocks. Currently 135 symbols across eight skills, zero missing. Mutation-checked rather than assumed: renaming maxOutputTokens, supportsFunctionCalls, TaskType.retrievalQuery, BuiltInAi.availability and Message.toolResponse each turns it red. It also exits 2 rather than 0 when it extracts nothing, so a check that examined nothing cannot pass for a clean one — that failure mode is why the first draft of this script looked fine while checking almost nothing. Release skill gains step 12d plus a Definition-of-Done line: map the change to the skill that covers it, then run the gate. The section says plainly what the gate cannot do — catch a symbol that still exists but changed meaning. getActiveStt(language:) went from a load-time property to a per-transcription one with no rename at all, and the script would have stayed green through it. Step 1d-bis needs no change: its repo-wide grep already covers packages/**/*.md, which is where the skills live. Verified rather than assumed.
A gate that checks names is not enough, and "read the skills carefully" in a checklist is not a step anyone performs. The missing piece was deciding WHICH skills a given release puts in doubt. tool/skills_review.sh takes the release range and, per skill, prints the API symbols that skill NAMES and this diff TOUCHED — added or removed lines only, so a symbol merely sitting near a change is not evidence. A skill with hits gets opened; a skill without gets skipped deliberately rather than forgotten. Validated against two real releases rather than reasoned about: 01801ef (STT language) flags flutter-gemma-speech with getActiveStt, language, SttModelType.whisper — exactly the skill whose prose the change invalidated a764009 (createChat flags flutter-gemma-tools, and leaves speech never passed tools) alone First run flagged 5 of 8 for an STT-only change, because String, StateError, name, text and must appear in every diff. Those carry no routing power, so they are filtered; it is 2 of 8 now. Flagging everything is the same as flagging nothing. Release 12d now reads: run the router, READ every flagged skill against the change, then run check_skills.sh. The section says which of the two scripts answers which question — names versus meaning — and that only reading closes the second.
Four independent reviews of the eight drafts found code that would not compile or would fail at runtime in five of them: ModelType.gemma3 (no such value), a switch over the sealed ModelResponse without ThinkingResponse, rag.search(vector, limit:) and Condition.equals (the API is searchSimilar(query:, topK:) and FieldEquals), getActiveModel(spec:) (no such parameter), LiteRtEmbeddingBackend attributed to the wrong package, SqliteVectorStore on web (throws), filters without filterSchema (silently ignored), and ONNX installs without fileType (routed to MediaPipe). Plus wrong claims: an explicit gpu/npu backend does fall back, .litertlm clamps rather than crashes below 1024, ONNX and built-in AI do have web arms, moonshine-tiny is English-only. Every item was checked against the source before it was changed. Seven skills now: flutter-gemma-litertlm folded into inference, because litertlm is the default engine and "add offline chat" names no engine, so the default path must be self-contained; flutter-gemma-tools renamed to function-calling. Platform setup (Podfile, entitlements, web/index.html) moved to inference/references/platform-setup.md. tool/check_skills.dart replaces the grep gate. It compiles every dart fence as a function body and every inline Type / Type.member as a declaration, then runs dart analyze inside the example app. The grep gate was green on four nonexistent APIs — gemma3 matched a model URL, limit: an unrelated argument. Verified both ways: 29 errors on the broken drafts; clean on the rewrite, and six mutations of the new text each turn it red. Also: CLAUDE.md said built-in AI and ONNX have no web (the source the drafts copied), and website models.md repeated the moonshine claim. One claim was removed rather than fixed: platform-setup.md said a Release build fails without disable-library-validation. The README asks for it in both files, but the example has it in Debug only; that needs a macOS release build to settle and is recorded, not asserted.
…setup The compile gate injected `package:flutter_gemma/flutter_gemma.dart` into every generated file, and none of the seven skills imported it. The engine packages do not re-export core, so every example failed with "Undefined name 'FlutterGemma'" while the gate stayed green. An agent given only the skills found it. Without the injected import the gate reports 161 errors on the previous text and none on this one. - every setup block shows the core import and a `flutter pub add` line - platform-setup: macOS post_install build phase, Android INTERNET for release builds, SPM deployment target, Xcode capabilities instead of a loose entitlements file, no iOS-only kernel keys on macOS, contents - rag: absolute store path on native (a bare name resolves against the working directory, not writable on Android or iOS), real EmbeddingGemma URLs and the gated-token caveat, upsert/removeDocument/clear - inference: real Gemma 4 URL and size, withProgress, systemInstruction, audio input as a whole WAV file, "Session is closed", sampler set on the first session, clamp only on native .litertlm, the token ships in the app - function-calling: modelType needed only on web and ONNX; the loop's contract (onToolCall type, maxToolTurns ends quietly, exceptions rethrown) - speech: typed PCM contract, recording at 16 kHz, a data-chunk reader instead of "strip 44 bytes", whisper-base, the verbatim TTS language error - builtin-ai: desktop Chrome only, unavailable states are not final, TimeoutException, a fallback to a downloaded model that compiles - mediapipe, onnx: static frameworks back, web image rules, ONNX web installs by repo URL - .pubignore: ship web/opfs_helper.js; the skills rule goes last - core README, the rag dartdoc and two site pages: absolute store path
Run from the example app, `dart run skills@ get` installed nothing. With no agent directory it cannot detect the agent and stops; given one with `--agent`, it lists the seven skills and asks for `--skill` or `--all`. `--all` installs all seven plus references/platform-setup.md.
A feature card in the landing grid and /docs/ai-assistants under Features: the install command, where each assistant's skills land (from the skills CLI's own agent table), how the skills load, the seven skills, and the defaults they spell out. Named apart from "Agent Skills", which is flutter_gemma_agent — skills the on-device model runs — and the page says so up front. Shell blocks use a bare fence: the site's highlighter knows only dart, and a bash fence fails the page with a null check in Highlighter.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Dart team's
skillsCLI (publisherlabs.dart.dev) lets a package ship instruction files that a user's coding assistant installs withdart run skills@ get— Claude Code, Codex, Cursor, Antigravity, Cline, Copilot, OpenCode.Seven skills, all in
flutter_gemmaflutter-gemma-inference.litertlmengine; empty replies,maxTokensnot shortening replies, "No inference engine can handle this model". Carriesreferences/platform-setup.md(Podfile, entitlements,web/index.html).flutter-gemma-function-callingFunctionCallResponse,Message.toolResponse, the built-in tool loop; the model describing an action instead of calling itflutter-gemma-ragflutter-gemma-speechVoiceSession; transcripts in English for non-English audioflutter-gemma-mediapipe.task/.binmodelsflutter-gemma-onnxflutter-gemma-builtin-aiAll in core rather than the packages they describe: one release path for every fix. The
.litertlmengine is folded intoinferencebecause it is the default — a request like "add offline chat" names no engine, so the default path has to be self-contained. The other engines are ones a user names explicitly, so their skills activate on that.Our own
.claude/skills/(release, build-native, review-pr, …) are maintainer tools and stay where they are.Review found five of eight drafts broken
An earlier draft of this PR had eight skills. Four independent reviews — spec conformance, comparison with ~30 published skills, a line-by-line fact-check against the sources, and a with/without usefulness test — found code that would not compile or would fail at runtime in five of them. Every one was verified against the source before being fixed:
ModelType.gemma3ModelType.gemmaItswitchoverModelResponsewithoutThinkingResponseFlutterGemma.rag.search(vector, limit:)searchSimilar(query:, topK:), and it takes textCondition.equals(...)FieldEquals(key:, value:)LiteRtEmbeddingBackendfromflutter_gemma_embeddingsflutter_gemma_litertlmSqliteVectorStore()on webUnimplementedError— web needsWebSqliteVectorStorefilterSchemagetActiveModel(spec: ...)for built-in AIspecparameter; install with.fromBundled(...)fileType: ModelFileType.onnxcpudoes —gpufalls back to CPU,nputo GPU then CPU.litertlmcrashes below 1024 tokensAdded traps no draft covered:
filterSchema,openSession/openChatfor concurrent conversations,stopGeneration,topK: 1greedy default, the.litertlmweb handshake, and reading the Hugging Face token fromString.fromEnvironmentinstead of the draft's inline token.An agent with only the skills found what the gate could not
A second round added a fifth check: an agent given nothing but the installed skills wrote six real apps — offline chat, tool calling, filtered note search, a German WAV transcript, built-in AI with a fallback, the chat on web. Compiled in the example app, all six failed on the same line: no skill imported
package:flutter_gemma/flutter_gemma.dart, and the engine packages do not re-export it. The gate was green because it injected that import itself. It no longer does; on the previous text it reports 161 errors.With the import added, every signature the agent had to guess was right. What it lacked were facts, now in the skills:
InferenceModel,InferenceChat,RetrievalResult,SpeechRecognizerin the exampleswithProgress;install()skips a file already on diskrag.initialize('rag.db')INTERNET— Flutter's template declares it for debug and profile only.litertlmpost_installbuild phase from the READMEdata-chunk reader, and recording at 16 kHz monoString.fromEnvironmentas secret handlingThe fact-check in the same round corrected:
.litertlmraisesmaxTokensonly natively; a secondcreateSessioncloses the first (Session is closed);modelTypeis needed only on web and ONNX; built-in AI on the web is desktop Chrome only (Edge runs Phi-4-mini) and itsunavailable*states are not final; MediaPipe needsuse_frameworks! :linkage => :staticback.The gate compiles the skills
tool/check_skills.dartturns every```dartfence into a function body and every inlineType/Type.memberin the prose into a declaration, then runsdart analyzeinside the example app, which depends on every package. It imports nothing for the skills beyonddart:typed_datafor its placeholders — a skill has to show every import its code needs.It replaces a grep-based check that was green on four APIs that did not exist:
gemma3matched a model URL,limit:an unrelated argument,spec:a parameter elsewhere,isThinkinga doc comment. A text search cannot tell "this name exists" from "this code is right".Verified in both directions:
withProgress,systemInstruction:,RetrievalResult,Endian.little,on TimeoutException,onMaxToolTurns:,supportImage:on a MediaPipe chat.tool/skills_review.sh <last-tag>routes a release's diff to the skills it puts in doubt, for the one thing compilation cannot catch — a symbol whose meaning changed. Release step 12d runs both.Distribution
packages/flutter_gemma/.pubignoreexcluded**/*.md, stripping everySKILL.mdfrom the archive — the skills would have reached nobody. Fixed and verified by running the CLI from the example app:dart run skills@ get --allinstalls all seven, plusreferences/platform-setup.md. Without--allit only lists them, and without an agent directory it needs--agent— the README now says both. The!skills/rule now comes last: the last matching rule wins, and theassets/anddocs/rules would otherwise strip those subdirectories from a skill.The same dry run showed
web/opfs_helper.jsmissing from the archive: onlycache_api.jswas whitelisted, whileWebStorageMode.streamingneeds the other and every web guide says to copy both from the package. Whitelisted.On the site
A feature card on the landing page and a docs page,
/docs/ai-assistants, under Features: the install command, which directory each assistant reads (from the CLI's own agent table), the seven skills, and the defaults they prevent. It is named AI Assistant Skills so it does not collide with Agent Skills —flutter_gemma_agent, skills the on-device model runs — and the page says so up front.Also fixed
CLAUDE.mdsaid built-in AI and ONNX have no web — the source the drafts copied that from.website/content/docs/models.mdsaid moonshine transcribes the language it hears.FlutterGemma.ragdartdoc and two site pages passed a bare name to the RAG store.Versions
flutter_gemma1.8.1 — the skills ship inside its archive.flutter_gemma_speechis untouched.Known, not resolved here
The README asks for
disable-library-validationin both macOS entitlement files; the example has it only inDebugProfile.entitlements. The skill follows the README without claiming either outcome; settling it needs a macOS release build.Three satellite READMEs repeat facts the skills corrected — builtin_ai says Edge runs Gemini Nano, rag_qdrant and both RAG examples pass a bare store name. Each needs its own package release.