Skip to content

Commit b48458a

Browse files
tombeckenhamclaudeautofix-ci[bot]
authored
docs: refresh AI SDK comparison against ai@6.0.200 (#745)
* chore: add activity-coverage dimension to gap-analysis skill Adds a sixth audit dimension (and an `activities` invocation scope) that diffs which of the 7 core activity kinds each provider ships an adapter for against what upstream offers, with a provider×activity grid in the report template. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: refresh AI SDK comparison against ai@6.0.200 Corrections (verified against ai-sdk.dev docs, npm, and vercel/ai as of June 2026): - ai@6 is dual CJS+ESM, not ESM-only; note v7 pre-release exists - ~38 first-party provider packages, not 50+ - Solid is community-maintained on the AI SDK side (frozen at an SDK-4-era version) - now a TanStack differentiator, not a wash - Lazy tool discovery is no longer unmatched: AI SDK 6 has Anthropic-only tool search with deferLoading; ours remains cross-provider - Narrow the middleware claim: experimental_transform, onToolCallStart/ Finish, prepareStep, and repairToolCall exist - the gap is the unified app-level middleware system - extendAdapter: AI SDK has stable customProvider()/createProviderRegistry(); the edge is literal-type narrowing vs string model ids - Code execution: provider-hosted tools exist (Anthropic, xAI, OpenAI); the edge is SDK-managed sandboxes you control - addToolResult renamed to addToolOutput in v6 New advantages (each adversarially verified to have no AI SDK equivalent): - Native AG-UI protocol section + wire-protocol table row - Hooks for Every Activity section + generation-hooks table row - Multi-Turn Structured Output section - Debug Logging section + table row - Code Mode Skills table row + LLM-writable skills prose Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci: apply automated fixes --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
1 parent ff267a5 commit b48458a

4 files changed

Lines changed: 182 additions & 33 deletions

File tree

.claude/skills/gap-analysis/SKILL.md

Lines changed: 44 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,16 @@ markdown report under `.agent/gap-analysis/`. **Do not edit source files.**
1515

1616
## Invocation
1717

18-
| Args | Scope |
19-
| -------------------------------- | --------------------------------------------------- |
20-
| `<provider>` (e.g. `openai`) | One provider — all four audit dimensions. |
21-
| `feature <feature>` (e.g. `tts`) | One feature row of the matrix across all providers. |
22-
| `models` | New-model diff for every provider. |
23-
| `--all` | Full sweep (fan out subagents, one per provider). |
24-
| _(none)_ | Ask the user which scope via AskUserQuestion. |
18+
| Args | Scope |
19+
| -------------------------------- | ----------------------------------------------------- |
20+
| `<provider>` (e.g. `openai`) | One provider — all audit dimensions. |
21+
| `feature <feature>` (e.g. `tts`) | One feature row of the matrix across all providers. |
22+
| `models` | New-model diff for every provider. |
23+
| `activities` | Activity-coverage diff: which of the 7 core activity |
24+
| | kinds each provider ships an adapter for, vs. what |
25+
| | upstream supports. (Dimension 6 only, all providers.) |
26+
| `--all` | Full sweep (fan out subagents, one per provider). |
27+
| _(none)_ | Ask the user which scope via AskUserQuestion. |
2528

2629
## Workflow
2730

@@ -44,12 +47,18 @@ markdown report under `.agent/gap-analysis/`. **Do not edit source files.**
4447
4. Capability-flag drift
4548
5. Telemetry / observability parity (usage tokens, cache/reasoning
4649
counts, request ids, logging asymmetry)
50+
6. Activity coverage (which of the 7 core activity kinds each provider
51+
ships an adapter for vs. what upstream supports) — this is the **only**
52+
dimension for the `activities` scope; it's also rolled into `--all`.
4753
5. **Fan out** for `--all`: launch one `Explore` subagent per provider, max 3
48-
in parallel. Each subagent returns the five-dimension findings for its
49-
provider; you synthesise into the combined report.
54+
in parallel. Each subagent returns the multi-dimension findings for its
55+
provider; you synthesise into the combined report. The `activities` scope
56+
does **not** fan out — derive the provider×activity matrix centrally from
57+
the adapter files (see dimension 6), since it's a fast mechanical diff.
5058
6. **Write the report** to `.agent/gap-analysis/YYYY-MM-DD-<scope>.md` using
5159
[references/report-template.md](references/report-template.md). Date is
52-
today's ISO date. `<scope>` is `openai` / `feature-tts` / `models` / `all`.
60+
today's ISO date. `<scope>` is `openai` / `feature-tts` / `models` /
61+
`activities` / `all`.
5362
7. **Print the report path and a 5-line summary** to the user.
5463

5564
## Critical rules
@@ -88,6 +97,31 @@ re-read it; this list is a snapshot:
8897
`multimodal-structured`, `summarize`, `summarize-stream`, `image-gen`, `tts`,
8998
`transcription`, `video-gen`.
9099

100+
## Known activities (7)
101+
102+
**Features** (above) are matrix rows about behaviours within an activity.
103+
**Activities** are the coarser-grained core capability kinds in `@tanstack/ai`
104+
— each has a `Base<Kind>Adapter` and a provider "supports" one only if its
105+
package ships an adapter of that kind. Canonical list is the `AdapterKind`
106+
union in `packages/ai/src/activities/index.ts` — always re-read it:
107+
108+
`text`, `summarize`, `image`, `audio`, `video`, `tts`, `transcription`.
109+
110+
A provider's activity surface is derived mechanically from its adapter files:
111+
`packages/ai-<provider>/src/adapters/`. Filename → activity-kind map:
112+
113+
| Adapter file | Activity kind |
114+
| ------------------------------------------------------------ | --------------- |
115+
| `text.ts` / `text-chat-completions.ts` / `responses-text.ts` | `text` |
116+
| `summarize.ts` | `summarize` |
117+
| `image.ts` | `image` |
118+
| `audio.ts` | `audio` |
119+
| `video.ts` | `video` |
120+
| `speech.ts` / `tts.ts` | `tts` |
121+
| `transcription.ts` | `transcription` |
122+
123+
(`cost.ts` is a helper, not an activity adapter.)
124+
91125
## Verification before finishing
92126

93127
Before printing the summary:

.claude/skills/gap-analysis/references/audit-checklist.md

Lines changed: 70 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,65 @@ Steps:
217217

218218
---
219219

220+
## 6. Activity coverage
221+
222+
**Input:** each provider's `packages/ai-<provider>/src/adapters/` directory.
223+
**Reference:** the `AdapterKind` union in `packages/ai/src/activities/index.ts`
224+
(7 kinds: `text`, `summarize`, `image`, `audio`, `video`, `tts`,
225+
`transcription`).
226+
**Upstream:** each provider's API reference / models page.
227+
228+
This dimension answers a coarser question than dimension 2: not "which
229+
behaviour within chat is exercised" but **"which whole activity kinds does the
230+
provider's API offer that we ship no adapter for at all."**
231+
232+
Steps:
233+
234+
1. For every provider, list its adapter files and map each to an activity kind
235+
using the filename→kind table in `SKILL.md` (§ Known activities). That
236+
yields the local provider×activity matrix. `text.ts`,
237+
`text-chat-completions.ts`, and `responses-text.ts` all count as `text`;
238+
`speech.ts` and `tts.ts` both count as `tts`; ignore `cost.ts` and other
239+
helpers.
240+
2. Build the full grid (rows = 9 providers, cols = 7 activity kinds). A cell is
241+
✅ if an adapter file of that kind exists, ❌ otherwise.
242+
3. For each ❌ cell, WebFetch the provider's API reference and check whether
243+
upstream offers that activity:
244+
- **Real gap** — upstream offers it, we ship no adapter, and there is no
245+
documented reason. High if it's a flagship activity for that provider
246+
(e.g. a TTS provider missing `transcription`), medium otherwise.
247+
- **Not a gap** — upstream genuinely doesn't offer that activity (e.g.
248+
Anthropic has no image/audio/video/tts/transcription endpoints; Groq has
249+
no image/video generation). Mark the cell ❌-by-design and omit from the
250+
gap list, but keep it in the grid so the report is self-contained.
251+
4. Note the inverse too: any **local adapter for an activity the provider no
252+
longer offers upstream** (rare; surface as a stale-adapter finding).
253+
254+
Map activity kind → upstream capability to look for:
255+
256+
| Activity kind | Upstream capability |
257+
| --------------- | ------------------------------------------------- |
258+
| `text` | Chat / messages / completions endpoint |
259+
| `summarize` | Any text completion (summarize is built on chat) |
260+
| `image` | Image generation endpoint |
261+
| `audio` | Music / sound / general audio generation endpoint |
262+
| `video` | Video generation endpoint |
263+
| `tts` | Text-to-speech endpoint |
264+
| `transcription` | Speech-to-text endpoint |
265+
266+
**Priority rubric:**
267+
268+
- Missing a core activity the provider's API clearly offers (e.g. OpenAI-class
269+
provider with no `image`/`tts`/`transcription`) → **high**.
270+
- Missing a secondary/media activity → **medium**.
271+
- ❌-by-design (upstream doesn't offer it) → **out-of-scope**, keep in grid only.
272+
- Local adapter for a now-removed upstream activity → **medium** (deprecate).
273+
274+
Render the grid in the report under its own "Activity coverage" section, with
275+
one bullet per real gap citing the upstream URL.
276+
277+
---
278+
220279
## Subagent dispatch (for `--all` scope)
221280

222281
When fan-out is needed, launch one `Explore` subagent per provider with a
@@ -225,12 +284,17 @@ prompt of this shape:
225284
> Audit the `<provider>` adapter at `packages/ai-<provider>/`
226285
> against upstream docs at the URLs in
227286
> `.claude/skills/gap-analysis/references/provider-doc-urls.md`. Walk
228-
> dimensions 1, 3, 4, and 5 from `audit-checklist.md`. Skip dimension 2
229-
> (the orchestrator handles cross-provider parity centrally) — but do
230-
> emit dimension-5 telemetry rows in the per-provider format; the
231-
> orchestrator stitches them into the cross-adapter table. Return
232-
> findings as markdown sections matching the report template — High /
233-
> Medium / Low / Out-of-scope — with upstream URLs cited for every claim.
287+
> dimensions 1, 3, 4, and 5 from `audit-checklist.md`. Skip dimensions 2
288+
> and 6 (the orchestrator handles cross-provider parity and the
289+
> activity-coverage grid centrally) — but do emit dimension-5 telemetry
290+
> rows in the per-provider format; the orchestrator stitches them into the
291+
> cross-adapter table. Return findings as markdown sections matching the
292+
> report template — High / Medium / Low / Out-of-scope — with upstream
293+
> URLs cited for every claim.
294+
295+
The orchestrator builds the dimension-6 activity-coverage grid itself from
296+
the adapter-file listing (a one-shot `ls packages/ai-*/src/adapters/`), then
297+
WebFetches each provider's API reference only for the ❌ cells.
234298

235299
Run at most 3 in parallel. Aggregate their returned markdown into the
236300
combined report.

.claude/skills/gap-analysis/references/report-template.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,29 @@ Replace every `{{placeholder}}`. Drop sections that have zero entries
9595

9696
---
9797

98+
## Activity coverage
99+
100+
> Per dimension 6 in audit-checklist.md. Which of the 7 core activity kinds
101+
> (`text`, `summarize`, `image`, `audio`, `video`, `tts`, `transcription`)
102+
> each provider ships an adapter for, vs. what upstream offers. Derived from
103+
> `packages/ai-<provider>/src/adapters/`.
104+
105+
| Provider | text | summarize | image | audio | video | tts | transcription |
106+
| ------------ | :---------: | :-------: | :---: | :---: | :---: | :-: | :-----------: |
107+
| {{provider}} | {{✅//—}} |||||||
108+
109+
Legend: ✅ adapter shipped · ❌ upstream offers it, no adapter (gap) · — upstream doesn't offer it (by design).
110+
111+
- **[{{provider}}] missing `{{activity}}` activity**
112+
- Upstream: [{{doc-title}}]({{doc-url}})
113+
- Current state: no `{{file}}.ts` under `packages/ai-{{provider}}/src/adapters/`
114+
- Suggested change: add a `{{provider}}{{Activity}}` adapter (mirror {{sibling provider that has it}}).
115+
- Effort: {{S / M / L}}
116+
117+
{{repeat per real activity gap}}
118+
119+
---
120+
98121
## Out-of-scopedocumented exclusions
99122

100123
> Listed for completeness; no action required. Each links to the comment

0 commit comments

Comments
 (0)