Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/publish-npm-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ on:
required: true
type: choice
options:
- a2ui
- react-ui
- react-headless
- react-lang
Expand Down
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ Try it yourself in the [Playground](https://www.openui.com/playground): generate
| Package | Best for | Description |
| :--------------------------------------------------------------------------------------------------------- | :----------------------------------------------- | :----------------------------------------------------------------------------------------------------------- |
| [`@openuidev/lang-core`](./packages/lang-core) | Framework-agnostic parsing and prompt generation | Core parser, prompt-generation, runtime-evaluation, and type layer with no React, Vue, or Svelte dependency |
| [`@openuidev/cli`](./packages/openui-cli) | Project scaffolding and prompt generation | CLI for creating new apps and generating system prompts or JSON schema from a library definition |
| [`@openuidev/langchain`](./packages/langchain) | LangChain and LangGraph agents | Agent transformer and server helpers that stream OpenUI through AG-UI |
| [`@openuidev/react-lang`](./packages/react-lang) | React rendering runtimes | Define component libraries, generate prompts, and render streamed OpenUI Lang in React |
| [`@openuidev/react-headless`](./packages/react-headless) | Bring-your-own React chat UI | Headless chat state, streaming adapters, and message format converters |
Expand All @@ -102,7 +103,7 @@ Try it yourself in the [Playground](https://www.openui.com/playground): generate
| [`@openuidev/vue-lang`](./packages/vue-lang) | Vue integrations | Vue 3 bindings for defining model-renderable components and rendering streamed OpenUI Lang |
| [`@openuidev/svelte-lang`](./packages/svelte-lang) | Svelte integrations | Svelte 5 bindings for defining model-renderable components and rendering streamed OpenUI Lang |
| [`@openuidev/browser-bundle`](./packages/browser-bundle) | CDN, iframe, and no-build embeds | Prebuilt browser bundle that ships the renderer, UI library, React, and styles as script + stylesheet assets |
| [`@openuidev/cli`](./packages/openui-cli) | Project scaffolding and prompt generation | CLI for creating new apps and generating system prompts or JSON schema from a library definition |
| [`@openuidev/a2ui`](./packages/a2ui) | A2UI with OpenUI Lang component payloads | Framework-agnostic A2UI v1.0 protocol client with an optional React renderer |
| [`@openuidev/openclaw-os-plugin`](https://github.com/thesysdev/openclaw-os/tree/main/packages/claw-plugin) | OpenClaw workspaces | OpenClaw OS plugin for serving OpenUI-powered OpenClaw workspaces |

Common starting points:
Expand All @@ -120,6 +121,9 @@ npm install @openuidev/langchain @langchain/langgraph
# Vue or Svelte runtime
npm install @openuidev/vue-lang
npm install @openuidev/svelte-lang

# A2UI protocol with OpenUI Lang component payloads
npm install @openuidev/a2ui zod
```

## Why OpenUI Lang
Expand Down Expand Up @@ -157,16 +161,17 @@ Detailed documentation is available at [openui.com](https://openui.com).
```
openui/
├── packages/
│ ├── lang-core/ # Framework-agnostic parser, prompt, and runtime layer
│ ├── openui-cli/ # CLI for scaffolding & prompt generation
│ ├── react-lang/ # Core runtime (parser, renderer, prompt generation)
│ ├── react-headless/ # Headless chat state & streaming adapters
│ ├── react-ui/ # Prebuilt chat layouts & component libraries
│ ├── react-email/ # React Email component library for generated emails
│ ├── lang-core/ # Framework-agnostic parser, prompt, and runtime layer
│ ├── langchain/ # LangChain/LangGraph streaming integration
│ ├── vue-lang/ # Vue runtime bindings for OpenUI Lang
│ ├── svelte-lang/ # Svelte runtime bindings for OpenUI Lang
│ ├── browser-bundle/ # Script-tag bundle for CDN / iframe / no-build embeds
│ └── openui-cli/ # CLI for scaffolding & prompt generation
│ └── a2ui/ # A2UI protocol with OpenUI Lang component payloads
├── skills/
│ └── openui/ # Claude Code skill for AI-assisted development
├── examples/
Expand Down
42 changes: 42 additions & 0 deletions benchmarks/A2UI_BENCHMARK.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# A2UI JSON vs A2UI + OpenUI Lang

Across seven existing UI fixtures, replacing A2UI's JSON component objects with OpenUI Lang statements reduced the complete protocol streams from **6,894 to 5,526 tokens**: **1,368 fewer tokens (19.8%)**. At a fixed 60 output tokens/second, that is **22.8 seconds less estimated decode time** across the corpus, or a **1.25x aggregate speedup**.

| Scenario | A2UI + JSON | A2UI + OpenUI Lang | Tokens saved | Reduction | JSON latency | OpenUI latency | Speedup |
| ------------------ | ----------: | -----------------: | -----------: | ---------: | -----------: | -------------: | --------: |
| simple-table | 249 | 210 | 39 | -15.7% | 4.15s | 3.50s | 1.19x |
| chart-with-data | 363 | 294 | 69 | -19.0% | 6.05s | 4.90s | 1.23x |
| contact-form | 570 | 370 | 200 | -35.1% | 9.50s | 6.17s | 1.54x |
| dashboard | 1574 | 1349 | 225 | -14.3% | 26.23s | 22.48s | 1.17x |
| pricing-page | 1656 | 1383 | 273 | -16.5% | 27.60s | 23.05s | 1.20x |
| settings-panel | 837 | 628 | 209 | -25.0% | 13.95s | 10.47s | 1.33x |
| e-commerce-product | 1645 | 1292 | 353 | -21.5% | 27.42s | 21.53s | 1.27x |
| **TOTAL** | **6894** | **5526** | **1368** | **-19.8%** | **114.90s** | **92.10s** | **1.25x** |

## What was held constant

- Both variants use A2UI v1.0 and contain the same compact `createSurface` plus `updateComponents` JSONL messages.
- Both encode the same parsed component tree from the existing OpenUI benchmark fixtures.
- The surface ID, catalog ID, message boundaries, and JSON serialization are identical.
- The only variable is `updateComponents.components`: flat catalog component objects for A2UI + JSON, statement strings for A2UI + OpenUI Lang.

The A2UI JSON side uses a custom catalog with the same component and prop names as the OpenUI fixture library. Its component graph is flattened with deterministic IDs, matching A2UI's adjacency-list model. Both streams are minified, so the comparison does not rely on JSON whitespace. This is a same-library encoding comparison, not a comparison between A2UI's Basic Catalog and the OpenUI library.

## Reproduce it

Build the workspace packages once from the repository root, then run the offline benchmark:

```bash
pnpm install --frozen-lockfile
cd benchmarks
pnpm install --ignore-workspace
pnpm generate:a2ui
pnpm test:a2ui
pnpm bench:a2ui
```

Generation and measurement are offline and use the checked-in `.oui` fixtures. `test:a2ui` checks that both protocol envelopes are otherwise identical and replays all seven OpenUI Lang streams through the new client and Lang parser. Token counts use `tiktoken` with the `gpt-5` encoding. Latency is an estimate at 60 output tokens/second; it does not include catalog or prompt tokens, model time-to-first-token, parsing/rendering CPU, transport overhead, or compression.

## Issue-ready summary

Add an experimental `@openuidev/a2ui` package that preserves the A2UI v1.0 lifecycle, data-model, action, RPC, capability, and error messages, while representing component lists as OpenUI Lang statement strings. The package maintains per-surface Lang source through statement-level patches, parses it with `@openuidev/lang-core`, and exposes an optional React surface renderer. The reproducible seven-scenario benchmark shows a 19.8% full-stream token reduction and 1.25x estimated decode speedup versus compact A2UI JSON.
25 changes: 23 additions & 2 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ Measures token efficiency and estimated generation latency of **OpenUI Lang** vs

All four formats encode exactly the same UI. The LLM always generates OpenUI Lang, then the parsed AST is projected into the other three formats.

The same fixture set also includes a focused **A2UI v1.0 + JSON vs A2UI v1.0 + OpenUI Lang** comparison. Both sides contain the same compact `createSurface` and `updateComponents` JSONL envelopes and the same parsed UI. The only variable is `updateComponents.components`: A2UI catalog component objects on one side, OpenUI Lang statement strings on the other.

See [A2UI_BENCHMARK.md](./A2UI_BENCHMARK.md) for the focused results and methodology.

## Methodology

1. Use a fixed set of seven prompts in `generate-samples.ts` (`simple-table`, `chart-with-data`, `contact-form`, `dashboard`, `pricing-page`, `settings-panel`, `e-commerce-product`).
Expand Down Expand Up @@ -44,7 +48,6 @@ Measured with `tiktoken` (`gpt-5` model encoder). Generated by GPT-5.2 at temper
| e-commerce-product | 2145 | 2449 | 2381 | 1166 | -45.6% | -52.4% | -51.0% |
| **TOTAL** | **9122** | **10180** | **9948** | **4800** | **-47.4%** | **-52.8%** | **-51.7%** |


## Running

### Prerequisites
Expand All @@ -55,7 +58,6 @@ Export `OPENAI_API_KEY` in your shell:
export OPENAI_API_KEY=sk-...
```


### 1. Generate samples (calls OpenAI)

```bash
Expand All @@ -72,11 +74,28 @@ pnpm bench

Reads the files in `samples/`, counts tokens with `tiktoken`, and prints token and latency tables.

### Focused A2UI comparison

The checked-in `.oui` fixtures are enough to reproduce the A2UI comparison without an API key:

```bash
pnpm generate:a2ui
pnpm test:a2ui
pnpm bench:a2ui
```

The first command projects each parsed fixture to both complete A2UI streams. The test verifies identical envelopes and replays every OpenUI Lang stream through `@openuidev/a2ui`. The final command counts full-stream tokens, prints the table, and writes machine-readable results to `a2ui-results.json`. Estimated latency uses the same fixed 60 output tokens/second as the main benchmark.

## File Layout

```text
benchmarks/
├── generate-samples.ts # Calls OpenAI, converts AST to all four formats
├── generate-a2ui-samples.ts # Offline projection to both A2UI streams
├── a2ui-converter.ts # AST -> A2UI JSON and Lang protocol streams
├── run-a2ui-benchmark.ts # Focused A2UI token and latency report
├── check-a2ui.ts # Envelope invariance and Lang replay checks
├── a2ui-results.json # Machine-readable focused benchmark results
├── run-benchmark.ts # Reads samples/, prints token/latency tables
├── thesys-c1-converter.ts # AST -> normalized Thesys C1 JSON converter
├── vercel-spec-converter.ts # AST -> shared json-render spec projection
Expand All @@ -91,6 +110,8 @@ benchmarks/
├── <scenario>.c1.json
├── <scenario>.vercel.jsonl
├── <scenario>.yaml
├── <scenario>.a2ui.jsonl
├── <scenario>.a2ui-openui.jsonl
└── metrics.json
```

Expand Down
150 changes: 150 additions & 0 deletions benchmarks/a2ui-converter.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
import type { ElementNode } from "@openuidev/lang-core";

export const A2UI_BENCHMARK_CATALOG_ID = "https://openui.com/catalog/default";

interface A2UIComponent {
id: string;
component: string;
[key: string]: unknown;
}

function isElementNode(value: unknown): value is ElementNode {
if (!value || typeof value !== "object" || Array.isArray(value)) return false;
const object = value as Record<string, unknown>;
return object.type === "element" && typeof object.typeName === "string";
}

function sanitizeId(value: string): string {
return value.replace(/[^A-Za-z0-9_-]/g, "_");
}

export function astToA2UIComponents(root: ElementNode): A2UIComponent[] {
const components: A2UIComponent[] = [];
const ids = new WeakMap<object, string>();
const usedIds = new Set<string>();
let anonymousId = 0;

const reserveId = (node: ElementNode, preferred?: string): string => {
const existing = ids.get(node);
if (existing) return existing;

let candidate = preferred ? sanitizeId(preferred) : "";
if (!candidate || usedIds.has(candidate)) {
do {
anonymousId += 1;
candidate = `c${anonymousId}`;
} while (usedIds.has(candidate));
}
usedIds.add(candidate);
ids.set(node, candidate);
return candidate;
};

const projectValue = (value: unknown): unknown => {
if (isElementNode(value)) return visit(value);
if (Array.isArray(value)) return value.map(projectValue);
if (value && typeof value === "object") {
return Object.fromEntries(
Object.entries(value)
.filter(([, child]) => child !== undefined)
.map(([key, child]) => [key, projectValue(child)]),
);
}
return value;
};

const visit = (node: ElementNode, forcedId?: string): string => {
const id = reserveId(node, forcedId ?? node.statementId);
if (components.some((component) => component.id === id)) return id;

const component: A2UIComponent = { id, component: node.typeName };
components.push(component);
for (const [key, value] of Object.entries(node.props)) {
if (value !== undefined) component[key] = projectValue(value);
}
return id;
};

visit(root, "root");
return components;
}

export function splitOpenUIStatements(source: string): string[] {
const statements: string[] = [];
let depth = 0;
let quote: false | '"' | "'" = false;
let escaped = false;
let start = 0;

for (let index = 0; index < source.length; index++) {
const character = source[index]!;
if (escaped) {
escaped = false;
continue;
}
if (quote && character === "\\") {
escaped = true;
continue;
}
if (quote) {
if (character === quote) quote = false;
continue;
}
if (character === '"' || character === "'") {
quote = character;
continue;
}
if (character === "(" || character === "[" || character === "{") depth += 1;
else if (character === ")" || character === "]" || character === "}") {
depth = Math.max(0, depth - 1);
} else if (character === "\n" && depth === 0) {
const statement = source.slice(start, index).trim();
if (statement) statements.push(statement);
start = index + 1;
}
}

const finalStatement = source.slice(start).trim();
if (finalStatement) statements.push(finalStatement);
return statements;
}

function createSurface(surfaceId: string) {
return {
version: "v1.0",
createSurface: {
surfaceId,
catalogId: A2UI_BENCHMARK_CATALOG_ID,
},
} as const;
}

function toJsonl(messages: unknown[]): string {
return `${messages.map((message) => JSON.stringify(message)).join("\n")}\n`;
}

export function astToA2UIJsonStream(root: ElementNode, surfaceId = "main"): string {
return toJsonl([
createSurface(surfaceId),
{
version: "v1.0",
updateComponents: {
surfaceId,
components: astToA2UIComponents(root),
},
},
]);
}

export function openUIToA2UILangStream(source: string, surfaceId = "main"): string {
return toJsonl([
createSurface(surfaceId),
{
version: "v1.0",
updateComponents: {
surfaceId,
components: splitOpenUIStatements(source),
},
},
]);
}
Loading
Loading