Design: directive seam — named inline commands with typed arguments (`@font(size: 18){…}`)

[wildthink]

Opening this per CONTRIBUTING ("for non-trivial features please open an issue first"). The work is already prototyped and green, but I'd rather have the design discussion before sending a PR — and I'm happy to reshape or drop it.

Branch: https://github.com/wildthink/swift-markdown-engine/tree/feat/directives

The gap

MarkdownExtension covers delimiter-shaped constructs beautifully. What it can't express is a construct with a name and typed arguments — InlineSyntax is open/close strings, so there's no way to write @font(size: 18){…} as an extension.

The proposal is a sibling seam, MarkdownDirective, deliberately built to the same isolation contract: it supplies syntax and presentation, never ranges.

config.directives = [FontDirective(), ColorDirective(), IconDirective()]

@font(size: 1.5em){half again}, @color(red){tinted}, @icon(star.fill, color: yellow)

The one design decision worth arguing about

Two forms, both tree-shaped: self-contained (@pagebreak, a leaf that draws a glyph) and container (@font(size: 18){text}, whose body is re-parsed).

There is deliberately no "applies to everything after me" form, even though that's the obvious reading of \font(size: 18). Two reasons, both from ARCHITECTURE.md:

1. MarkdownASTStyler composes attributes on descent. A font comes from the enclosing node, not from document position. A stream-stateful directive would have to mutate state between siblings, which that model has no place for.
2. Restyling and tokenization are block-scoped. A directive whose effect crossed block boundaries would invalidate scopedRanges and the BlockScopedTokenizer memo, taking per-keystroke cost from O(edit) to O(document-after-edit).

Scoping to a body keeps both intact, and composition then falls out of the existing walk rather than fighting it — @font(size: 18){**bold**} is bold and 18pt, and the same call inside a heading keeps the heading's weight, because the recursion just carries the transformed font down.

If you'd want the "rest of the block" form later, it's reachable as a bounded block-level directive without touching this — but I'd rather not build it speculatively.

How it fits the stated constraints

• No new dependencies. Core target untouched on that front.
• No new case threaded through parser, styler, and renderer. This was the constraint that shaped the implementation most. Directives project into the AST as extension-shaped nodes under a reserved directive. id namespace, so InlineNode, buildTree, offsetNodes, InlineASTAdapter, MarkdownToken, and shrinkInlineMarkers are all unchanged — and directives inherit marker shrink, caret reveal, token projection, incremental restyle, and rich copy for free.
• Built-ins classify first. The scanner runs after every built-in in matchClaimedSpan, same as extension spans.
• Markers shrink, they don't disappear. Self-contained glyphs reuse the inline-LaTeX collapse mechanism; source characters are never removed, so selection, find, copy, and undo still see them.
• Caches. DirectiveRegistry is carried by ExtensionRegistry, so the directive fingerprint folds into the one grammar fingerprint every parse cache already keys on — no second key threaded anywhere. A directive-free registry produces a byte-identical fingerprint to today, so no existing document re-parses.
• Public surface. Purely additive and opt-in; every new public symbol has a DocC comment. Nothing changes unless directives are registered.

Two safety rules make it enable-able over an existing corpus without migration: registered names only (@home in prose stays literal unless home is registered), and a left-boundary rule (name@example.com never opens a directive).

Upstream footprint

Everything else lives in new files under Sources/MarkdownEngine/Directives/, plus one styler and one coordinator extension file.

File	Lines	What
InlineParser.swift	10	scanner hook in matchClaimedSpan
MarkdownExtension.swift	33	ExtensionRegistry carries the directive registry + folds its fingerprint
MarkdownEditorConfiguration.swift	12	directives + directiveSettings
MarkdownASTStyler.swift	16	.ext branch for directive composition; Ctx widened to internal
MarkdownHTMLRenderer.swift	52	directive rendering for clean copy
MarkdownPasteboardWriter.swift / NativeTextView+Copy.swift	11	thread directives to the copy path
NativeTextViewWrapper.swift / NativeTextViewCoordinator*.swift	47	autocomplete hooks

~180 lines across 9 existing files.

Autocomplete

Names and argument values, riding the seam the [[wiki-link]] picker already established — onCaretRectChange, onInlinePreviewKey, plus a new onDirectiveCompletion / pendingDirectiveCompletion pair. No picker UI ships in the engine; the demo's is ~60 lines.

Value candidates come from MarkdownDirective.valueCompletions(for:prefix:), whose default already answers anything the schema declares (closed keyword sets, booleans) — a directive implements it only when its domain is dynamic. FlagDirective offers every ISO region that way, matching on code or localised country name, with no shipped dataset (codes from Locale.Region, names from the user's locale, flag computed from regional-indicator scalars).

Worth noting: completion can't use the AST. Mid-typing, @ico and @icon(sta are exactly what the parser rejects, so DirectiveCompletionScanner is a separate backwards scan bounded to 256 chars per caret move. It reuses the parser's boundary rule, so it can never offer something the parser would refuse.

Testing

132 new tests, 385 passing, no regressions. swift build and swift test green; demo builds and runs. Coverage includes boundary and rejection cases, argument coercion and every diagnostic kind, composition in both directions, glyph degradation (an unresolvable symbol leaves the source visible rather than collapsing to an invisible gap), scoped-vs-full restyle equivalence, and the places the picker must stay shut (code spans, selections, IME composition, raw source mode).

Known follow-ups, not done here

• .latexImage / .latexBounds are now the generic glyph channel for four constructs (LaTeX, images, tables, directives) and the name is misleading. I reused rather than renamed to keep the diff small — a rename touches four files and is probably better as its own PR.
• Argument coercion isn't memoised; it re-derives per directive node per restyle. Cheap at any realistic density, but it's the remaining un-cached path.
• Directives are inline-only. Block-level (@layout(columns: 2) applying to the following block) would touch BlockParser and the restyle window — deliberately out of scope.

Questions for you

1. Is a second seam the right shape at all, or would you rather see this generalised into MarkdownExtension (e.g. an optional argument grammar on InlineSyntax)?
2. @ as the default marker — it avoids the double load \ already carries here (CommonMark escapes plus the LaTeX vocabulary behind $…$), and it's configurable per registry and per directive. Reasonable default for this project?
3. Should the bundled directives (FontDirective et al.) ship in core at all, or would you prefer the seam alone with those as demo-only examples?
4. Happy to split into smaller PRs along the phase boundaries (parsing → composition → glyphs → autocomplete) if that reviews better — the branch is already four coherent commits.

Also flagging: you have perf/typing-hotpath, perf/large-doc-switch, and fix/wikilink-uuid-loss-and-large-files-perf in flight touching the parse hot path and coordinator — the same files this touches. Nothing conflicts against main today, but I'll rebase if any of those land first.

[luca-chen198]

Thanks for opening this first, and for building it to the isolation contract rather than around it — directives riding InlineNode.ext under the directive. namespace, with a byte-identical fingerprint when nothing's registered, is exactly what makes me comfortable adding it.

1. Keep it a separate seam, don't fold it into MarkdownExtension. The shapes genuinely differ (delimiter pair vs. name + typed args + body), and teaching InlineSyntax about parens/braces/schemas would leak directive-only concepts into every plain extension — the opposite of the isolation I moved highlight/strikethrough out of core to get. Parallel seam, shared node, is the smaller thing.

2. @ is a good default. It dodges \ and the CommonMark escapes, and registered-names-only + the left-boundary rule keep @home and emails literal. Keep it overridable, and I'd reject emoji/multi-scalar markers so dispatch stays cheap. Worth one doc line that @ is the usual mention trigger, in case an embedder collides.

3. I'd draw the line a bit tighter than the branch: ship Font and Color in core as the reference directives (off by default, no call sites — same as Highlight/Strikethrough), and move Flag, Icon, PageBreak, Emoji to the demo. The first two are pure presentation; the rest carry curated data or print semantics, which is app policy, not an engine primitive. Emoji already being demo-only is the right precedent.

4. Yes, phase it — the four commits map cleanly to four PRs. Just note that the whole projection design (the namespace, the .ext fork, the registry/fingerprint plumbing) lands in PR1 even though its payoff shows up in PR2, so PR1 is really "sign off the projection," not just parsing.

One real ask before the restyle-heavy parts land: keeping per-keystroke cost at O(edit) is the thing I guard hardest, and nothing tests it for directives yet (each restyle re-coerces the full arg list, and the memo ships off). Could you add a perf scenario on a directive-heavy paragraph in a large doc? If it bites, wire the memo.

On branches: you can ignore perf/typing-hotpath and perf/large-doc-switch — both are outdated and I've removed them. Only fix/wikilink-uuid-loss-and-large-files-perf is still in flight and touches these files, so a rebase after it lands should be all you need.

[wildthink]

Thanks — that's a genuinely useful review, and the perf ask turned up something I'd have shipped blind otherwise. All four points addressed on the branch.

1 & 2 — kept as a separate seam; kept @ overridable. Added the doc line about @ being the conventional mention trigger, with the advice to pick a different marker rather than disambiguate at the trigger (two pickers competing for one keystroke is worse than a second marker). Multi-scalar markers were already rejected at registration but silently — now documented on DirectiveRegistrySettings and covered by a test that an emoji marker drops out rather than half-matching a lone surrogate.

3 — done, drawn where you asked. Core ships FontDirective and ColorDirective only, off by default with no call sites. @icon, @flag, @pagebreak moved to Demo/DemoDirectives.swift alongside @emoji. The core test suites now use their own fixtures instead of the bundled directives, which is better hermetics anyway — they exercise the seam rather than the examples.

4 — noted, and agreed on PR1 being "sign off the projection." The namespace, the .ext fork, and the registry/fingerprint plumbing all land there with only parsing to show for it.

On the perf ask

Your instinct to ask was right; your hypothesis about the cause wasn't, and neither was mine.

The O(edit) guarantee holds. A directive-dense paragraph costs the same scoped restyle whether it sits in a 50-paragraph or a 400-paragraph document — 3.2ms either way. Nothing directive-related scales with the document.

But argument coercion isn't where the time goes, so I have not wired the memo. Breaking down a scoped restyle of one paragraph:

	40 directives	120 directives
scoped restyle, total	3.2ms	14.5ms
├ parse	1.6ms	10.3ms
└ argument coercion	0.45ms	1.35ms

Coercion is under 10%. Memoising it would buy ~9% and add cache-invalidation surface, so it seemed like the wrong trade — say the word if you'd rather have it anyway.

The real cost is density within the edited paragraph, and it isn't a directive property. At equal span counts every construct shows the same curve, with directives marginally cheaper than highlights:

n spans	==highlight==	[text](url)	@font(…){…}
40	0.59ms	0.43ms	0.52ms
240	10.95ms	8.44ms	10.39ms

6× the spans costs ~19× the parse regardless of what produced them, which points at the quadratic span-containment passes (buildTree's isChild, scanLinkFamily's claimed-overlap scan). That's pre-existing and orthogonal to this work, so I filed it separately as #109 with the measurements rather than smuggling a parser change into a feature PR.

So the perf suite asserts what directives actually own — cost is flat in document size, and directives are not worse than equivalent built-in spans — and deliberately asserts no absolute per-paragraph budget, since that number belongs to #109.

One assertion I wrote and then removed: comparing scoped restyles of a single paragraph (~0.25ms) against a 2× bound. It failed, and measuring properly showed why — at that magnitude run-to-run variance exceeds 2×, so it was a noise detector, not a regression detector. Replaced with a full-document comparison where the signal clears the variance; stable across repeated runs.

All numbers are from one machine in a debug build, so treat the ratios as the claim and the absolutes as indicative.

State

Branch is updated and pushed: https://github.com/wildthink/swift-markdown-engine/tree/feat/directives — 389 tests green, swift build/swift test clean, demo builds and runs.

Happy to open PR1 (the projection) whenever suits you, or to hold if you'd rather look at #109 first, since a change there would touch the same parse path. And thanks for the branch cleanup note — I'll rebase after fix/wikilink-uuid-loss-and-large-files-perf lands.

[luca-chen198]

Go ahead and open PR1. It sits above the scans #109 is about, so I'd rather review it now and rebase later than leave it waiting.
And thanks for measuring instead of taking my word on the coercion. Splitting #109 out rather than folding it in is a big part of why I'm fine with this seam. For PR1 I'm mainly signing off the projection, so spell out the namespace choice and the fingerprint property in the description and keep the parsing thin.
Thx for the contribution