feat(relay,mcpl): markdown-preserving split sends + opt-in inline audio (RFC-006)

PORTAL-RFC-006-split-sends-and-inline-audio.md

@@ -0,0 +1,114 @@
+# PORTAL RFC-006 — Markdown-preserving split sends + inline audio
+
+- **Status:** ✅ Implemented (2026-07-08)
+- **Author:** Wolfram (drafted with Claude Code)
+- **Date:** 2026-07-08
+- **Affects:** `portal-protocol` (`rpc.ts` `SendMessageResult`, `message.ts` `PortalMessage.partOf` + `PortalAttachment.duration/waveform`), `portal-relay` (`discord-markdown.ts` NEW, `relay.ts`, `webhook-pool.ts`, `message-store.ts`, `discord-bot.ts`), `portal-mcpl` (`server.ts`, `agent-state.ts`, `tools.ts`, `agent.ts`)
+- **Protocol version:** additive only — no `PORTAL_PROTOCOL_VERSION` bump (older clients never see the new optional fields).
+- **Provenance:** ports chapterx `fix/markdown-across-message-splits` (PR #13) and `feat/audio-input` (PR #12, with membrane PR #29 context) onto portal's relay architecture.
+
+---
+
+## 1. Markdown-preserving split sends (relay)
+
+**Problem.** The relay had no handling for Discord's 2000-char content limit: an
+over-long `send_message` bubbled a raw discord.js error up as `INTERNAL`. And a
+naive split would corrupt markdown — Discord parses each message independently,
+so a code fence or emphasis straddling a boundary breaks everything after it.
+
+**Design.**
+
+- `portal-relay/src/discord-markdown.ts` — the splitter engine, ported verbatim
+  (modulo formatting) from chapterx. `splitPreservingMarkdown(text, 2000)`
+  closes every construct open at a chunk boundary and reopens it in the next
+  chunk, records the synthetic `bridgeOpen`/`bridgeClose` strings per chunk,
+  caps fence info-strings so a reopener can't blow the budget, and avoids
+  fusing synthetic delimiters with adjacent runs. Keep in sync upstream.
+- **Send** (`relay.ts sendMessage`): content ≤ 2000 → unchanged single-send
+  path. Longer → split; parts go out via `WebhookPool.sendMany`, which enqueues
+  ALL parts as one item on the persona's webhook queue so no same-webhook send
+  can interleave between them (single sends delegate to the same path). Files
+  ride on the last part. A mid-sequence failure records the parts that DID land
+  (attributable/editable) and fails the RPC with `DISCORD_ERROR` naming their
+  relay ids, so the caller can delete them or send the remainder.
+- **Result shape**: `SendMessageResult.messageIds?` lists every part (first =
+  `messageId`). `PortalMessage.partOf?` marks continuation parts so clients can
+  group them.
+- **Store** (`message-store.ts`): refs/attribution rows carry
+  `bridgeOpen`/`bridgeClose`/`parts` (first part)/`partOf` (continuations) —
+  persisted, so bridge stripping and whole-set edit/delete survive restarts.
+- **Bridge stripping** (`relay.ts buildPortalMessage` → `stripBridges`): the one
+  choke point where Discord messages become `PortalMessage`s (live events,
+  fetch_history, pins) removes the synthetic markers, so agents always see
+  their original unbroken markdown. A fence reopener whose info string Discord
+  rewrote (mentions in `cleanContent`) is matched on the fence marker run alone.
+- **Echo/record race** (found in live verification): the gateway's create echo
+  for our own webhook message can beat the sending RPC's `store.record` — for
+  split sends it's near-certain (part 1's echo lands while later parts are
+  still posting), which would dispatch live events unstripped and without
+  `partOf`. Two-sided fix: `sendMany` fires an `onSent` callback per part so
+  the relay records each part the moment it exists, and `onDiscordMessage`
+  HOLDS an own-webhook echo with no store ref until the send path records and
+  flushes it (`flushEcho`; 5 s timer as safety net). Symmetrically, surplus
+  parts deleted by an edit leave store cleanup to the gateway's delete echo
+  (like `delete_message`), so the `message_delete` event keeps its relay id.
+- **Edit** (`edit_message`): redirected to the first part; new content is
+  re-split and written across the existing parts; surplus parts are deleted;
+  bridge metadata is rewritten (`setSplitMeta`). Needing MORE parts than the
+  original send is refused with `INVALID_PARAMS` (Discord can't insert messages
+  in place) — shorten, or delete and resend.
+- **Delete** (`delete_message`): fans out over all parts (both own-webhook and
+  moderation paths). Every part is attempted; any failure still surfaces as
+  `DISCORD_ERROR` (never a silent success). A surplus part whose delete fails
+  during an edit keeps its store ref so it stays addressable.
+
+**Verified:** `portal-relay/test/discord-markdown.test.ts` (140 tests, ported
+from chapterx) and `test/split-send.test.ts` (sendMany ordering/contiguity/
+partial failure, onSent per-part callback, store round-trip, stripBridges,
+split→strip reassembly). **Live-verified** (2026-07-08) against a real guild
+with `scripts/rfc6-live.mjs` — 21/21 checks: 4-part split send, stripped live
+events + history + reassembly, `partOf` back-links, shrink edit, collapse with
+surplus deletion, grow-edit refusal, delete fan-out, audio attachment CDN
+round-trip, native 🐘 reaction.
+
+## 2. Inline audio for agents (mcpl) + protocol audio metadata
+
+**Problem.** Audio attachments (voice messages, music files) reached agents only
+as URL text notes — an audio-capable model behind portal could never hear them.
+
+**Design.**
+
+- **Protocol**: `PortalAttachment` gains optional `duration` (seconds) and
+  `waveform` (base64 preview) — Discord voice-message fields the relay now
+  captures from discord.js (`discord-bot.ts`).
+- **Opt-in, per channel, default OFF** (mirrors the RFC-A reactions pattern):
+  durable `audioChannels` set on `AgentState`, toggled by the new
+  `set_audio_visibility` tool. Content-shaping only — wake behavior unchanged
+  (`chat:has-audio` tagging was already there; it now also catches
+  extension-detected audio with no content-type).
+- **Inlining** (`server.ts buildContent`): when the channel is opted in, audio
+  attachments are fetched (15s timeout) and delivered as MCPL
+  `{type:'audio', data, mimeType}` blocks (`mcpl-core` already declares
+  `AudioContent`). Detection: `audio/*` content-type OR filename extension
+  (Discord's `contentType` is optional). MIME is normalized (MP3 aliases
+  `audio/mpeg`, `audio/mpeg3`, `audio/x-mpeg-3` → `audio/mp3`; parameters
+  stripped) so provider layers (e.g. membrane ≥ PR #29 behind connectome) can
+  map it. Caps: 12 MiB raw per clip (~16 MiB base64), max 2 clips per message;
+  over-cap or failed fetches degrade to a text note (now with duration).
+- **Oversized flag**: an opted-in agent seeing a >12 MiB clip reacts natively
+  🐘 via the existing `react` RPC (shared-bot reaction — idempotent across
+  multiple agents), so the sender knows nobody can hear it. Ported from
+  chapterx's `oversized_audio_emote`.
+- **Non-goals**: no relay-side audio fetching/caching (the relay stays a thin
+  id/attribution layer; each opted-in agent fetches from the Discord CDN
+  directly), no transcription, no voice-channel support, no Claude Code channel
+  inlining (`server-cc.ts` payloads are plain strings — attachments stay URL
+  lines there).
+
+**Verified:** `portal-mcpl/test/audio.test.ts` (opt-in toggle + serialization,
+MIME normalization, detection fallback chain). **Live-verified end-to-end with
+a real model** (2026-07-08, `scripts/mimo-audio-live.mjs`): a spoken .wav
+posted by one persona reached a MiMo v2.5-driven persona as an mcpl audio
+block (`buildContent`, exported for this), was mapped to an OpenRouter
+`input_audio` part (membrane's adapter shape), and the model quoted the spoken
+sentence verbatim in its reply.

README.md

@@ -138,10 +138,11 @@ Wiring examples: `portal-mcpl/examples/mcpl-servers.json`,
 
 | Capability | Outbound (agent→Discord) | Inbound (Discord→agent) |
 |---|---|---|
-| Text messages | ✅ | ✅ |
+| Text messages | ✅ (over-long sends auto-split, markdown-preserving — RFC-006) | ✅ (split parts served with original markdown restored) |
 | Replies | ✅ (quoted jump-link; webhooks lack native reply) | ✅ `replyToId` resolved |
 | Threads | ✅ (parent webhook + threadId) | ✅ as channels w/ parentId |
 | Images / files | ✅ inline base64 `bytes` (RFC-003; path-files default-off) | ✅ inlined as MCPL image blocks (≤5 MB) + notes for non-images |
+| Audio / voice messages | ✅ generic file `bytes` | ✅ opt-in inline audio blocks (≤12 MB, per-channel `set_audio_visibility` — RFC-006) + `duration`/`waveform` metadata |
 | Mentions | ✅ persona→role; human `@name`→`<@id>` | ✅ role→persona routing + full `mentions` |
 | Reactions | ✅ pseudo + visible | ✅ native `reaction_add`/`reaction_remove` |
 | Typing | ✅ bot-level (anonymous — not per-persona) | n/a |
@@ -173,6 +174,13 @@ greenfield agents:
 
 See `PORTAL-RFC-00{1,2,3}-*.md`.
 
+- **RFC-006 — split sends + inline audio.** Over-long sends are split
+  markdown-preservingly (fences/emphasis closed + reopened per part; agents see
+  their original markdown restored via bridge stripping; edit/delete span all
+  parts). Audio attachments reach opted-in agents as playable MCPL audio blocks
+  with voice-message `duration`/`waveform` on the wire. See
+  `PORTAL-RFC-006-split-sends-and-inline-audio.md`.
+
 ## Design decisions
 
 - **No DMs** — webhooks can't post to DMs. A later web surface + a `/dm-link`

portal-mcpl/src/agent-state.ts

@@ -32,6 +32,8 @@ interface SerializedState {
   subscriptions?: string[];
   /** Channels opted into live reaction visibility (durable, default off). */
   reactionChannels?: string[];
+  /** Channels opted into inline audio delivery (durable, default off). */
+  audioChannels?: string[];
 }
 
 const PREVIEW_LEN = 140;
@@ -50,6 +52,10 @@ export class AgentState {
    *  Reactions from these channels surface in context but NEVER wake the agent
    *  (tagged `chat:reaction`, which matches no wake policy). Durable. */
   private reactionChannels = new Set<string>();
+  /** Channels opted into inline audio (per-channel, default off). When ON,
+   *  audio attachments from the channel are fetched and delivered as MCPL
+   *  audio blocks instead of URL notes. Durable. */
+  private audioChannels = new Set<string>();
   private listeners: Array<() => void> = [];
 
   /** Notify on any persistence-relevant change (watermark / ping / subscription). */
@@ -124,6 +130,26 @@ export class AgentState {
     return [...this.reactionChannels];
   }
 
+  // ── Inline audio (durable, per-channel opt-in; default off) ──
+
+  /** Opt a channel in/out of inline audio delivery. Returns true if changed. */
+  setAudioVisibility(channelId: string, enabled: boolean): boolean {
+    const changed = enabled ? !this.audioChannels.has(channelId) : this.audioChannels.has(channelId);
+    if (!changed) return false;
+    if (enabled) this.audioChannels.add(channelId);
+    else this.audioChannels.delete(channelId);
+    this.emitChange();
+    return true;
+  }
+
+  isAudioVisible(channelId: string): boolean {
+    return this.audioChannels.has(channelId);
+  }
+
+  audioVisibilityList(): string[] {
+    return [...this.audioChannels];
+  }
+
   /** Advance the watermark for a channel (optionally only up to a message),
    *  clearing unseen + pending pings at/under that point. */
   markRead(channelId: string, uptoCreatedAt?: string): void {
@@ -195,6 +221,7 @@ export class AgentState {
       pings: this.pings,
       subscriptions: [...this.subscriptions],
       reactionChannels: [...this.reactionChannels],
+      audioChannels: [...this.audioChannels],
     };
   }
 
@@ -204,6 +231,7 @@ export class AgentState {
     s.pings = data.pings ?? [];
     s.subscriptions = new Set(data.subscriptions ?? []);
     s.reactionChannels = new Set(data.reactionChannels ?? []);
+    s.audioChannels = new Set(data.audioChannels ?? []);
     return s;
   }
 }

portal-mcpl/src/agent.ts

@@ -186,6 +186,16 @@ export class PortalAgent {
           ? `Reaction visibility ON for ${channelId}. Reactions there now appear in your context as they happen (they never wake you). Subscribe to the channel if you aren't already, so its reactions reach you.`
           : `Reaction visibility OFF for ${channelId}.`;
       }
+      case 'set_audio_visibility': {
+        const channelId = str(args.channelId);
+        const enabled = Boolean(args.enabled);
+        // Durable per-channel opt-in; persisted via onChange. Content-shaping
+        // only — delivery/wake behavior is unchanged (see server.ts buildContent).
+        this.state.setAudioVisibility(channelId, enabled);
+        return enabled
+          ? `Inline audio ON for ${channelId}. Audio attachments there now arrive as playable audio blocks (oversized clips still degrade to URL notes).`
+          : `Inline audio OFF for ${channelId}.`;
+      }
       case 'list_pins':
         return this.client.call('list_pins', { channelId: str(args.channelId) });
       case 'get_pending_pings':