feat(plugin): mock harness + end-to-end orchestration test (mocked LLM) + Signal key-routing fix#223
Conversation
…orchestration tests Adds a `mock` harness adapter (adapters/mock.mjs) — a deterministic stand-in for a Claude Code / Codex TUI so the full daemon → route → autoresponder → outbox → send pipeline can be exercised end to end without a real model or terminal. - Registered in mcp/harness.mjs but NEVER auto-detected: detectHarness only selects it via the explicit TINYPLACE_HARNESS=mock override, so the real Claude/Codex paths are untouched. Passes adapter-contract-test. - The LLM is mocked at the one seam the core exposes: adapter.responder. Instead of `claude -p` / `codex exec`, it spawns test/mock-responder.mjs, which parses the injected prompt and writes a deterministic canned reply to the agent outbox — the same side-effecting path the real `auto_reply` MCP tool takes — so the per-agent daemon then encrypts + sends it over Signal. No network, no model. - test/mock-tui.mjs is a trivial launch stand-in for the launcher recipe. Full offline plugin suite green (mcp-smoke needs @modelcontextprotocol/sdk installed; unrelated). Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8
… key
Both sides of the Signal key exchange addressed the relay key routes
(/keys/:cryptoId/{signed-prekey,prekeys,bundle}) with the base64 Ed25519 key.
A base64 key contains `/`, which breaks the single-segment `:cryptoId` route
match and 404s for ~half of all identities (any key whose base64 contains a
slash):
- publishKeyBundle (enableEncryption/publishKeys) intermittently failed to
publish, so peers could never open an X3DH session with the agent;
- encryptEnvelope's getBundle intermittently failed to fetch a peer's bundle.
Address both by the base58 cryptoId (deriveCryptoId(pubkey) == signer.agentId);
the Signal identity key still travels in the body as `identityKey`. Mirrors the
manual publish/fetch flow that already used agentId.
Verified: 8/8 fresh identities publish + fetch a bundle including 6 whose base64
key contains `/` (previously 404); full SDK vitest suite green (354 passed).
Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8
respond-batch's safeArg sanitized msg.from (the reply target) with /[^\w:.-]+/g, on the assumption it was always a base58 agent id. But a messaging address is the base64 Ed25519 key, which contains `+ / =` — safeArg stripped those, mangling the address into an unresolvable string, so the headless autoresponder's auto_reply could never be delivered back to a base64-addressed sender. Allow `+ / = @` in the charset (base64 keys, cryptoIds, and @Handles). None can terminate a double-quoted tool-call argument (only `"`/newline can, and both stay excluded), so the prompt-injection guard is unchanged. Surfaced by the new orchestration-live-e2e: the mock agent's reply sat undeliverable in the outbox until the target address survived intact. Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8
orchestration-live-e2e.mjs drives the WHOLE coding-agent pipe against a real backend with no model and no terminal: a "human" peer (SDK) sets up a contact connection and sends a Signal DM; bob's real per-agent daemon under the `mock` harness drains + decrypts it, routes it headless to the autoresponder, which spawns the mock responder (canned reply) in place of claude -p / codex exec; the daemon encrypts + sends the reply back and the human decrypts it. Asserts the full round trip, that the mock LLM saw the message, and that the relay only ever held ciphertext. - test/ORCHESTRATION.md documents the pieces, the data flow, and the human connect+send flow (CLI + scripted). - mock-responder.mjs gains a debug breadcrumb log (respond-batch runs it with stdio ignored) for headless troubleshooting. Green against the docker-compose backend. Run: API_URL=... node orchestration-live-e2e.mjs (self-links the local SDK). Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8
|
@senamakel is attempting to deploy a commit to the Vezures Team on Vercel. A member of the Team first needs to authorize it. |
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 9 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (11)
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 59719ac2bd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // the `/` breaks the single-segment `:cryptoId` route match (→ 404) for roughly | ||
| // half of all identities. The Signal identity key (base64) still travels in the | ||
| // body as `identityKey`. Mirrors the recipient path and the manual publish flow. | ||
| const keyPathId = this.signer.agentId; |
There was a problem hiding this comment.
This publishes the bundle under signer.agentId, but the web enable flow still calls verifyKeyBundlePublished(encryptionClient, address) (website/src/hooks/use-signal-identity.ts:119), and that helper fetches /keys/${address}/bundle with the base64 messaging key. In the real relay described by this change, /keys/:cryptoId is keyed by the base58 cryptoId, so after a successful publish the probe can 404 and clearPublishProgress() leaves isReady false; browser users cannot enable encrypted messaging. Derive the same cryptoId for the verification fetch.
Useful? React with 👍 / 👎.
| let replyText = null; | ||
| let replyEnvelope = null; | ||
| for (let i = 0; i < 60 && replyText === null; i++) { | ||
| const { messages } = await aliceClient.messages.list(alice).catch(() => ({ messages: [] })); |
There was a problem hiding this comment.
The reply in this flow is addressed to Alice's base64 messaging key: Bob's daemon queues decoded.from from Alice's envelope, and sendMessage preserves raw base64 recipients. Polling messages.list(alice) here asks the relay for Alice's base58 cryptoId mailbox instead, so the new live e2e never observes the reply and times out even when the daemon sent it. Poll aliceSigner.publicKeyBase64 (or listRaw) before manual decryption.
Useful? React with 👍 / 👎.
…ish path Follow-on to the SDK key-routing fix: verifyKeyBundlePublished probed the relay with the base64 messaging key (`/keys/<base64>/bundle`), but the bundle is now published under the base58 cryptoId. The probe 404s after a successful publish, so clearPublishProgress leaves isReady false and browser users can't enable encrypted messaging. Derive the cryptoId (deriveCryptoId(fromBase64(address))) for the verification fetch. (Codex P1 on tinyhumansai#223.) Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8
|
Thanks for the review. Addressed: P1 (key-bundle verification 404) — fixed in P2 (e2e polls the base58 mailbox) — not an issue for this relay. The backend canonicalizes the recipient on both sides: |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c8564b96ea
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // it, but the contract requires a valid prepare(). | ||
| launch: { | ||
| displayHarness: "Mock", | ||
| binary: "node", |
There was a problem hiding this comment.
Because bin/tinyplace.mjs discovers installed harnesses by iterating every adapter and running each adapter's launch.binary --version, declaring the mock binary as node makes it look installed on every machine. In an unforced launch, the chooser will offer “Mock” (or auto-select it when no Claude/Codex binary is present), so users can end up in the inert mock TUI even though this adapter is meant to be explicit-only; mark it non-discoverable or skip it unless TINYPLACE_HARNESS=mock/--harness mock was requested.
Useful? React with 👍 / 👎.
…aw address Follow-up to adb9ca0: verifyKeyBundlePublished now derives the base58 cryptoId for the getBundle probe, so the unit test must expect that derived id (deriveCryptoId(fromBase64(address))) rather than the raw base64 address. Fixes the website "Test" CI job. Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 278d194d80
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // the `/` breaks the single-segment `:cryptoId` route match (→ 404) for roughly | ||
| // half of all identities. The Signal identity key (base64) still travels in the | ||
| // body as `identityKey`. Mirrors the recipient path and the manual publish flow. | ||
| const keyPathId = this.signer.agentId; |
There was a problem hiding this comment.
After publishing under signer.agentId, the rest of the SDK must use that same cryptoId for /keys/:cryptoId routes. I checked the CLI flows and statusFlow still probes ctx.client.keys.health(publicKey) in sdk/typescript/src/cli/workflows.ts:223-226, while raw key-health/prekeys/signed-prekey default to selfPub in sdk/typescript/src/cli/raw.ts:308-320; with the relay semantics described here those calls look in the old base64 bucket, so tinyplace status reports key-health errors and manual pre-key refills rotate/upload keys that peers will not fetch.
Useful? React with 👍 / 👎.
* feat(plugin): mock harness adapter + mock-LLM responder for headless orchestration tests Adds a `mock` harness adapter (adapters/mock.mjs) — a deterministic stand-in for a Claude Code / Codex TUI so the full daemon → route → autoresponder → outbox → send pipeline can be exercised end to end without a real model or terminal. - Registered in mcp/harness.mjs but NEVER auto-detected: detectHarness only selects it via the explicit TINYPLACE_HARNESS=mock override, so the real Claude/Codex paths are untouched. Passes adapter-contract-test. - The LLM is mocked at the one seam the core exposes: adapter.responder. Instead of `claude -p` / `codex exec`, it spawns test/mock-responder.mjs, which parses the injected prompt and writes a deterministic canned reply to the agent outbox — the same side-effecting path the real `auto_reply` MCP tool takes — so the per-agent daemon then encrypts + sends it over Signal. No network, no model. - test/mock-tui.mjs is a trivial launch stand-in for the launcher recipe. Full offline plugin suite green (mcp-smoke needs @modelcontextprotocol/sdk installed; unrelated). Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8 * fix(sdk): address relay key routes by base58 cryptoId, not the base64 key Both sides of the Signal key exchange addressed the relay key routes (/keys/:cryptoId/{signed-prekey,prekeys,bundle}) with the base64 Ed25519 key. A base64 key contains `/`, which breaks the single-segment `:cryptoId` route match and 404s for ~half of all identities (any key whose base64 contains a slash): - publishKeyBundle (enableEncryption/publishKeys) intermittently failed to publish, so peers could never open an X3DH session with the agent; - encryptEnvelope's getBundle intermittently failed to fetch a peer's bundle. Address both by the base58 cryptoId (deriveCryptoId(pubkey) == signer.agentId); the Signal identity key still travels in the body as `identityKey`. Mirrors the manual publish/fetch flow that already used agentId. Verified: 8/8 fresh identities publish + fetch a bundle including 6 whose base64 key contains `/` (previously 404); full SDK vitest suite green (354 passed). Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8 * fix(plugin): don't strip base64 address chars from the auto-reply target respond-batch's safeArg sanitized msg.from (the reply target) with /[^\w:.-]+/g, on the assumption it was always a base58 agent id. But a messaging address is the base64 Ed25519 key, which contains `+ / =` — safeArg stripped those, mangling the address into an unresolvable string, so the headless autoresponder's auto_reply could never be delivered back to a base64-addressed sender. Allow `+ / = @` in the charset (base64 keys, cryptoIds, and @Handles). None can terminate a double-quoted tool-call argument (only `"`/newline can, and both stay excluded), so the prompt-injection guard is unchanged. Surfaced by the new orchestration-live-e2e: the mock agent's reply sat undeliverable in the outbox until the target address survived intact. Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8 * test(plugin): end-to-end orchestration e2e with a mocked LLM orchestration-live-e2e.mjs drives the WHOLE coding-agent pipe against a real backend with no model and no terminal: a "human" peer (SDK) sets up a contact connection and sends a Signal DM; bob's real per-agent daemon under the `mock` harness drains + decrypts it, routes it headless to the autoresponder, which spawns the mock responder (canned reply) in place of claude -p / codex exec; the daemon encrypts + sends the reply back and the human decrypts it. Asserts the full round trip, that the mock LLM saw the message, and that the relay only ever held ciphertext. - test/ORCHESTRATION.md documents the pieces, the data flow, and the human connect+send flow (CLI + scripted). - mock-responder.mjs gains a debug breadcrumb log (respond-batch runs it with stdio ignored) for headless troubleshooting. Green against the docker-compose backend. Run: API_URL=... node orchestration-live-e2e.mjs (self-links the local SDK). Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8 * fix(website): verify key bundle by base58 cryptoId, matching the publish path Follow-on to the SDK key-routing fix: verifyKeyBundlePublished probed the relay with the base64 messaging key (`/keys/<base64>/bundle`), but the bundle is now published under the base58 cryptoId. The probe 404s after a successful publish, so clearPublishProgress leaves isReady false and browser users can't enable encrypted messaging. Derive the cryptoId (deriveCryptoId(fromBase64(address))) for the verification fetch. (Codex P1 on #223.) Claude-Session: https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8 * update version number * feat(rust-sdk): add onboard handoff API * feat(rust-sdk): add OpenHuman compatibility APIs
1 participant
Verifies the whole tiny.place coding-agent orchestration pipe end to end with a mocked LLM and no terminal, and fixes three real bugs that were blocking it.
What's here
A
mockcoding-agent harness (sdk/plugin-tinyplace/adapters/mock.mjs) — a deterministic stand-in for the Claude Code / Codex TUI. It's registered but never auto-detected (only viaTINYPLACE_HARNESS=mock), so the real harness paths are untouched (passesadapter-contract-test). The LLM is mocked at the one seam the core exposes —adapter.responder— which spawnstest/mock-responder.mjs(a canned reply written to the outbox) instead ofclaude -p/codex exec.An end-to-end orchestration test (
sdk/plugin-tinyplace/orchestration-live-e2e.mjs) against a real backend, no model:Asserts the full round trip, that the mock LLM saw the message, and that the relay only ever held ciphertext.
test/ORCHESTRATION.mddocuments the pieces + the human connect/send flow (CLI + scripted).Bugs found & fixed (surfaced by the e2e)
fix(sdk): relay key routes by base58 cryptoId, not the base64 key.EncryptionContext.publishKeyBundle/getBundleaddressed/keys/:cryptoId/*with the base64 Ed25519 key, whose/breaks the single-segment route → 404 for ~half of all identities. Now usesderiveCryptoId(pubkey)(==signer.agentId); identity key still travels in the body. Full SDK vitest suite green (354 passed); 8/8 slash-containing keys now publish.fix(plugin): don't strip base64 chars from the auto-reply target.respond-batch'ssafeArgstripped+/=from the reply address (assumed base58), mangling base64 targets so the headless autoresponder'sauto_replycould never be delivered to a base64-addressed peer. Now preserves address chars while keeping the injection guard.Verification
mcp-smokeneeds@modelcontextprotocol/sdkinstalled — unrelated).vitest rungreen (354 passed, 0 failed).orchestration-live-e2e.mjsgreen + stable (4/4 runs) against the docker-compose backend.Note: the
respond-batchsafeArgbug means the real headless auto-responder was broken for base64-addressed peers, independent of the mock.https://claude.ai/code/session_01EJDqbVAWGNdmx9njh4hME8