Add X API research ingestion and familiar publishing

[CompleteDotTech]

Add X API research ingestion and familiar publishing

Implement a local-first X API integration for Cave. Research Desk must be able to retrieve current X post data, save normalized source snapshots, and attach them to familiar-scoped research work. The appropriate familiar workspace room must support drafting and explicitly approved publishing to X without leaking credentials or silently retrying external writes. Scope includes secure account connection, cost-aware bounded reads, durable provenance, room-specific UX, and end-to-end tests.

Bead metadata

• Bead ID: cave-8i8q5

• Status: open

• Priority: P1

• Surface: Missing

• Authoritative owner: GitHub Copilot CLI

• Labels: comms, integrations, research, security, x-api

• Parent: None

• Dependencies: None

Acceptance criteria

An X account/app can be connected without exposing credentials to the browser or familiar subprocesses; Research Desk can look up X post URLs and run bounded recent searches, save normalized durable snapshots, and attach saved posts to a research mission; Comms Operations can draft an X post and publish only after explicit human confirmation, then record the returned post ID and canonical URL; data and actions are correctly familiar-scoped; auth expiry, rate limits, billing exhaustion, malformed responses, deleted posts, and ambiguous write failures are handled fail-closed; focused tests, API contracts, lint, typecheck, design gates, and native-room verification pass.

Source of truth

Beads is authoritative. This draft card is a GitHub Teamwork visibility mirror and should be reconciled from the Bead rather than edited as an independent task.

[CompleteDotTech]

Acceptance audit — verified against code at 63f140139, not against notes

The bead notes have been wrong before, so nothing below is taken on trust. Every verdict is backed by the production code path, and where a claim was testable it was tested.

Verdict: not closable yet. Four gaps were wiring and are fixed in #4858. Three remain and are filed as their own issues.

Per-criterion verdicts

#	Claim	Verdict	Evidence
1a	Connect / refresh / scope upgrade / disconnect	was PARTIAL → fixed in #4858	Connect could not complete: GET /api/x/connection used xOAuthService.status(), which reports only whether a flow is running. familiar-x-section.tsx:604-648 polls it and settles on oauthFlowId/oauthOutcome; absent those it fell through to !next.activeFlow and reported a successful authorization as "X authorization didn't grant the requested permission", skipping the grant save. Refresh is single-flight (x-credentials.ts:391-433), a failed refresh writes nothing (:400-415), rotation is validated (:217-259), writes are atomic (local-encrypted-vault.ts:258-284). Scope upgrade displays granted scopes from the bundle (familiar-x-section.tsx:894) and replaces the bundle only after exchange and account validation (x-oauth.ts:234-251).
1b	Credentials never reach the browser	MET	getConnectionStatus() (x-credentials.ts:448-458) constructs a fresh object with no token member; the type is pinned by a type-level equality assertion (x-credentials.test.ts:31-38) and JSON.stringify is asserted token-free (:1402). The client re-sanitizes with a strict key allowlist (familiar-x-section.tsx:105-162). Zero console.* in the entire X tree.
1c	Credentials never reach a familiar subprocess	MET	Spawn env is built by harnessSpawnEnv() (harness-spawn-env.ts:180-202), which restores only keys enumerated from vault.yaml. X_OAUTH_TOKEN_BUNDLE is not declared there, and setLocalEncryptedSecret never touches process.env. Repo-wide, that key appears only in x-credentials.ts, its test, and the spec. Hardening gap, not a leak: POST /api/vault has no reserved-key denylist, so deliberately creating a mapping of that name would adopt the bundle into shared scope. Also note nothing regression-tests the absence — it holds by construction.
2	Post-URL lookup; bounded recent search	MET	The cap is a hardcoded constant, not a default: x-client.ts:363 sets max_results=10 and the function signature accepts no maxResults at all, so a crafted {maxResults: 100} is never read. Second clamp at :380 (.slice(0, 10)), third client-side. Asserted on a 12-post response at x-client.test.ts:180-191. No pagination — next_token appears only in an allowlist of tolerated meta keys and is discarded. URL parsing accepts x.com/twitter.com, rejects non-post URLs before any request (x-api.ts:87-107), and the route re-derives the id rather than trusting the client.
3	Durable normalized snapshots; no permanent raw copy	was PARTIAL → fixed in #4858	The durable record is exactly the identity-only shape, built field-by-field and read back through a strict key-count check (x-sources.ts:242-262, :600-612); a test smuggles text/author/metrics/raw payload in and asserts none persist (x-sources.test.ts:215-249). But the lifecycle half was unwired: sweepExpiredXCache had zero production callers, so expiry was purely lazy and per-post-id, and a post nobody looked up again kept its text, author id and handle on disk indefinitely. purgeXSourceCache was likewise uncalled, so disconnect left post bodies behind.
4	Mission attach, hydration, and removal after the run	UNMET — filed as cave-v3ajh	Attach works but is a bookmark on the X-side record only. setXSourceMissionAttached writes x-sources/<familiar>.json; the mission's own sources.json is untouched and no sourceType: "x-post" entry is ever created. ResearchSourceRef (research-missions.ts:144-156) has none of provider/externalId/availability, and parseResearchSource is a strict allowlist that would drop them anyway. There is no hydration and no cleanup: runtime/x appears nowhere in src/, research-mission-runner.ts imports nothing from x-sources/x-client, and contains zero finally blocks. A user attaches a post, sees "X source attached to the mission", and the run never sees it. (Not to be confused with research-link-materialization.ts, which handles X articles with the opposite contract.)
5	Draft → explicit confirmation → publish exactly the text shown	MET on the load-bearing property; spec divergences filed as cave-uajyn	The publish route reads familiarId/publicationId/confirmationToken and never touches body.text (publish/route.ts:119-136); publishXPublication has no text parameter at all. The text sent is read from the server's own store and the token recomputed from it under the file lock (x-publications.ts:467-494). postId and canonicalUrl are recorded and durability-tested (:525-532), and a published record missing either is rejected on load. Divergences: no Modal, no connected account and no "No location will be added." in the confirmation view, receipt stores the body rather than a textSha256, and the token is a deterministic HMAC rather than a random one-time expiring one.
6	Familiar scoping of every read and write	MET, with one exception	requireXCapability (x-access.ts:64-71) resolves the binding from persisted config, never from the request, and tests === true so missing/malformed fails closed. All nine /api/x/* handlers gate; no route reads a capability claim from a body or query. Cross-familiar mission access is blocked at sources/route.ts:111-114 with authorize-before-mutate ordering. Exception: GET /api/research/recommendations calls listSavedXSources(familiarId) with no X capability check, so it returns X-derived data for a familiar whose xResearchEnabled is false. Bounded and familiar-scoped, no upstream call — but "every X read rejects a disabled capability" does not hold literally.
7	Fail-closed on the six failure classes	10 of 11 categories MET; the eleventh fixed in #4858	Auth expiry (one refresh, then fail — and writes never refresh post-dispatch), 429 with a real retryAt and no auto-retry, 402, malformed JSON and unexpected schema, and the definite-vs-ambiguous asymmetry on writes all have genuine production paths and real tests. Deleted posts did not: markXPostAvailability purges the cache and marks the record durably, but nothing called it — the 404 was mapped to a response and dropped, and the deletion mark existed only as React state (research-x-sources.tsx:842-847), so it vanished on reload.
8	The exclusions are still excluded	MET	The create-post body is JSON.stringify({ text }), asserted byte-for-byte (x-client.test.ts:202). Repo-wide grep for in_reply_to_tweet_id, quote_tweet_id, media_ids, poll_options, geo.place_id, reply_settings returns zero X hits. Only five X endpoints are touched anywhere and none is a DELETE. createXPost is imported by exactly one file. No familiar-facing X tool exists — the MCP registry lists no X server. Nothing feeds X content into search, embeddings, memory or training; the one consumer (research-topic-recommendations.ts) passes through only id, postId, the user's own note and a timestamp, and never reads the cache.

Repository gates

All six X routes are registered in api-contracts.test.ts:387-392, and that listing is enforced by exact equality against a filesystem walk (:627), not advisory. Test wiring is complete. typecheck, lint, check:tests-wired and build are clean at the audit head.

What #4858 changes

It wires the four functions that shipped with no production caller, and removes src/lib/server/x-connection-route.ts — an unreferenced, untested second implementation of the connection route, extracted alongside its wired sibling x-oauth-start-route.ts and then never bound. The first two defects above are precisely the wiring it carried.

The new tests are behavioural, not more source pins: they drive the real handlers over a temp cave home with no network and no X credentials. That distinction is the root cause here — account-routes.test.ts and research-routes.test.ts scan route source text, and a regex can confirm the calls a file makes but never the call it forgot. Every assertion fails against the previous code, and each fix was individually reverted to confirm a test catches it.

Cannot be verified without live X credentials

No live X call was made and none was attempted. The spec's own release gate — lookup, search, refresh, publish-success and ambiguous-write smoke evidence against X — has no evidence in the repository and remains open. Discharging it needs an OpenCoven test account, a configured client ID, and API credits. Two narrower questions also need live access: whether X rotates the refresh token on every refresh (which determines how fast cave-267oy bites), and whether the hasScopes rejection at x-oauth.ts:236 ever fires in practice.

To close this issue

Land #4858, then cave-v3ajh (mission hydration — the only remaining criterion that is materially unimplemented). cave-uajyn and cave-267oy are a product decision and a concurrency hardening respectively, and neither blocks the acceptance criteria as written.

[CompleteDotTech]

Closure review: keeping this issue open.

PR #4988 currently remains open at head 69cb5bb, with required checks running. The authoritative root Bead cave-8i8q5 remains open, and cave-uajyn remains in progress.

PR #4867 merged the runtime hydration and cleanup path, but its own completion record deliberately leaves the agent-workspace sources.json representation unimplemented; cave-v3ajh therefore remains in progress rather than being closed on merge alone.

The release acceptance evidence is also still missing: no live X lookup, bounded search, refresh, publish-success, or ambiguous-write smoke has been recorded against an OpenCoven test account with configured client credentials and API credits. Native-room verification remains part of the issue acceptance criteria.

Do not close until the open PR and Beads are reconciled and the live X acceptance record exists.