You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We lack an end-to-end integration test that proves a SOURCE Phantom.Workspaces session can invoke the GitHub Copilot SDK running on a REMOTE Phantom.Workspaces user-computer-profile (provider github-copilot, CopilotSdkChatClient), such that (i) a source-session tool (workspace-gui / workspace-entity / agent-session / current-session targeting the SOURCE session) executes LOCALLY on the source session, (ii) the SDK's built-in PowerShell/shell tool runs on the REMOTE instance under the SDK session node in the source's transcript, and (iii) the full transcript plus the Copilot SDK session id round-trip through PERSISTENCE on the SOURCE. This is the remote-chat-client-session.md topology: the router (AgentChat, tool dispatch, steering, persistence) stays on the SOURCE; only the SDK chat client is hosted remotely and relayed back via ChatClientOverTransport. Several concrete wiring gaps stand between "a CopilotSdkChatClient can run" and this acceptance scenario; this bug enumerates them.
Desired Test
A hermetic test (or small test class) that constructs a full AgentChat on the SOURCE (local) instance whose IChatClient is a ChatClientOverTransport that reaches a REMOTE (second in-process / loopback) instance where a CopilotSdkChatClient is built by the transport listener from the wire-carried AgentDefinition. The remote side must be reachable via the transport (e.g. an in-process HubRelayHarness or a reverse-http loopback), NOT via ambient network. One user turn must exercise:
Source-session tool executes on the SOURCE. The scripted remote SDK response emits a tool call for a source-targeted tool (e.g. a workspace-gui, workspace-entity, or agent-session/current-session tool targeting the SOURCE AgentSessionId). The tool must be dispatched by the local router to the LOCAL executor (LocalTrustedExecutor), NOT forwarded remotely, and must execute against the source AgentChat's session context (verified via a spy on the source IRunningAgentChatFactory / AgentServices.CurrentSessionContext). Its FunctionResultContent must be delivered back into the same turn.
Built-in PowerShell tool runs on the REMOTE, under the SDK session node. The scripted response also triggers the Copilot SDK's built-in shell/powershell tool. Because the SDK self-invokes its built-in tools where the SDK lives, that tool must execute on the REMOTE user-computer-profile (verified by a marker only observable on the remote ΓÇö e.g. a captured working directory or a spy on the remote shell process factory). The corresponding transcript items must appear on the SOURCE as children of the SDK session's node (root sink, SessionEvent.AgentId == null; no ParentToolCallIdPropertyName), not lost and not attributed to a sub-agent.
History + persistence round-trip on the SOURCE. After the turn, closing and re-opening the AgentChat against the SOURCE's InMemoryAgentPersistenceStore (same shape for MongoDbAgentPersistenceStore) must restore: the user message, assistant text deltas, BOTH tool-call and tool-result items with their parent/child positioning intact, AND the persisted Copilot SDK session id ΓÇö verified by asserting CopilotSdkChatClient.SetResumeSessionId is called with the restored id on the resumed remote client (per AgentChat.cs:373).
Proposed test names (matching the Subject_Scenario_ExpectedOutcome PascalCase convention used by CopilotSdkChatClientTests, CopilotByokTests, and ByokTransportScenarioTests):
Location: this is a transport-scenario test ΓÇö it belongs in features/Phantom.Workspaces.Transport.Tests/Scenarios/ alongside ByokTransportScenarioTests.cs, using HubRelayHarness (features/Phantom.Workspaces.Transport.Tests/Infrastructure/HubRelayHarness.cs) or the equivalent loopback infrastructure to stand up an in-process REMOTE. If the required AgentChat assembly + persistence surface makes it more natural to live under features/Phantom.Workspaces.IntegrationTests/, mirror it there ΓÇö but the primary test MUST exercise a real transport hop and a real AgentChat end-to-end.
Gaps To Fill
Server-side chat-client host is INCOMPLETE and not registered in production.
What exists today: ChatClientTransportListener (features/Phantom.Workspaces.Transport/Chat/ChatClientTransportListener.cs) wraps a pre-builtIChatClient ΓÇö its constructor at :12 takes an IChatClient chatClient and the channel handler at :20-29 simply delegates every chat-client channel to that fixed instance. There is no wire-side path that (a) reads the AgentDefinition off the request payload, or (b) constructs a CopilotSdkChatClient from it via AgentFactory.CreateChatClient / CreateChatClientAsync (features/Phantom.Workspaces.Llm.Core/AgentFactory.cs:210, 224, 301 ΓÇö with the github-copilot provider dispatch at :278 / :353, and the underlying Copilot-SDK client builder starting at CreateGitHubCopilotClient:814+). And the listener is not hosted in production (see docs/design/unified-transport-production-cutover.md ΓÇö the transport server / ReverseExecutionDispatcher at features/Phantom.Workspaces.Transport/ReverseHttp/ReverseExecutionDispatcher.cs do not register a chat-client listener today).
What's missing: a hosted, production-registered path where the REMOTE user-computer-profile receives an AgentDefinition on the chat-client channel and builds a CopilotSdkChatClient for that turn.
What to build: extend ChatClientTransportListener (or introduce a companion listener) to accept an AgentDefinition in the chat-client open payload and construct the executor client via AgentFactory.CreateChatClient(AgentDefinition) / CreateChatClientAsync(...); register the listener on the remote transport server (mirror the ReverseExecutionDispatcher pattern).
Client entry path force-nulls persistence ΓÇö wrong for this topology.
What exists today: TransportTrustedExecutor.CreateAgentChatAsync (features/Phantom.Workspaces.Llm.Core/Transport/TransportTrustedExecutor.cs:43-65) builds the local AgentChat wired to a remote ChatClientOverTransport via services with { ChatClientOverride = chatClient, AgentPersistenceStoreOverride = NullAgentPersistenceStore.Instance } ΓÇö the persistence store is unconditionally replaced with NullAgentPersistenceStore.Instance at :56. For a full remote-executor topology that was correct (persistence lived on the remote); for the remote-chat-client-session.md topology (router + persistence LOCAL, only chat client remote) it is wrong ΓÇö the source's persistence store must be honoured so tool items + CopilotSdkSessionId land on the source.
What to build: distinguish the two topologies at the TransportTrustedExecutor entry point (via ExecutorTopology from the design doc) and, for the "remote chat client only" case, thread the source's real IAgentPersistenceStore through (do not force-null).
Split tool routing / ExecutorTopology is not yet realized.
What exists today: ExecutorTargetResolver (features/Phantom.Workspaces.Llm.Core/Transport/ExecutorTargetResolver.cs) already tags workspace-gui (:14) and workspace-entity (:17) tools as ExecutorTarget.GuiLocal (:41), and provides ForTool / ForKind (:32, :54). Tool dispatch selection is handled by DeferredTrustedExecutorSelector (features/Phantom.Workspaces/Trust/DeferredTrustedExecutorSelector.cs) choosing between LocalTrustedExecutor (features/Phantom.Workspaces.Llm.Core/Trust/LocalTrustedExecutor.cs) and TransportTrustedExecutor.
What's missing: an explicit ExecutorTopology (per remote-chat-client-session.md) that pins all source-session tools (GUI + entity + agent-session/current-session when targeting the source) to LocalTrustedExecutor while OTHER tools travel over the transport. Note: the Copilot SDK's OWN built-in tools (shell/powershell) are self-invoked by the SDK where the SDK lives (i.e. on the REMOTE), not routed through our ITrustedExecutor at all ΓÇö so surfacing them under the SDK session node depends entirely on the adapter (see gap 5 and Copilot SDK adapter silently drops unhandled SDK session events (no default arm; self-invoke bypasses framework tool loop) #1312).
What to build: an ExecutorTopology selector honoured by DeferredTrustedExecutorSelector that implements the split routing above, plus a test-facing knob to instantiate the "router local, chat client remote" topology in the harness.
No hermetic remote Copilot SDK BYOK harness.
What exists today: CopilotSdkChatClient construction of the SDK is hard-wired to spawn an external copilot process ΓÇö CopilotSdkChatClient.cs:483 calls RuntimeConnection.ForStdio(this.cliPath) and :486 calls new CopilotClient(clientOptions) (mirrored at :1157-1182 in ListModelsAsync). The only end-to-end Copilot-SDK BYOK test, CopilotByokTests.CopilotProvider_Byok_AgainstTestServer_EndToEnd (features/Phantom.Workspaces.Llm.Core.Tests/CopilotByokTests.cs:109-137), is explicitly opt-in via COPILOT_BYOK_E2E and requires a real copilot CLI located by CopilotCliLocator (features/Phantom.Workspaces.Llm.Core.Tests/CopilotCliLocator.cs:16-51).
What's missing: a deterministic way to drive the REMOTE Copilot SDK session without a real copilot.exe. Options: (a) inject an ICopilotClient / CopilotSession seam so the remote side can supply a fake session producing scripted SessionEvents; (b) an in-process fake CLI (stdio protocol test double) that RuntimeConnection.ForStdio can talk to on the remote instance.
What to build: a ScriptedCopilotSdkSession (or FakeCopilotClient) test double + factory seam on CopilotSdkChatClient so tests can inject it into the REMOTE-side instance. Helper methods: EnqueueAssistantDelta, EnqueueToolCall(name, args), EnqueueToolResult(...), EnqueueBuiltinShellCall(...), EnqueueSessionIdle, plus a stable SessionEstablished id.
Built-in tool node hierarchy ΓÇö adapter is fragile and remote-under-session-node is unproven.
What exists today: CopilotSdkStreamAdapter.TranslateCopilotSdkSessionEvents maps ToolExecutionStartEvent (features/Phantom.Workspaces.Llm.Core/CopilotSdkStreamAdapter.cs:113) and ToolExecutionCompleteEvent (:121) into ChatResponseUpdates tagged with event.AgentId; the outer switch (sessionEvent) at :93 has no default arm, silently dropping unmapped events ΓÇö the exact class of bug tracked by Copilot SDK adapter silently drops unhandled SDK session events (no default arm; self-invoke bypasses framework tool loop) #1312. CopilotSubAgentRouter (features/Phantom.Workspaces.Llm.Core/CopilotSubAgentRouter.cs:27-40) routes items with a non-null ParentToolCallId to the correct sub-agent sink; root-tagged items go to the SDK session node.
What to build: a scripted event sequence on the REMOTE that emits root-AgentId shell tool-execution events; on the SOURCE, assert the resulting transcript items are on the root/session sink and appear as children of the SDK session node in AgentChat history.
Persistence of tool items round-trip on the SOURCE is untested.
What exists today: IncrementalPersistenceChatHistoryProvider.SetCopilotSdkSessionId (features/Phantom.Workspaces.Llm.Core/IncrementalPersistenceChatHistoryProvider.cs:48) and the CopilotSdkSessionId writes on PersistedAgent (:107, :143); resume wiring in AgentChat at features/Phantom.Workspaces.Llm.Core/AgentChat.cs:373 (SetResumeSessionId), :374 / :378 (SetCopilotSdkSessionId), and :377 (SessionEstablished subscription).
What's missing: no test round-trips a REMOTE-chat-client turn (whose tool items must be persisted on the SOURCE) through the source's IAgentPersistenceStore and verifies (a) full transcript equality including both tool-call and tool-result items with their ParentToolCallId tags, (b) that SetResumeSessionId is invoked on the resumed REMOTE client with the previously-established SDK session id.
What to build: as part of the harness, close and re-open the source AgentChat against the same InMemoryAgentPersistenceStore, assert full transcript restoration, and spy on the resumed remote client to confirm SetResumeSessionId receives the correct id.
ByokTransportScenarioTests doesn't cover Copilot SDK BYOK, tools, or persistence.
What exists today: features/Phantom.Workspaces.Transport.Tests/Scenarios/ByokTransportScenarioTests.cs hard-codes "provider": "github-models" (not github-copilot) in the AgentDefinition JSON (:24); CreateByokExecutor (:29-41) queues one streaming text update on a DeterministicTestChatClient and returns a bare IChatClient ΓÇö no AgentChat, no tool registration, no persistence, no assertion about tool routing. Combined with the server-side gap in Bump actions/checkout from 4 to 7 #1 above, the transport layer has zero coverage for the Copilot SDK BYOK + AgentChat + tools + persistence path.
What to build: the desired test above.
Affected / Relevant Files
| File | Role |
|---|---|
| features/Phantom.Workspaces.Transport/Chat/ChatClientTransportListener.cs | Server-side chat-client host. Today wraps a pre-built IChatClient (ctor :12); needs to build a CopilotSdkChatClient from a wire-carried AgentDefinition. |
| features/Phantom.Workspaces.Transport/ReverseHttp/ReverseExecutionDispatcher.cs | Transport server dispatch; must register the chat-client listener for production. |
| features/Phantom.Workspaces.Llm.Core/Transport/TransportTrustedExecutor.cs | Client entry (CreateAgentChatAsync:43-65). Force-nulls AgentPersistenceStoreOverride at :56 ΓÇö must NOT for this topology. |
| features/Phantom.Workspaces/Trust/DeferredTrustedExecutorSelector.cs | Chooses LocalTrustedExecutor vs TransportTrustedExecutor per tool call; hosts the ExecutorTopology decision. |
| features/Phantom.Workspaces.Llm.Core/Trust/LocalTrustedExecutor.cs | Local executor for source-session tools. |
| features/Phantom.Workspaces.Llm.Core/CopilotSdkChatClient.cs | The Copilot SDK adapter. Hard-wires RuntimeConnection.ForStdio at :483 and new CopilotClient(...) at :486 (mirrored :1166 / :1182); forwards ChatOptions.Tools at :308. Needs a factory seam. |
| features/Phantom.Workspaces.Llm.Core/AgentFactory.cs | CreateChatClient / CreateChatClientAsync (:210, :224, :301); github-copilot dispatch at :278 / :353; Copilot SDK client builder from :814+. Called by the remote listener to construct the SDK client from AgentDefinition. |
| features/Phantom.Workspaces.Llm.Core/CopilotSdkStreamAdapter.cs | Event → ChatResponseUpdate translation. switch at :93 (no default arm — #1312); tool-execution events at :113 / :121. |
| features/Phantom.Workspaces.Llm.Core/IncrementalPersistenceChatHistoryProvider.cs | Persists tool items + CopilotSdkSessionId (:48, :107, :143). Must run on the SOURCE. |
| features/Phantom.Workspaces.Transport.Tests/Infrastructure/HubRelayHarness.cs | In-process loopback harness for the transport; foundation for the REMOTE side in the desired test. |
| features/Phantom.Workspaces.Transport.Tests/Scenarios/ByokTransportScenarioTests.cs | Provider is github-models (~:24), bare IChatClient, no tools/persistence ΓÇö evidence of the coverage gap. |
| features/Phantom.Workspaces.Llm.Core.Tests/CopilotByokTests.cs | Only real Copilot-SDK BYOK e2e path (:109-137), opt-in via COPILOT_BYOK_E2E. |
| features/Phantom.Workspaces.Llm.Core.Tests/CopilotCliLocator.cs | Locates the real copilot CLI for e2e tests. |
| features/Phantom.Workspaces.Llm.Core.Tests/CopilotSdkChatClientTests.cs | Hand-fed Channel<ChatResponseUpdate> ΓÇö proves an update-channel seam exists but not one at CopilotClient/CopilotSession. |
| docs/design/remote-chat-client-session.md | The topology design this bug is the acceptance test for. |
| docs/design/unified-transport-production-cutover.md | Production hosting/registration of transport listeners. |
Design / Fix (to enable the test)
Concrete pieces, in order:
Complete + host the server-side chat-client listener. Extend ChatClientTransportListener to accept an AgentDefinition in the chat-client open payload and build the executor client via AgentFactory.CreateChatClient(AgentDefinition) / CreateChatClientAsync(...) (dispatch through github-copilot at AgentFactory.cs:278 / :353 → Copilot SDK client builder at :814+). Register the listener on the transport server / ReverseExecutionDispatcher per docs/design/unified-transport-production-cutover.md.
Keep source persistence LOCAL for this topology. In TransportTrustedExecutor.CreateAgentChatAsync, drop the unconditional AgentPersistenceStoreOverride = NullAgentPersistenceStore.Instance (:56) when the resolved ExecutorTopology is "router-local, chat-client-remote"; thread the source IAgentPersistenceStore through instead.
Wire the split executor topology. Introduce/honour an ExecutorTopology per docs/design/remote-chat-client-session.md: source-session tools (workspace-gui, workspace-entity, and agent-session / current-session when targeting the source) resolve to LocalTrustedExecutor; other tools resolve to TransportTrustedExecutor. The SDK's own built-in tools self-execute on the remote instance and are surfaced via the adapter (see Tool visualization: factory-based tool render with inline inspector #4/Slash commands: registry, /cwd handler, and chat-input completion #5).
Introduce a hermetic remote Copilot SDK harness. Refactor CopilotSdkChatClient so CopilotClient construction (:486, :1182) goes through an internal factory delegate an injected ICopilotClientFactory (or ctor-injected Func<CopilotClientOptions, ICopilotClient>) can override. Build a ScriptedCopilotSdkSession in features/Phantom.Workspaces.Llm.Core.Tests/ that plays a queued script of SessionEvents (including root-AgentId built-in-tool events) and emits a stable SessionEstablished id.
Persistence round-trip on the SOURCE. Close/re-open the source AgentChat against InMemoryAgentPersistenceStore; assert transcript equality including both tool items with tags; assert CopilotSdkChatClient.SetResumeSessionId is invoked on the resumed REMOTE client with the previously-established SDK session id (AgentChat.cs:373).
Dependencies / cross-references: docs/design/remote-chat-client-session.md (the topology design), #1312 (dropped-events fix is a prerequisite for asserting unknown-event surfacing), #1306 (agent-session toolset registration ΓÇö exercised in-situ by test 1), #1308 (get_current_session parent/session reporting ΓÇö informs test 1's assertions), #1309 (route CreateAgentChatAsync through IRunningAgentChatFactory).
Expected Tests
| Test | Asserts |
|---|---|
| RemoteCopilotSdkSession_SourceSessionToolInvokedOnSourceInstance | Given a SOURCE AgentChat whose IChatClient is a ChatClientOverTransport reaching a REMOTE in-process instance running CopilotSdkChatClient (built by the extended ChatClientTransportListener from the wire-carried AgentDefinition), and a source-targeted tool (workspace-gui / workspace-entity / agent-session targeting the source) registered on the SOURCE, a scripted SDK tool call for that tool is dispatched by the source's router to LocalTrustedExecutor (verified via spy) and executes against the SOURCE session's AgentServices.CurrentSessionContext / IRunningAgentChatFactory, NOT on the remote instance. The FunctionResultContent is delivered back into the same turn on the SOURCE. |
| RemoteCopilotSdkSession_BuiltinPowerShellToolRunsOnRemoteUnderSessionNode | Given the same setup, a scripted ToolExecutionStartEvent / ToolExecutionCompleteEvent pair with root AgentId == null for a built-in shell/powershell tool executes on the REMOTE instance (verified by a REMOTE-only marker, e.g. spy on the remote shell/PowerShell factory or a captured remote working directory), and the resulting transcript items on the SOURCE are written to the root/session sink (no CopilotSdkStreamAdapter.ParentToolCallIdPropertyName), appear as children of the SDK session node in AgentChat history, and are not dropped by the adapter. |
| RemoteCopilotSdkSession_HistoryAndPersistenceRoundTripOnSource | After the turn, closing and re-opening the SOURCE AgentChat against the SOURCE's InMemoryAgentPersistenceStore restores the full transcript: user message, assistant text, BOTH tool-call and tool-result items (source-local tool AND remote built-in tool) with ParentToolCallId tags intact. On restore, CopilotSdkChatClient.SetResumeSessionId (per AgentChat.cs:373) is invoked on the resumed REMOTE client with the SDK session id established during the original turn, and PersistedAgent.CopilotSdkSessionId matches. |
Acceptance = all three pass with a hermetic loopback transport (HubRelayHarness or equivalent) and a scripted remote Copilot SDK session; no ambient network, no real copilot.exe.
Relationship to design
This bug is the end-to-end acceptance test for docs/design/remote-chat-client-session.md ΓÇö the "local router + remote Copilot SDK chat client + split tool execution" topology. Source-session tools (GUI, workspace-entity, and current/agent-session tools targeting the source) execute on the SOURCE via LocalTrustedExecutor; the SDK's built-in tools self-invoke on the REMOTE where the SDK lives; the router, transcript, and persistence stay on the SOURCE; and the SDK session id round-trips through the SOURCE's persistence. Closing this bug demonstrates the design works end-to-end.
Considered / Background
An earlier framing of this bug scoped the test as a purely LOCAL Copilot SDK BYOK AgentChat (no transport hop) in features/Phantom.Workspaces.Llm.Core.Tests/, with proposed names CopilotSdkSession_ByokScripted_*. That scenario is a strict subset of the coverage above and is retained here only as motivation ΓÇö the acceptance target is the REMOTE topology described above.
Implementation Sub-Items
In dependency order (items at the same level may be done in parallel):
Summary
We lack an end-to-end integration test that proves a SOURCE Phantom.Workspaces session can invoke the GitHub Copilot SDK running on a REMOTE Phantom.Workspaces user-computer-profile (provider
github-copilot,CopilotSdkChatClient), such that (i) a source-session tool (workspace-gui/workspace-entity/agent-session/current-sessiontargeting the SOURCE session) executes LOCALLY on the source session, (ii) the SDK's built-in PowerShell/shell tool runs on the REMOTE instance under the SDK session node in the source's transcript, and (iii) the full transcript plus the Copilot SDK session id round-trip through PERSISTENCE on the SOURCE. This is theremote-chat-client-session.mdtopology: the router (AgentChat, tool dispatch, steering, persistence) stays on the SOURCE; only the SDK chat client is hosted remotely and relayed back viaChatClientOverTransport. Several concrete wiring gaps stand between "aCopilotSdkChatClientcan run" and this acceptance scenario; this bug enumerates them.Desired Test
A hermetic test (or small test class) that constructs a full
AgentChaton the SOURCE (local) instance whoseIChatClientis aChatClientOverTransportthat reaches a REMOTE (second in-process / loopback) instance where aCopilotSdkChatClientis built by the transport listener from the wire-carriedAgentDefinition. The remote side must be reachable via the transport (e.g. an in-processHubRelayHarnessor a reverse-http loopback), NOT via ambient network. One user turn must exercise:Source-session tool executes on the SOURCE. The scripted remote SDK response emits a tool call for a source-targeted tool (e.g. a
workspace-gui,workspace-entity, oragent-session/current-sessiontool targeting the SOURCEAgentSessionId). The tool must be dispatched by the local router to the LOCAL executor (LocalTrustedExecutor), NOT forwarded remotely, and must execute against the sourceAgentChat's session context (verified via a spy on the sourceIRunningAgentChatFactory/AgentServices.CurrentSessionContext). ItsFunctionResultContentmust be delivered back into the same turn.Built-in PowerShell tool runs on the REMOTE, under the SDK session node. The scripted response also triggers the Copilot SDK's built-in shell/powershell tool. Because the SDK self-invokes its built-in tools where the SDK lives, that tool must execute on the REMOTE user-computer-profile (verified by a marker only observable on the remote ΓÇö e.g. a captured working directory or a spy on the remote shell process factory). The corresponding transcript items must appear on the SOURCE as children of the SDK session's node (root sink,
SessionEvent.AgentId == null; noParentToolCallIdPropertyName), not lost and not attributed to a sub-agent.History + persistence round-trip on the SOURCE. After the turn, closing and re-opening the
AgentChatagainst the SOURCE'sInMemoryAgentPersistenceStore(same shape forMongoDbAgentPersistenceStore) must restore: the user message, assistant text deltas, BOTH tool-call and tool-result items with their parent/child positioning intact, AND the persisted Copilot SDK session id ΓÇö verified by assertingCopilotSdkChatClient.SetResumeSessionIdis called with the restored id on the resumed remote client (perAgentChat.cs:373).Proposed test names (matching the
Subject_Scenario_ExpectedOutcomePascalCase convention used byCopilotSdkChatClientTests,CopilotByokTests, andByokTransportScenarioTests):RemoteCopilotSdkSession_SourceSessionToolInvokedOnSourceInstanceRemoteCopilotSdkSession_BuiltinPowerShellToolRunsOnRemoteUnderSessionNodeRemoteCopilotSdkSession_HistoryAndPersistenceRoundTripOnSourceLocation: this is a transport-scenario test ΓÇö it belongs in
features/Phantom.Workspaces.Transport.Tests/Scenarios/alongsideByokTransportScenarioTests.cs, usingHubRelayHarness(features/Phantom.Workspaces.Transport.Tests/Infrastructure/HubRelayHarness.cs) or the equivalent loopback infrastructure to stand up an in-process REMOTE. If the requiredAgentChatassembly + persistence surface makes it more natural to live underfeatures/Phantom.Workspaces.IntegrationTests/, mirror it there ΓÇö but the primary test MUST exercise a real transport hop and a realAgentChatend-to-end.Gaps To Fill
Server-side chat-client host is INCOMPLETE and not registered in production.
What exists today:
ChatClientTransportListener(features/Phantom.Workspaces.Transport/Chat/ChatClientTransportListener.cs) wraps a pre-builtIChatClientΓÇö its constructor at:12takes anIChatClient chatClientand the channel handler at:20-29simply delegates everychat-clientchannel to that fixed instance. There is no wire-side path that (a) reads theAgentDefinitionoff the request payload, or (b) constructs aCopilotSdkChatClientfrom it viaAgentFactory.CreateChatClient/CreateChatClientAsync(features/Phantom.Workspaces.Llm.Core/AgentFactory.cs:210, 224, 301ΓÇö with thegithub-copilotprovider dispatch at:278/:353, and the underlying Copilot-SDK client builder starting atCreateGitHubCopilotClient:814+). And the listener is not hosted in production (seedocs/design/unified-transport-production-cutover.mdΓÇö the transport server /ReverseExecutionDispatcheratfeatures/Phantom.Workspaces.Transport/ReverseHttp/ReverseExecutionDispatcher.csdo not register a chat-client listener today).What's missing: a hosted, production-registered path where the REMOTE user-computer-profile receives an
AgentDefinitionon thechat-clientchannel and builds aCopilotSdkChatClientfor that turn.What to build: extend
ChatClientTransportListener(or introduce a companion listener) to accept anAgentDefinitionin thechat-clientopen payload and construct the executor client viaAgentFactory.CreateChatClient(AgentDefinition)/CreateChatClientAsync(...); register the listener on the remote transport server (mirror theReverseExecutionDispatcherpattern).Client entry path force-nulls persistence ΓÇö wrong for this topology.
What exists today:
TransportTrustedExecutor.CreateAgentChatAsync(features/Phantom.Workspaces.Llm.Core/Transport/TransportTrustedExecutor.cs:43-65) builds the localAgentChatwired to a remoteChatClientOverTransportviaservices with { ChatClientOverride = chatClient, AgentPersistenceStoreOverride = NullAgentPersistenceStore.Instance }ΓÇö the persistence store is unconditionally replaced withNullAgentPersistenceStore.Instanceat:56. For a full remote-executor topology that was correct (persistence lived on the remote); for theremote-chat-client-session.mdtopology (router + persistence LOCAL, only chat client remote) it is wrong ΓÇö the source's persistence store must be honoured so tool items +CopilotSdkSessionIdland on the source.What to build: distinguish the two topologies at the
TransportTrustedExecutorentry point (viaExecutorTopologyfrom the design doc) and, for the "remote chat client only" case, thread the source's realIAgentPersistenceStorethrough (do not force-null).Split tool routing /
ExecutorTopologyis not yet realized.What exists today:
ExecutorTargetResolver(features/Phantom.Workspaces.Llm.Core/Transport/ExecutorTargetResolver.cs) already tagsworkspace-gui(:14) andworkspace-entity(:17) tools asExecutorTarget.GuiLocal(:41), and providesForTool/ForKind(:32,:54). Tool dispatch selection is handled byDeferredTrustedExecutorSelector(features/Phantom.Workspaces/Trust/DeferredTrustedExecutorSelector.cs) choosing betweenLocalTrustedExecutor(features/Phantom.Workspaces.Llm.Core/Trust/LocalTrustedExecutor.cs) andTransportTrustedExecutor.What's missing: an explicit
ExecutorTopology(perremote-chat-client-session.md) that pins all source-session tools (GUI + entity +agent-session/current-sessionwhen targeting the source) toLocalTrustedExecutorwhile OTHER tools travel over the transport. Note: the Copilot SDK's OWN built-in tools (shell/powershell) are self-invoked by the SDK where the SDK lives (i.e. on the REMOTE), not routed through ourITrustedExecutorat all ΓÇö so surfacing them under the SDK session node depends entirely on the adapter (see gap 5 and Copilot SDK adapter silently drops unhandled SDK session events (no default arm; self-invoke bypasses framework tool loop) #1312).What to build: an
ExecutorTopologyselector honoured byDeferredTrustedExecutorSelectorthat implements the split routing above, plus a test-facing knob to instantiate the "router local, chat client remote" topology in the harness.No hermetic remote Copilot SDK BYOK harness.
What exists today:
CopilotSdkChatClientconstruction of the SDK is hard-wired to spawn an externalcopilotprocess ΓÇöCopilotSdkChatClient.cs:483callsRuntimeConnection.ForStdio(this.cliPath)and:486callsnew CopilotClient(clientOptions)(mirrored at:1157-1182inListModelsAsync). The only end-to-end Copilot-SDK BYOK test,CopilotByokTests.CopilotProvider_Byok_AgainstTestServer_EndToEnd(features/Phantom.Workspaces.Llm.Core.Tests/CopilotByokTests.cs:109-137), is explicitly opt-in viaCOPILOT_BYOK_E2Eand requires a realcopilotCLI located byCopilotCliLocator(features/Phantom.Workspaces.Llm.Core.Tests/CopilotCliLocator.cs:16-51).What's missing: a deterministic way to drive the REMOTE Copilot SDK session without a real
copilot.exe. Options: (a) inject anICopilotClient/CopilotSessionseam so the remote side can supply a fake session producing scriptedSessionEvents; (b) an in-process fake CLI (stdio protocol test double) thatRuntimeConnection.ForStdiocan talk to on the remote instance.What to build: a
ScriptedCopilotSdkSession(orFakeCopilotClient) test double + factory seam onCopilotSdkChatClientso tests can inject it into the REMOTE-side instance. Helper methods:EnqueueAssistantDelta,EnqueueToolCall(name, args),EnqueueToolResult(...),EnqueueBuiltinShellCall(...),EnqueueSessionIdle, plus a stableSessionEstablishedid.Built-in tool node hierarchy ΓÇö adapter is fragile and remote-under-session-node is unproven.
What exists today:
CopilotSdkStreamAdapter.TranslateCopilotSdkSessionEventsmapsToolExecutionStartEvent(features/Phantom.Workspaces.Llm.Core/CopilotSdkStreamAdapter.cs:113) andToolExecutionCompleteEvent(:121) intoChatResponseUpdates tagged withevent.AgentId; the outerswitch (sessionEvent)at:93has nodefaultarm, silently dropping unmapped events ΓÇö the exact class of bug tracked by Copilot SDK adapter silently drops unhandled SDK session events (no default arm; self-invoke bypasses framework tool loop) #1312.CopilotSubAgentRouter(features/Phantom.Workspaces.Llm.Core/CopilotSubAgentRouter.cs:27-40) routes items with a non-nullParentToolCallIdto the correct sub-agent sink; root-tagged items go to the SDK session node.What's missing: an end-to-end assertion that a REMOTE-emitted built-in shell/powershell
ToolExecutionStart/CompleteEvent(rootAgentId) survives the transport hop and lands on the SOURCE as children of the SDK session node (root sink), and that new/unknown event kinds are not silently dropped (blocked on Copilot SDK adapter silently drops unhandled SDK session events (no default arm; self-invoke bypasses framework tool loop) #1312).What to build: a scripted event sequence on the REMOTE that emits root-
AgentIdshell tool-execution events; on the SOURCE, assert the resulting transcript items are on the root/session sink and appear as children of the SDK session node inAgentChathistory.Persistence of tool items round-trip on the SOURCE is untested.
What exists today:
IncrementalPersistenceChatHistoryProvider.SetCopilotSdkSessionId(features/Phantom.Workspaces.Llm.Core/IncrementalPersistenceChatHistoryProvider.cs:48) and theCopilotSdkSessionIdwrites onPersistedAgent(:107,:143); resume wiring inAgentChatatfeatures/Phantom.Workspaces.Llm.Core/AgentChat.cs:373(SetResumeSessionId),:374/:378(SetCopilotSdkSessionId), and:377(SessionEstablishedsubscription).What's missing: no test round-trips a REMOTE-chat-client turn (whose tool items must be persisted on the SOURCE) through the source's
IAgentPersistenceStoreand verifies (a) full transcript equality including both tool-call and tool-result items with theirParentToolCallIdtags, (b) thatSetResumeSessionIdis invoked on the resumed REMOTE client with the previously-established SDK session id.What to build: as part of the harness, close and re-open the source
AgentChatagainst the sameInMemoryAgentPersistenceStore, assert full transcript restoration, and spy on the resumed remote client to confirmSetResumeSessionIdreceives the correct id.ByokTransportScenarioTestsdoesn't cover Copilot SDK BYOK, tools, or persistence.What exists today:
features/Phantom.Workspaces.Transport.Tests/Scenarios/ByokTransportScenarioTests.cshard-codes"provider": "github-models"(notgithub-copilot) in theAgentDefinitionJSON (:24);CreateByokExecutor(:29-41) queues one streaming text update on aDeterministicTestChatClientand returns a bareIChatClientΓÇö noAgentChat, no tool registration, no persistence, no assertion about tool routing. Combined with the server-side gap in Bump actions/checkout from 4 to 7 #1 above, the transport layer has zero coverage for the Copilot SDK BYOK + AgentChat + tools + persistence path.What to build: the desired test above.
Affected / Relevant Files
| File | Role |
|---|---|
|
features/Phantom.Workspaces.Transport/Chat/ChatClientTransportListener.cs| Server-side chat-client host. Today wraps a pre-builtIChatClient(ctor:12); needs to build aCopilotSdkChatClientfrom a wire-carriedAgentDefinition. ||
features/Phantom.Workspaces.Transport/ReverseHttp/ReverseExecutionDispatcher.cs| Transport server dispatch; must register the chat-client listener for production. ||
features/Phantom.Workspaces.Llm.Core/Transport/TransportTrustedExecutor.cs| Client entry (CreateAgentChatAsync:43-65). Force-nullsAgentPersistenceStoreOverrideat:56ΓÇö must NOT for this topology. ||
features/Phantom.Workspaces.Llm.Core/Transport/ExecutorTargetResolver.cs| Tagsworkspace-gui/workspace-entitytools asGuiLocal(:14,:17,:41). ||
features/Phantom.Workspaces/Trust/DeferredTrustedExecutorSelector.cs| ChoosesLocalTrustedExecutorvsTransportTrustedExecutorper tool call; hosts theExecutorTopologydecision. ||
features/Phantom.Workspaces.Llm.Core/Trust/LocalTrustedExecutor.cs| Local executor for source-session tools. ||
features/Phantom.Workspaces.Llm.Core/CopilotSdkChatClient.cs| The Copilot SDK adapter. Hard-wiresRuntimeConnection.ForStdioat:483andnew CopilotClient(...)at:486(mirrored:1166/:1182); forwardsChatOptions.Toolsat:308. Needs a factory seam. ||
features/Phantom.Workspaces.Llm.Core/AgentFactory.cs|CreateChatClient/CreateChatClientAsync(:210,:224,:301);github-copilotdispatch at:278/:353; Copilot SDK client builder from:814+. Called by the remote listener to construct the SDK client fromAgentDefinition. ||
features/Phantom.Workspaces.Llm.Core/CopilotSdkStreamAdapter.cs| Event →ChatResponseUpdatetranslation.switchat:93(no default arm — #1312); tool-execution events at:113/:121. ||
features/Phantom.Workspaces.Llm.Core/CopilotSubAgentRouter.cs| RoutesParentToolCallId-tagged items to sub-agent sinks; root sink handles SDK-session-level items. ||
features/Phantom.Workspaces.Llm.Core/IncrementalPersistenceChatHistoryProvider.cs| Persists tool items +CopilotSdkSessionId(:48,:107,:143). Must run on the SOURCE. ||
features/Phantom.Workspaces.Llm.Core/AgentChat.cs| Resume wiring:373(SetResumeSessionId),:374/:378(SetCopilotSdkSessionId),:377(SessionEstablished). ||
features/Phantom.Workspaces.Transport.Tests/Infrastructure/HubRelayHarness.cs| In-process loopback harness for the transport; foundation for the REMOTE side in the desired test. ||
features/Phantom.Workspaces.Transport.Tests/Scenarios/ByokTransportScenarioTests.cs| Provider isgithub-models(~:24), bareIChatClient, no tools/persistence ΓÇö evidence of the coverage gap. ||
features/Phantom.Workspaces.Llm.Core.Tests/CopilotByokTests.cs| Only real Copilot-SDK BYOK e2e path (:109-137), opt-in viaCOPILOT_BYOK_E2E. ||
features/Phantom.Workspaces.Llm.Core.Tests/CopilotCliLocator.cs| Locates the realcopilotCLI for e2e tests. ||
features/Phantom.Workspaces.Llm.Core.Tests/CopilotSdkChatClientTests.cs| Hand-fedChannel<ChatResponseUpdate>ΓÇö proves an update-channel seam exists but not one atCopilotClient/CopilotSession. ||
docs/design/remote-chat-client-session.md| The topology design this bug is the acceptance test for. ||
docs/design/unified-transport-production-cutover.md| Production hosting/registration of transport listeners. |Design / Fix (to enable the test)
Concrete pieces, in order:
Complete + host the server-side chat-client listener. Extend
ChatClientTransportListenerto accept anAgentDefinitionin thechat-clientopen payload and build the executor client viaAgentFactory.CreateChatClient(AgentDefinition)/CreateChatClientAsync(...)(dispatch throughgithub-copilotatAgentFactory.cs:278/:353→ Copilot SDK client builder at:814+). Register the listener on the transport server /ReverseExecutionDispatcherperdocs/design/unified-transport-production-cutover.md.Keep source persistence LOCAL for this topology. In
TransportTrustedExecutor.CreateAgentChatAsync, drop the unconditionalAgentPersistenceStoreOverride = NullAgentPersistenceStore.Instance(:56) when the resolvedExecutorTopologyis "router-local, chat-client-remote"; thread the sourceIAgentPersistenceStorethrough instead.Wire the split executor topology. Introduce/honour an
ExecutorTopologyperdocs/design/remote-chat-client-session.md: source-session tools (workspace-gui,workspace-entity, andagent-session/current-sessionwhen targeting the source) resolve toLocalTrustedExecutor; other tools resolve toTransportTrustedExecutor. The SDK's own built-in tools self-execute on the remote instance and are surfaced via the adapter (see Tool visualization: factory-based tool render with inline inspector #4/Slash commands: registry, /cwd handler, and chat-input completion #5).Introduce a hermetic remote Copilot SDK harness. Refactor
CopilotSdkChatClientsoCopilotClientconstruction (:486,:1182) goes through an internal factory delegate an injectedICopilotClientFactory(or ctor-injectedFunc<CopilotClientOptions, ICopilotClient>) can override. Build aScriptedCopilotSdkSessioninfeatures/Phantom.Workspaces.Llm.Core.Tests/that plays a queued script ofSessionEvents (including root-AgentIdbuilt-in-tool events) and emits a stableSessionEstablishedid.Surface remote built-in tool events under the SDK session node. Fix the adapter default arm (Copilot SDK adapter silently drops unhandled SDK session events (no default arm; self-invoke bypasses framework tool loop) #1312) so unknown events aren't dropped, and assert that root-
AgentIdshell/powershellToolExecutionStart/CompleteEvents arrive at the source's root sink (noParentToolCallIdPropertyName) and appear as children of the SDK session node inAgentChathistory.Persistence round-trip on the SOURCE. Close/re-open the source
AgentChatagainstInMemoryAgentPersistenceStore; assert transcript equality including both tool items with tags; assertCopilotSdkChatClient.SetResumeSessionIdis invoked on the resumed REMOTE client with the previously-established SDK session id (AgentChat.cs:373).Dependencies / cross-references:
docs/design/remote-chat-client-session.md(the topology design), #1312 (dropped-events fix is a prerequisite for asserting unknown-event surfacing), #1306 (agent-session toolset registration ΓÇö exercised in-situ by test 1), #1308 (get_current_sessionparent/session reporting ΓÇö informs test 1's assertions), #1309 (routeCreateAgentChatAsyncthroughIRunningAgentChatFactory).Expected Tests
| Test | Asserts |
|---|---|
|
RemoteCopilotSdkSession_SourceSessionToolInvokedOnSourceInstance| Given a SOURCEAgentChatwhoseIChatClientis aChatClientOverTransportreaching a REMOTE in-process instance runningCopilotSdkChatClient(built by the extendedChatClientTransportListenerfrom the wire-carriedAgentDefinition), and a source-targeted tool (workspace-gui/workspace-entity/agent-sessiontargeting the source) registered on the SOURCE, a scripted SDK tool call for that tool is dispatched by the source's router toLocalTrustedExecutor(verified via spy) and executes against the SOURCE session'sAgentServices.CurrentSessionContext/IRunningAgentChatFactory, NOT on the remote instance. TheFunctionResultContentis delivered back into the same turn on the SOURCE. ||
RemoteCopilotSdkSession_BuiltinPowerShellToolRunsOnRemoteUnderSessionNode| Given the same setup, a scriptedToolExecutionStartEvent/ToolExecutionCompleteEventpair with rootAgentId == nullfor a built-in shell/powershell tool executes on the REMOTE instance (verified by a REMOTE-only marker, e.g. spy on the remote shell/PowerShell factory or a captured remote working directory), and the resulting transcript items on the SOURCE are written to the root/session sink (noCopilotSdkStreamAdapter.ParentToolCallIdPropertyName), appear as children of the SDK session node inAgentChathistory, and are not dropped by the adapter. ||
RemoteCopilotSdkSession_HistoryAndPersistenceRoundTripOnSource| After the turn, closing and re-opening the SOURCEAgentChatagainst the SOURCE'sInMemoryAgentPersistenceStorerestores the full transcript: user message, assistant text, BOTH tool-call and tool-result items (source-local tool AND remote built-in tool) withParentToolCallIdtags intact. On restore,CopilotSdkChatClient.SetResumeSessionId(perAgentChat.cs:373) is invoked on the resumed REMOTE client with the SDK session id established during the original turn, andPersistedAgent.CopilotSdkSessionIdmatches. |Acceptance = all three pass with a hermetic loopback transport (
HubRelayHarnessor equivalent) and a scripted remote Copilot SDK session; no ambient network, no realcopilot.exe.Relationship to design
This bug is the end-to-end acceptance test for
docs/design/remote-chat-client-session.mdΓÇö the "local router + remote Copilot SDK chat client + split tool execution" topology. Source-session tools (GUI, workspace-entity, and current/agent-session tools targeting the source) execute on the SOURCE viaLocalTrustedExecutor; the SDK's built-in tools self-invoke on the REMOTE where the SDK lives; the router, transcript, and persistence stay on the SOURCE; and the SDK session id round-trips through the SOURCE's persistence. Closing this bug demonstrates the design works end-to-end.Considered / Background
An earlier framing of this bug scoped the test as a purely LOCAL Copilot SDK BYOK
AgentChat(no transport hop) infeatures/Phantom.Workspaces.Llm.Core.Tests/, with proposed namesCopilotSdkSession_ByokScripted_*. That scenario is a strict subset of the coverage above and is retained here only as motivation ΓÇö the acceptance target is the REMOTE topology described above.