Extend MCP host adapter: HTTP transport, OAuth, full lifecycle beyond stdio MVP

[mertsatilmaz]

PR #107 (merged) added the deterministic local stdio MCP host runtime, including:

• stdio session lifecycle (initialize, list_tools, tool_call, close)
• runtime env/cwd support
• evidence forging protections
• bounded string/collection/depth/schema sizes
• canonical tool naming and server identity preservation

What remains for a full MCP host adapter:

• HTTP transport (Streamable HTTP and/or legacy SSE)
• OAuth / auth flows per the MCP auth spec
• Resources, prompts, sampling beyond tools
• CLI flag exposure (`--mcp-target` currently runs a workflow callable; expose the host runtime directly)
• Live integration tests beyond the deterministic fixture

Scope was deliberately limited in PR #107 to land stdio first.

[Flames4fun]

Hi! I’d like to work on this issue, it looks very interesting and aligned with what I’m learning around MCP and agent runtimes.

Could you please assign it to me if possible? :))

[Flames4fun]

Hi @mertsatilmaz , I wanted to check in on this before opening another PR.

I noticed the issue is still open, and since some time has passed after PR 100 and PR 107, I don’t want to assume that the remaining scope is exactly the same as when I first split the work.

From my side, the way I’m reading the current state is that the config foundation and the deterministic stdio host runtime are already in place. What seems to be missing now is the final user-facing integration around that work: wiring the stdio host path into the CLI/runner flow, documenting the implementation that actually landed, and adding an end-to-end test that exercises the feature from the public entry point instead of only through the internal runtime/test layer.

That is what I was planning to do in PR3: keep it focused on finishing the stdio host adapter path cleanly, with a small docs update and final fixture-based coverage. I would avoid mixing in HTTP transport, OAuth/auth flows, resources/prompts/sampling, or broader live integration tests unless you think one of those should be prioritized now. My concern is mostly reviewability; I think CLI/docs/E2E for the stdio path is already a coherent PR, while adding the rest would make it a much larger change.

Before I start pushing code, could you confirm if that direction makes sense for this issue and the v0.3.0 milestone? Or would you rather have PR3 target a different missing piece first, like HTTP transport or auth?

[mertsatilmaz]

Thanks for the careful check-in, @Flames4fun — and yes, your read is exactly right. #100 (config) and #107 (stdio runtime) are in; the gap is that the stdio host isn't reachable from the public CLI yet (today only --mcp-target, the workflow adapter, is wired). Your proposed PR3 — CLI/runner wiring + docs + one E2E test for the stdio path — is the correct next step and a coherent, reviewable size. Please don't fold in HTTP/OAuth/resources/prompts/sampling; those map to design phases 4–6 and stay under this issue.

Since #96's title is really the HTTP/OAuth/full-lifecycle extension (v0.3.0), I've opened a focused v0.2.0 issue for the stdio-CLI work so PR3 has a clean home: #142. Please target that one. Concrete pointers:

• New flags --mcp-host-target module:function + --mcp-runtime-config <path.yaml>, mirroring the existing --mcp-target add_argument + dispatch in cli.py; add mutual-exclusivity guards.
• New runner.run_scenario_with_mcp_host_target(...) that loads config via mcp_runtime.load_mcp_runtime_config and calls mcp_host.run_mcp_host_target / async_run_mcp_host_target.
• Keep the naming distinct in docs: --mcp-target (workflow adapter, local callable) vs --mcp-host-target (real stdio host). Add an "MCP host adapter" section to docs/adapters.md.
• E2E test can reuse tests/fixtures/mcp_servers/filesystem_server.py.
  Looking forward to it — ping me if any of the wiring is ambiguous.

[Flames4fun]

Hi @mertsatilmaz , I’d like to keep helping on the next phase. I’m thinking MCP HTTP transport would be the right next piece to split out and tackle first, before touching OAuth or resources/prompts/sampling. My idea would be to keep the first PR narrow: add the HTTP transport path alongside the existing stdio runtime, preserve current stdio behavior, start with an unauthenticated Streamable HTTP MVP if that matches your direction, reuse the existing canonical trace shape, and add focused tests around the new transport.

Does that sound like the right next sub-issue to pick up, or would you rather prioritize a different part of #96 first?

[mertsatilmaz]

Sorry for the slow reply, @Flames4fun — that's on me, and thanks for your patience.

Yes, your instinct is right. HTTP transport is the correct next slice of #96, ahead of OAuth and resources/prompts/sampling, and an unauthenticated Streamable HTTP MVP alongside the existing stdio runtime is exactly the scope I want. Keep the current stdio behavior as the default, reuse the canonical trace shape, and add focused transport tests — the narrow first PR you described is the right shape.

I've opened #161 for it so it has a clean home, with the scope written out. It's yours. #96 stays open for the rest (OAuth, SSE, resources/prompts/sampling, default-deny roots).

One thing you should know: PR #160 (an unrelated bugfix) arrived with a draft Streamable-HTTP/SSE connector bundled into it. I've asked for that part to be dropped precisely because this is your assigned work — but the diff may be useful reference material if you want a look before you start.