feat: token vault support [KOKO-4550]

[LorenzoScebba]

Summary

Adds support for the new top-level token_vault property on MCP Servers (Konnect spec AIGatewayTokenVault), which resolves the upstream credential per request via Kong's Token Vault instead of a static credential. Implemented for both directions: convert lowers it into the ai-mcp-proxy plugin's auth record; revert lifts it back.

Requires kong-ee PR Kong/kong-ee#21073 (auth.provider: token_vault + auth.token_vault on ai-mcp-proxy).

Mapping

# AI Gateway MCP Server            # decK / ai-mcp-proxy
mcp_servers:                       plugins:
  - token_vault:                     - name: ai-mcp-proxy
      directory: my-directory         config:
      provider: my-upstream-provider    auth:
      encryption_secrets: [...]           provider: token_vault
      redis: {...}                        token_vault: {directory: ..., provider: ..., encryption_secrets: ..., redis: ...}

• Lives in internal/aimap/tokenvault.go so both directions share it (per repo convention).
• The nested redis block (Konnect AIGatewayRedisCloudConfiguration) flattens to the Kong redis-ee schema's prefixed keys: keepalive.pool_size → keepalive_pool_size, sentinel.master → sentinel_master, cluster.nodes → cluster_nodes[{ip,port}] vs sentinel.nodes[{host,port}], cloud_authentication.type → auth_provider with provider-prefixed fields (aws_access_key_id, azure_client_id, gcp_service_account_json, …). Field names verified against kong-ee's redis/v2/schema.lua.
• encryption_secrets are referenceable ({vault://...}) and pass through verbatim.

Hard errors (failAt, independent of -strict)

Mirroring the plugin's own schema/entity checks — emitting a config Kong would reject is never useful:

• token_vault together with config.upstream.auth (both lower to the same plugin auth record, which carries one provider).
• redis without encryption_secrets.
• token_vault present but missing directory/provider (would otherwise emit auth.token_vault: {} silently).

Revert

revert recognizes auth.provider: token_vault on an MCP route and lifts it into the top-level token_vault field (its position in the AI Gateway schema) instead of config.upstream.auth; only fields present in the deck YAML are set, so the round-trip gate stays byte-identical with zero warnings. Agents are unaffected — ai-a2a-proxy doesn't support token_vault (off/aws_iam only).

Testing

• Golden cases both directions: convert/testdata/64_mcp_token_vault, revert/testdata/52_mcp_token_vault (covered by TestRoundTrip).
• Unit tests: aimap flatten/unflatten fixpoint + omissions preserved; convert lowering + all three failAt paths; revert missing-block warn (incl. -strict).
• E2E (e2e-tests/mcp-server/token_vault_gates_tools_and_enrolls/): runs against kong/kong-ai-gateway-dev:35727837758-070f4458537dcb4fe4d6c7321447cf902d2e07c9 with a mock Kong Identity service (KONG_IDENTITY_SERVICE), mock upstream MCP server, and Redis. Verifies the full enrollment lifecycle: schema-valid config at startup → no subject token = 500 → gated caller sees only the virtual token_vault_authenticate/token_vault_check_authentication_status tools → authenticate returns the enrollment URL → check_authentication_status confirms enrollment → credential lands in the encrypted Redis L2 cache → enrolled caller sees the real upstream tools, with the upstream receiving the exchanged credential, not the subject token.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[LorenzoScebba]

/ksai review this pr

[kong-api-ops]

✅ KSAI Review: Finished

1. 15:46 UTC Tracing error-handling paths across the changed conversion logic
2. 15:48 UTC Tracing error paths and checking related revert logic
3. 15:51 UTC Auditing conversion logic and end-to-end test coverage
4. 15:53 UTC Reading upstream and conversion code to trace review findings

---

Run report (federated) · 1 paid run · $1.1482 total

#	Engine	Result	Model	Turns	Cost
1	opencode	success	glm-5.3/high	71	$1.1478

Reviewing cost $1.1478; deciding how to run it cost $0.0004

[kong-api-ops]

Check	Result
Scope	Token Vault lowering for MCP servers: convert guards and emit, shared aimap mapping, revert lifting, testdata, e2e test.
Mandate	go-code-review (Go files) + default-code-review (docs, YAML, Python, shell)
Findings	1 Medium, 1 Low
Findings audit	Audited by kreview:findings-auditor. 2 stand, 0 removed; 1 downgraded to Low with corrected mechanism.

The change is coherent. Round-trip, golden, and unit tests cover both directions. The two findings are edge and environment issues, not core logic defects.

Additional Risk pass: no leaked secrets; test fixtures use vault references or fake values. No new security surface, no resource leaks, no Critical issues. No additional concerns found.