feat(llm): add a system one client that prefers the ai-gateway

[rafaeelaudibert]

Problem

• Code that asks Jev a System One question can reach only TypeSafe, a third-party service approved for experiments without customer data.
• The Go ai-gateway already serves posthog/hogference/jevk5-fp8-0.2, a Jev build PostHog hosts, on POST /v1/systemone.
• No shared client reaches it, the way build_openai_client reaches the gateway for chat models.
• The next layer, feat(growth): turn suggestion backend #106432, needs one for the turn suggestion judge.

Changes

• build_system_one_client returns a client for the ai-gateway where AI_GATEWAY_URL and AI_GATEWAY_API_KEY are set.
• TypeSafe is opt-in. A caller that passes a TypeSafeFallback gets TypeSafe where no gateway is configured, through the existing egress budget.
• A caller that passes no fallback never reaches TypeSafe. Without a gateway it gets SystemOneNotConfigured, so customer data cannot reach a third party by accident.
• The fallback names its own model, because TypeSafe runs a different Jev build with different calibration.
• The gateway client sends the gateway key as a bearer and the usual X-PostHog-* attribution headers, and bills the key owner's wallet.
• The gateway client refuses a choice question with more than 16 options before it sends, because JevK5 answers one letter per option.
• It refuses a non-https gateway URL unless it points at this machine, so the key never travels in clear.
• Mechanical: the System One request and answer types move from posthog/egress/typesafe/ to posthog/llm/system_one.py, so both servers share them. The TypeSafe errors now subclass the shared ones.
• No behavior change for users: nothing calls the builder until feat(growth): turn suggestion backend #106432.
• Not moved: products/ml_inference keeps its own gateway client for now. It could move to this builder once its owners agree.

How did you test this code?

• New posthog/llm/test_system_one_client.py catches these regressions:
  • picking TypeSafe while the gateway is configured, or the wrong model for a server;
  • reaching TypeSafe for a caller that passed no fallback;
  • a request that misses /v1/systemone, the bearer, or the attribution headers;
  • a gateway failure that returns a partial result instead of raising;
  • an oversized choice question that reaches the network.
• The existing TypeSafe egress tests pass unchanged after the move.
• Not run: a real request to the ai-gateway. The local machine has no gateway configured.

Release status

• No feature flag controls this change
• This change is behind a feature flag and is not available to users
• This change makes a previously flagged feature available to everyone

Automatic notifications

• Publish to changelog?

Docs update

posthog/egress/typesafe/README.md now points callers at the shared types and the builder.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5

• Inserted into stack #106737 below its first caller, at the user's request.
• Skills invoked: /migrating-llm-gateway-callers, /stacking-prs, /writing-tests, /writing-pr-descriptions.
• CodeRabbit CLI pass skipped by the user's standing choice.
• No duplicate: a search for open System One client PRs found none.

🤖 Generated with Claude Code

[github-actions]

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Playwright — all passed

All tests passed.

View test results →

[greptile-apps]

[Medium risk] Refactors System One API types into a shared module.

The PR appears safe to merge; no outstanding findings or new actionable issues remain.

_{Reviews (2) · Last reviewed commit: "fix(llm): accept typeless system one ans..."}

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

📝 Walkthrough

Walkthrough

The change adds shared System One request and response types, body construction, and response validation. It adds client selection between the configured AI gateway and an explicitly configured TypeSafe fallback. The TypeSafe client now uses the shared contract, and its package exports, documentation, and test imports are updated. New tests cover client selection, gateway requests and failures, and request limits.

Priority: ➖ Normal

Merge Risk: 🔵 Low · up to 71a42

This adds a System One client that prefers the PostHog-hosted gateway. Nothing calls it yet, so users see no change. The README does not fully state when requests go to TypeSafe instead of the gateway. Operators could misjudge where data is sent, so correct the wording as a small follow-up.

Security Architecture Review

Security architecture risk: 🟡 Moderate · up to 71a42

The new client has safeguards against accidental third-party use, but a future call can send request data and a service credential to whichever secure gateway URL is configured. The destination and caller-data policies need to remain explicit when the client is adopted.

Retained concerns

• Medium · security · inferred: The new System One path verifies transport security, but not that the configured gateway host is an approved destination. If deployment configuration points to another HTTPS host when this client is used, that host receives request state and the gateway bearer. Configuration control and effective caller exposure are not established.

Security review details

Security Blast Radius

• inferred — For a future invocation, the exposed assets would be the supplied state and the configured gateway credential at the selected destination. The supplied evidence does not establish that a tenant can choose that destination or that the new builder is already used in production.

Security Findings and Attack Paths

• inferred — If an unauthorized party can change the deployment gateway URL, a future System One call could transmit its state and bearer to that party's HTTPS host. Control of that setting is unproven; this is a conditional path, not a verified exploit.

Trust Boundaries and Controls

• observed — TypeSafe remains an explicitly selected, budgeted external route; possession of its API key alone does not enable fallback in the new builder. Gateway selection instead uses the configured URL and bearer with attribution headers.

Resilience and Maintainability Implications

• observed — The new gateway client creates its HTTP transport per call and has no visible persistent reservation state. The fallback delegates to the existing TypeSafe transport; its underlying reservation cleanup behavior was not established by the scoped evidence.

Hardening Proposals

• proposed — Before adopting the builder for customer-data callers, verify who can set the gateway URL and constrain it to approved destinations where deployment controls do not already provide that guarantee.

🚥 Pre-merge checks | ✅ 1

✅ Passed checks (1 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description is complete and follows the required structure. It explains the problem, user-visible and mechanical changes, testing coverage and limits, release status, documentation, and agent cont…

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

_{Comment @coderabbitai help to get the list of available commands.}

[trunk-io]

   

View Full Report ↗︎ ⋅ Docs

[coderabbitai]

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (1)

posthog/egress/typesafe/README.md-8-9 (1)

8-9: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

State the full gateway selection rule.

Line 8 says the client uses the ai-gateway when AI_GATEWAY_URL is set. That is incomplete. _usable_gateway also requires AI_GATEWAY_API_KEY. It also requires an https URL unless the host is loopback. If the URL uses plain HTTP, a caller that passed a TypeSafeFallback goes to TypeSafe with no error. Only a warning is logged. Operators need this rule to know which third party gets the request.

📝 Proposed wording

-It reaches the ai-gateway where `AI_GATEWAY_URL` is set.
-It falls back to this domain only when the caller passes a `TypeSafeFallback`, and every caller that does so meets the usage policy below.
+It reaches the ai-gateway where `AI_GATEWAY_URL` (https, or plain http on the local machine) and `AI_GATEWAY_API_KEY` are both set.
+Otherwise it falls back to this domain, but only when the caller passes a `TypeSafeFallback`. Every caller that does so meets the usage policy below.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: PostHog/posthog/.coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: 29fd52e3-9104-4d3a-8327-4ac2f68c3600

📥 Commits

Reviewing files that changed from the base of the PR and between 0432a8e and 71a423b.

📒 Files selected for processing (7)

• posthog/egress/test/test_typesafe.py
• posthog/egress/typesafe/README.md
• posthog/egress/typesafe/__init__.py
• posthog/egress/typesafe/client.py
• posthog/llm/system_one.py
• posthog/llm/system_one_client.py
• posthog/llm/test_system_one_client.py

Included review availability: Your plan provides up to 12 included reviews per hour; 4 remain after this review.

[stamphog]

Approved.

This adds a new, currently-uncalled client builder plus a mechanical type move with matching test updates — no production behavior changes yet since nothing invokes the new builder until a follow-on PR. It touches secret/API-key handling for an internal AI gateway, but two independent agent reviews (Greptile, CodeRabbit) examined the diff and raised only a minor doc-completeness nit, with a corroborating 👍 reaction, giving adequate independent assurance for that sensitive surface.

• 👍 on the PR from greptile-apps[bot].
• CodeRabbit flagged that the README doesn't state the full gateway-selection rule (also requires AI_GATEWAY_API_KEY and https) — cosmetic/doc-only, not blocking.

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✓	no deny categories matched
size	✓	565L, 4F substantive, 752L/7F incl. docs/generated/snapshots — within ceiling
tier	✓	T1-agent / T1d-complex (752L, 7F, single-area, feat)
stamphog 2.1.0		.stamphog/policy.yml @ b65b97c · reviewed head b65b97c

[trunk-io]

😎 Stack merged successfully - details.

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-09-25 18:17 UTC	Run
prod-us	✅ Deployed	2026-09-25 18:30 UTC	Run
prod-eu	✅ Deployed	2026-09-25 18:32 UTC	Run