feat: Phoenix-MCP calibration loop is live in prod — flip the chip green#84
Conversation
…p green
The deployed audit now does a genuine per-request MCP round-trip (read the
glasshat-calibration dataset + write each correction back) against a Cloud-SQL-backed
Phoenix on Cloud Run, verified against prod:
- eval 4.0 to 2.84 (= 4.0 minus 0.8x1.45, NOT the plus/minus-2 cap, so the p25/p75
fix holds);
- "Phoenix MCP Server running on stdio" twice per eval (read + write-back);
- the 4 corrected (yellow, criterion, low) cells grew n 7 to 8 (write-back landed).
Since it is genuinely live, flip the honesty signals from "wired" (amber) to "live":
- deploy.sh: API memory 512Mi to 2Gi (the npx phoenix-mcp Node subprocess + the
Python eval OOM-killed 512Mi -> 503; 2Gi fits both).
- deployment.ts: phoenixmcp chip wired->live, consultantMode table-prior->phoenix-mcp,
header comment (all five pillars live).
- ProofStrip / ProofTimeline / ProofReceipt (+ tests): caveat text + assertions flip
to live.
- Landing ArizeBand: "wired"->"live" loop, "Phoenix . MCP (live)" badge.
- README + Devpost: Phoenix-MCP disclosures updated to the live state (honest, no
more "deployed image uses the table prior").
Web gates: typecheck . lint . test (74) green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 31 minutes and 58 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (10)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request transitions the Phoenix MCP calibration loop to a live status on the deployed demo, updating the documentation, UI components, and tests to reflect this change. Additionally, the Cloud Run deployment configuration is updated to increase the container memory to 2Gi to support the Node subprocesses. Feedback highlights a visual inconsistency in ProofTimeline.tsx where the text color was updated to green but the surrounding container and arrow remain amber. It is also recommended to pre-install @arizeai/phoenix-mcp in the container image rather than invoking it via npx on every request to reduce latency and memory overhead.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| ↑ | ||
| </span> | ||
| <span className="font-medium text-[var(--color-warn)]">Phoenix MCP</span> | ||
| <span className="font-medium text-[var(--color-good)]">Phoenix MCP</span> |
There was a problem hiding this comment.
Changing the text color of 'Phoenix MCP' to var(--color-good) (green) to reflect its live status creates a visual inconsistency with the surrounding elements. The arrow icon (line 192) and the parent container's border/background classes (lines 188-189) are still hardcoded to use var(--color-warn) (amber). Please update the container and arrow to use var(--color-good) as well to ensure consistent coloring for the live state.
| # The live Phoenix-MCP audit spawns an `npx @arizeai/phoenix-mcp` Node subprocess | ||
| # per request (read consult + write-back) ON TOP of the Python eval; 512Mi OOM-kills | ||
| # the container (→ 503). 2Gi fits Python + Node with headroom. | ||
| API_MEMORY="${API_MEMORY:-2Gi}" |
There was a problem hiding this comment.
Spawning an npx @arizeai/phoenix-mcp Node subprocess on every single request introduces significant latency and memory overhead (requiring the increase to 2Gi). To optimize performance and reduce request latency, consider pre-installing @arizeai/phoenix-mcp globally or locally in the container image and invoking it directly via node or its binary executable instead of using npx.
1 participant
Builds on #83 (the three bug fixes) + the GCP Phoenix (Cloud Run + Cloud SQL) + the seeded
glasshat-calibrationdataset. The deployed audit now does a genuine per-request MCP round-trip, verified against prod:Phoenix MCP Server running on stdiologged twice per eval (read consult + write-back) — the table fallback never spawns npx;(yellow, criterion, low)cells grew n 7 to 8 — the write-back landed in the live dataset.Since it is genuinely live, the honesty signals flip from wired (amber) to live (green):
phoenix-mcpNode subprocess + the Python eval OOM-killed 512Mi (-> 503); 2Gi fits both. (Root-caused from the prod logs.)wired->live,consultantModetable-prior->phoenix-mcp, header (all five pillars live).Phoenix . MCP (live)badge.Honest framing preserved: the credentialed Agent-Engine run (trace tree + hit@13) stays the separate genuine-AX evidence. Web gates: typecheck . lint . test (74) green.
Next (this session): redeploy via deploy.sh, then the live chip goes green and re-verify.