Skip to content

Share deployment topology mapping between benchmarks and evals - #1304

Merged
cquil11 merged 2 commits into
masterfrom
fix/eval-physical-gpu-topology
Oct 9, 2026
Merged

cquil11 merged 2 commits into
masterfrom
fix/eval-physical-gpu-topology

Conversation

@cquil11

@cquil11 cquil11 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Benchmark and eval ingestion had separate topology calculations. Eval could multiply TP by EP and count mirrored aggregate metadata as two workers, making a four-GPU SGLang eval appear as 32 GPUs.

Route performance results, individual eval artifacts, and collected eval results through one shared deployment-topology resolver. It owns physical GPU counting, worker multiplicity, aggregate role mirroring, and disaggregation. AgentX SGLang TP/EP share devices; physical node placement alone does not imply separate prefill/decode engines. Explicit GPU counts remain authoritative, and historical fixed-sequence count conventions are preserved.

This addresses the eval mappings found in SemiAnalysisAI/InferenceX#3701, SemiAnalysisAI/InferenceX#3782, and SemiAnalysisAI/InferenceX#3744. Existing database rows are not rewritten; already-ingested incorrect configurations require a separate audited reconciliation.


Note

Medium Risk
Changes ETL config identity (GPU counts, disagg flags, aggregate mirroring) for benchmarks and evals; incorrect mapping could mis-key configs until reconciliation, though behavior is heavily test-covered.

Overview
Introduces resolveConfigTopology() in packages/db/src/etl/topology.ts and routes benchmark, individual eval, and agg eval mappers through it, replacing duplicated resolveParallelism / buildEvalConfig logic.

Config identity fixes: legacy role GPU fallbacks now multiply per-worker TP×EP (or AgentX TP×PP×PCP) by worker replicas on both paths; SGLang/Dynamo-SGLang AgentX treats EP/DCP as sharing TP devices; single-node SGLang rows with mirrored 1P/1D role fields collapse to one aggregate config (zero role workers, same GPU count on both sides). Disaggregation is driven by framework aliases and non-zero decode worker pools, not is_multinode or prefill workers alone.

Docs in docs/data-pipeline.md describe the shared resolver and note that fixes apply on future ingest only—existing rows need audited reconciliation, not blind re-ingest.

Reviewed by Cursor Bugbot for commit eb61c96. Bugbot is set up for automated code reviews on this repo. Configure here.

@cquil11
cquil11 requested a review from adibarra as a code owner October 9, 2026 16:59
@vercel

vercel Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
inferencemax-app Ready Ready Preview Oct 9, 2026 5:13pm UTC

Request Review

@cquil11 cquil11 changed the title Fix physical GPU counts and aggregate topology in SGLang eval ingestion Share deployment topology mapping between benchmarks and evals Oct 9, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit eb61c96. Configure here.

Comment thread packages/db/src/etl/topology.ts
@cquil11
cquil11 merged commit 102ed12 into master Oct 9, 2026
30 checks passed
@cquil11
cquil11 deleted the fix/eval-physical-gpu-topology branch October 9, 2026 18:44

This branch was successfully deployed

1 active deployment
Preview — eb61c96c Deployed Oct 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant