[devalshahamd]

• Description: AgentX profile captures started as soon as AIPerf entered the measured phase and used synthetic splitter defaults, so traces sampled ramp-up and the wrong kernel mix. Wait 300s after the measured phase (AGENTX_PROFILE_START_DELAY_S), stop after 256 engine steps or 100s, and analyze AgentX traces with a 64-step window (R=0.1, measured conc, OSL 512).
• Linked issue(s): none
• Tests: updated test_aiperf_client_sh.py, test_kernel_request_handlers_units.py, test_profile_and_kernel_handlers.py. Run: pytest src/hyperloom/inference_optimizer/tests/test_aiperf_client_sh.py src/hyperloom/inference_optimizer/tests/test_kernel_request_handlers_units.py src/hyperloom/inference_optimizer/tests/test_profile_and_kernel_handlers.py
• CHANGELOG.md entry under [Unreleased]: yes
• Breaking changes: no (AgentX profile defaults change: delay 300s, window 100s, 256 steps; override with AGENTX_PROFILE_START_DELAY_S / AGENTX_PROFILE_WINDOW_S / HYPERLOOM_PROFILE_MAX_ITERS)
• PR addresses single concern: yes
• Root cause is upstream (Magpie/TraceLens/GEAK/IntelliKit/AgentKernelArena), ticket filed: no; Hyperloom’s AgentX client and TraceLens argv, not an upstream bug

[jiaqiang-dot-liu]

No blocking issues overall -- the rocprofiler-sdk 1.3.2 explanation resolves my
main concern, and the scoping is clean: non-AgentX behavior is unchanged on every
path I traced.

One request: please also validate on SGLang.

max_iters reaches the two frameworks through different mechanisms -- vLLM via
--profiler-config.max_iterations on the serve line (_workload_envs.py:1541),
SGLang via num_steps in the /start_profile body (:1601). Your root cause,
the image, and the Kimi-K3 run are all vLLM-side, and the three new
test_materialize_profile_agentx_* cases all use _profile_yaml(..., "vllm", ...),
so SGLang's 256 currently ships unvalidated.

Two suggested fixes:

1. _workload_envs.py:1168 still reads "That bound is a host-RAM limit ... a
   DeepSeek-V4 profile round was OOM-killed mid-capture three times in a row."
   Per your explanation that attribution no longer holds, and it describes a
   bound this PR just raised 32x. The next person to hit an OOM will find that
   one clue and follow it toward lowering the step count rather than changing
   the image. Please reword it and record the rocprofiler-sdk 1.3.2 / v29
   constraint (plus the known-good version) in the _AGENTX_PROFILE_MAX_ITERS
   comment -- it isn't captured anywhere in the repo today.

2. The trace-flush budget wasn't resized. aiperf_client.sh:662 records it as
   measured on "GLM-5.3 (sglang, TP=8, one 20s window)", with the first rank file
   landing at t+350s. The window is now 100s with ~32x the steps, so traces are
   much larger, but AGENTX_TRACE_FLUSH_TIMEOUT_S and the 900s default are
   unchanged. This surfaces as a silent trace_flush_timeout rather than a loud
   failure.

Housekeeping: the branch has merge conflicts and needs a rebase; CI only ran
CodeQL, Gitleaks, and readthedocs, so could you post the local pytest output for
the three test files in the description?

The smaller items (--split-conc can emit a literal "0", AgentX now crosses
cap=128 with no log line, the 256 steps and the 100s window live in separate
files with no cross-reference) can ride along with next week's usability pass.

[qianghan-amd]

Have tries both this PR and #1684 on a arbor/hyperloom agentx campaign. with the both PRs applied, arbor can identify the large gain in prefill phase much faster.

[rbrugaro-amd]

@devalshahamd this addresses an important issue. When the prefill is missing from a profiling is not getting flagged and HL will not look into prefill which is where we can get real uplift in agentic workflows frequently. Can we try to get this merged?
I created a sibling PR #1684 with some hardening that in future would catch other profiling gaps and raise them as such for agent specialist to have to address it instead of working around it and missing a real optimization oportunity.

[chaojhou]

Closing as superseded. This changes aiperf_client.sh, which #1922 removed when AgentX moved onto Magpie's native contract, so it can no longer apply. The capture-window requirement (300 s delay, 256-step cap) now lives in #1948 and will be implemented in the generated Magpie benchmark YAML by the follow-up to #1922, with no new AGENTX_PROFILE_START_DELAY_S env toggle. Thanks @devalshahamd. The requirement itself stands and is tracked there.