http_replay set-path-param: 0-based leading-empty segment indexing causes ~33% path mis-targeting

[Serhatcck]

Summary

http_replay's set-path-param mutation indexes path segments 0-based including the leading empty segment, which the LLM miscounts a meaningful fraction of the time — it targets the wrong segment (e.g. the resource-name segment instead of the id), producing invalid request-targets that the app rejects. Empirically measured mis-target rate: ~33% of set-path-param uses in a controlled test.

Mechanism

Mutate.setPathParam (src/replay/mutate.ts) splits on / and indexes the resulting array, which starts with an empty string:

/api/items/1/reviews  ->  ["", "api", "items", "1", "reviews"]

So the id 1 is at position 3, not 2. A tester that reasons "the id is the 3rd path element" (1-based, or ignoring the leading empty) passes position 2 and rewrites the resource-name segment instead, yielding /api/2/1/reviews etc. — a route the app does not have.

Evidence (controlled test — real pipeline, live local app)

6 real proxy-tester-idor runs (opus) on a seeded GET /api/items/1/reviews-shaped request (id at position 3), plus supplementary injection/o4-mini runs:

• 1 of 3 set-path-param usages mis-targeted (33%) — passed position 2 (resource segment) instead of 3 (id) -> the mutated sweep hit /api/{N}/1/reviews -> 500 "Unexpected path" for every value -> the intended id enumeration tested a non-existent route.
• The failure reproduced a previously-observed live incident exactly.

Why it is intermittent, not catastrophic (calibration)

Two existing mitigations keep the blast radius small — this is a real but low/medium-severity ergonomics issue, not a hard break:

1. The prompt steers away from it. src/agent/prompt/vuln/idor/prompt.txt demonstrates only set-target (full-path rewrite) for path ids and never set-path-param; the positional API is not surfaced in the tool schema. So many runs avoid the positional op entirely.
2. Models often self-correct. In the observed mis-target, the model saw the run of 500s and switched to set-target -> 200s, so that run was not a no-op. But recovery is non-deterministic — the prior live incident was the same mis-target that happened not to recover.

Contributing factors

• The position semantics (0-based, leading-empty-included) are documented only in the mutate.ts docstring, not in the tool parameter description (src/tool/http-replay.ts MUTATION name just says "path segment position as string ... for set-path-param").
• setPathParam silently no-ops on an out-of-range/non-numeric position (returns the request unchanged) instead of erroring — so a bad index fails quietly.

Fix options (low-risk)

1. Make the tool schema self-documenting — the name description for set-path-param should state the convention with an example ("0-based, counts the leading empty segment: /api/items/1/x -> id is position 3"). Cheapest, since the prompt already prefers set-target.
2. Offer a value-based path-param op — replace a path segment by matching its current value (e.g. replace "1"), removing the index-miscount surface entirely.
3. Optionally, make setPathParam error loudly on an out-of-range position instead of silently no-op'ing.

[Serhatcck]

Fixed on dev (commit c842c35, pending push).

The set-path-param op's name description now documents the 0-based convention that COUNTS the leading empty segment, and steers path-id rewrites to set-target instead of positional indexing. Description-only change — no engine/behavior change (proportionate fix).

Verified (controlled LLM test): old description drove ~43% mis-target on set-path-param (part of ~33% across the wider op battery); the clarified description + set-target diversion brought path-id rewrites to 0% mis-target, with query/header/cookie ops unaffected.