fix: resolve the gateway target for A/B test invocation URLs (#1854) (#1874)

* fix: stop putting the runtime name in A/B test invocation URLs (#1854)

`getInvocationUrl` built config-bundle URLs from `record.agent` — the RUNTIME
name — but a gateway invocation path segment must be a gateway TARGET name.
Whenever the two differed, every request to the printed URL failed:

  {"success":false,"error":"No Target found for Target name: CustomerSupportAB"}

A config-bundle test has no target of its own to substitute. Its variants are
configuration bundles, it attaches to the whole gateway, and the service splits
traffic with a gateway rule — so the path segment is whichever gateway target
the caller invokes, which the CLI cannot know. (Confirmed against the service:
config-bundle tests store only `configurationBundle` per variant, and the
config-bundle e2e creates one on a gateway with no targets at all.)

So report the gateway base URL and name what to append, rather than guessing a
path that 404s. Target-based tests are unchanged — their variants *are* targets,
so `variants[0].targetName` is a real path.

  Gateway URL: https://<gatewayId>.gateway.bedrock-agentcore.<region>.amazonaws.com
    → append /<gateway-target>/invocations (see `agentcore status --json`)

`view ab-test --json` reports `gatewayUrl` + `invocationUrlHint` for
config-bundle tests instead of `invocationUrl`, so scripts reading
`.invocationUrl` get nothing rather than a URL that fails. Target-based keeps
`invocationUrl` unchanged.

Also corrects two comments that documented the bug as intended behaviour:
`docs/ab-tests.md` ("config-bundle uses the agent name") and a claim in
RunABTestFlow that targets deploy as `${project}-${target}` — the L3 CDK
deploys each target under its spec name verbatim.

* fix: resolve the gateway target for A/B test invocation URLs (#1854)

Config-bundle tests printed an invocation URL built from the runtime name, but a
gateway path segment must be a gateway TARGET name, so requests 404'd with
"No Target found for Target name: <runtime>".

Resolve the gateway target(s) fronting the runtime at create time — the link
lives in `agentCoreGateways[].targets[].httpRuntime.runtime`, which create()
already loads — and persist the result on the record:

  - exactly one match  -> `targetName`       -> a complete `invocationUrl`
  - several matches     -> `targetCandidates` -> `invocationUrlCandidates` (pick one)
  - none                -> neither            -> `gatewayUrl` + `invocationUrlHint`

This fixes both same-name and different-name configurations (the previous
gateway-base-only output regressed the same-name case and left the workflow
manual), and keeps `.invocationUrl` populated whenever a single target is known.
The runtime name is never used as the path. Target-based tests are unchanged.

`getInvocationUrl` stays synchronous — resolution happens once at create and is
read from the record — so `view --json`, `printABTestDetail`, and the TUI detail
view need no plumbing changes.

Also corrects two comments that documented the bug as intended behaviour.

---------

Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>

Author: jariy17

docs/ab-tests.md

@@ -115,14 +115,25 @@ Promote does not deploy — review the change and run `agentcore deploy` to roll
 
 ## Invocation URL
 
-`view ab-test <id>` shows an **Invocation URL** derived from the test's gateway. Send traffic there and the gateway
-splits it between the variants per the configured weights:
+`view ab-test <id>` shows a URL derived from the test's gateway. Send traffic there and the gateway splits it between
+the variants per the configured weights:
 
 ```
-https://<gatewayId>.gateway.bedrock-agentcore.<region>.amazonaws.com/<target-or-agent>/invocations
+https://<gatewayId>.gateway.bedrock-agentcore.<region>.amazonaws.com/<gateway-target>/invocations
 ```
 
-(target-based uses the control target's path; config-bundle uses the agent name.)
+The path segment is always a **gateway target** name — never a runtime name. Config-bundle tests carry no target of
+their own (their variants are configuration bundles), so the CLI resolves the gateway target(s) fronting the `--runtime`
+under test when the test is created:
+
+- **one matching target** (the common case, including target-based tests) — a complete **Invocation URL**.
+- **several matching targets** (e.g. a canary beside prod) — one **Invocation URL** per target; pick the one to send
+  traffic to.
+- **no matching target** — the **Gateway URL** only; append `/<gateway-target>/invocations` yourself, using
+  `agentcore status --json` to list the gateway's targets.
+
+With `--json` the field mirrors these cases: `invocationUrl` (single), `invocationUrlCandidates` (several), or
+`gatewayUrl` + `invocationUrlHint` (none).
 
 ## Results
 

docs/commands.md

@@ -1267,7 +1267,9 @@ agentcore archive ab-test -i <ab-test-id>
 View job history and details. Works for all four job types — `recommendation`, `batch-evaluation`, `ab-test`, and
 `insights`. With no `[id]` it lists every job of that type; with an `[id]` it shows that job's detail (status, inputs,
 and results). Without `--json` the command launches the interactive TUI; with `--json` it prints a machine-readable
-record (the `ab-test` detail also includes `invocationUrl`).
+record (the `ab-test` detail also includes an invocation URL — `invocationUrl`, `invocationUrlCandidates`, or
+`gatewayUrl` + `invocationUrlHint` depending on how many gateway targets front the runtime; see
+[A/B tests](ab-tests.md#invocation-url)).
 
 ```bash
 # List all jobs of a type
@@ -1279,7 +1281,7 @@ agentcore view insights
 # Detail for one job (JSON is non-interactive)
 agentcore view recommendation <id> --json
 agentcore view batch-evaluation <id> --json
-agentcore view ab-test <id> --json     # JSON includes invocationUrl + results
+agentcore view ab-test <id> --json     # JSON includes gateway URL fields + results
 ```
 
 Each `view <type>` subcommand accepts the same argument and flags:

src/cli/commands/view/command.tsx

@@ -1,7 +1,14 @@
 import { ConfigIO, JobNotFoundError, serializeResult } from '../../../lib';
 import { createJobEngine } from '../../operations/jobs';
 import type { ABTestJobRecord, JobType } from '../../operations/jobs';
-import { getInvocationUrl, printABTestDetail, printABTestHistory } from '../../operations/jobs/ab-test/format';
+import {
+  INVOCATION_PATH_HINT,
+  getGatewayBaseUrl,
+  getInvocationUrl,
+  getInvocationUrlCandidates,
+  printABTestDetail,
+  printABTestHistory,
+} from '../../operations/jobs/ab-test/format';
 import { printBatchEvaluationDetail, printBatchEvaluationHistory } from '../../operations/jobs/batch-evaluation/format';
 import { printInsightsDetail, printInsightsHistory } from '../../operations/jobs/insights/format';
 import { printRecommendationDetail, printRecommendationHistory } from '../../operations/jobs/recommendation/format';
@@ -44,6 +51,23 @@ const TYPE_META: Record<
   },
 };
 
+/**
+ * URL fields for `view ab-test --json`.
+ *
+ * When exactly one gateway target is known (target-based control, or a config-bundle runtime that
+ * resolved to a single target), emit the complete `invocationUrl`. When several targets front the
+ * runtime, emit `invocationUrlCandidates` so the consumer can choose. When none is known, emit
+ * `gatewayUrl` + `invocationUrlHint` so the path can be built by hand. The runtime name is never used
+ * as the path — that was the #1854 bug.
+ */
+function abTestUrlFields(record: ABTestJobRecord): Record<string, string | string[] | undefined> {
+  const url = getInvocationUrl(record);
+  if (url) return { invocationUrl: url };
+  const candidates = getInvocationUrlCandidates(record);
+  if (candidates.length) return { invocationUrlCandidates: candidates };
+  return { gatewayUrl: getGatewayBaseUrl(record), invocationUrlHint: INVOCATION_PATH_HINT };
+}
+
 function registerViewSubcommand(viewCmd: Command, type: JobType) {
   const meta = TYPE_META[type];
 
@@ -65,8 +89,7 @@ function registerViewSubcommand(viewCmd: Command, type: JobType) {
             if (!record) {
               throw new JobNotFoundError(`${meta.label} "${id}" not found.`);
             }
-            const extra =
-              type === 'ab-test' ? { invocationUrl: getInvocationUrl(record as unknown as ABTestJobRecord) } : {};
+            const extra = type === 'ab-test' ? abTestUrlFields(record as unknown as ABTestJobRecord) : {};
             console.log(JSON.stringify(serializeResult({ success: true, ...record, ...extra })));
             return { job_type: type };
           });

src/cli/operations/jobs/ab-test/__tests__/format.test.ts

@@ -1,5 +1,5 @@
 import type { ABTestJobRecord } from '../../shared/types';
-import { printABTestDetail } from '../format';
+import { getInvocationUrl, getInvocationUrlCandidates, printABTestDetail } from '../format';
 import { afterEach, describe, expect, it, vi } from 'vitest';
 
 function baseRecord(overrides: Partial<ABTestJobRecord> = {}): ABTestJobRecord {
@@ -45,3 +45,83 @@ describe('printABTestDetail — gateway filter', () => {
     expect(output).toContain('Gateway filter: none');
   });
 });
+
+const GW_BASE = 'https://gw-abc.gateway.bedrock-agentcore.us-east-1.amazonaws.com';
+
+const cfgBundle = (overrides: Partial<ABTestJobRecord> = {}) =>
+  baseRecord({ mode: 'config-bundle', agent: 'CustomerSupportAB', variants: [], ...overrides });
+
+describe('getInvocationUrl', () => {
+  it('target-based: builds a full invocation URL from the control target name', () => {
+    expect(getInvocationUrl(baseRecord())).toBe(`${GW_BASE}/ctrl/invocations`);
+  });
+
+  it('target-based: returns undefined when the control target name is missing', () => {
+    expect(getInvocationUrl(baseRecord({ variants: [] }))).toBeUndefined();
+  });
+
+  it('config-bundle: builds a full URL from the target resolved at create time', () => {
+    expect(getInvocationUrl(cfgBundle({ targetName: 'customer-support-ab' }))).toBe(
+      `${GW_BASE}/customer-support-ab/invocations`
+    );
+  });
+
+  // Regression for #1854: `agent` holds the RUNTIME name, which is not a valid gateway path segment.
+  // With no resolved target, no complete URL is emitted (candidates / base URL cover those cases).
+  it('config-bundle: returns undefined when no single target resolved (never the runtime name)', () => {
+    expect(getInvocationUrl(cfgBundle())).toBeUndefined();
+    expect(getInvocationUrl(cfgBundle({ targetCandidates: ['a', 'b'] }))).toBeUndefined();
+  });
+
+  it('returns undefined for a gateway ARN it cannot parse', () => {
+    expect(getInvocationUrl(baseRecord({ gatewayArn: 'not-an-arn' }))).toBeUndefined();
+  });
+});
+
+describe('getInvocationUrlCandidates', () => {
+  it('builds one URL per candidate target', () => {
+    expect(getInvocationUrlCandidates(cfgBundle({ targetCandidates: ['prod', 'canary'] }))).toEqual([
+      `${GW_BASE}/prod/invocations`,
+      `${GW_BASE}/canary/invocations`,
+    ]);
+  });
+
+  it('is empty when a single target resolved or none did', () => {
+    expect(getInvocationUrlCandidates(cfgBundle({ targetName: 'only' }))).toEqual([]);
+    expect(getInvocationUrlCandidates(cfgBundle())).toEqual([]);
+  });
+});
+
+describe('printABTestDetail — invocation URL', () => {
+  afterEach(() => {
+    vi.restoreAllMocks();
+  });
+
+  function capture(record: ABTestJobRecord): string {
+    const spy = vi.spyOn(console, 'log').mockImplementation(vi.fn());
+    printABTestDetail(record);
+    return spy.mock.calls.map(c => c.join(' ')).join('\n');
+  }
+
+  it('prints a complete invocation URL for target-based tests', () => {
+    expect(capture(baseRecord())).toContain(`Invocation URL: ${GW_BASE}/ctrl/invocations`);
+  });
+
+  it('prints a complete invocation URL when a config-bundle target uniquely resolved', () => {
+    const output = capture(cfgBundle({ targetName: 'customer-support-ab' }));
+    expect(output).toContain(`Invocation URL: ${GW_BASE}/customer-support-ab/invocations`);
+  });
+
+  it('lists candidate URLs when several targets front the runtime', () => {
+    const output = capture(cfgBundle({ targetCandidates: ['prod', 'canary'] }));
+    expect(output).toContain(`${GW_BASE}/prod/invocations`);
+    expect(output).toContain(`${GW_BASE}/canary/invocations`);
+  });
+
+  it('falls back to the gateway URL and hint when no target could be resolved', () => {
+    const output = capture(cfgBundle());
+    expect(output).toContain(`Gateway URL: ${GW_BASE}`);
+    expect(output).toContain('append /<gateway-target>/invocations');
+    expect(output).not.toContain('CustomerSupportAB');
+  });
+});

src/cli/operations/jobs/ab-test/__tests__/resolve.test.ts

@@ -0,0 +1,74 @@
+import type { AgentCoreProjectSpec } from '../../../../../schema';
+import { resolveRuntimeTargetNames } from '../resolve';
+import { describe, expect, it } from 'vitest';
+
+type GatewaysOnly = Pick<AgentCoreProjectSpec, 'agentCoreGateways'>;
+
+/** Project spec carrying only the gateway targets the resolver reads. */
+function specWithTargets(targets: unknown[]): GatewaysOnly {
+  return { agentCoreGateways: [{ name: 'my-gw', targets }] } as unknown as GatewaysOnly;
+}
+
+const httpTarget = (name: string, runtime: string) => ({
+  name,
+  targetType: 'httpRuntime',
+  httpRuntime: { runtime },
+});
+
+describe('resolveRuntimeTargetNames', () => {
+  it('returns the single httpRuntime target routing to the runtime', () => {
+    const spec = specWithTargets([httpTarget('customer-support-ab', 'CustomerSupportAB')]);
+    expect(resolveRuntimeTargetNames('my-gw', 'CustomerSupportAB', spec)).toEqual(['customer-support-ab']);
+  });
+
+  it('picks only the matching target when the gateway serves several runtimes', () => {
+    const spec = specWithTargets([
+      httpTarget('orders', 'OrdersAgent'),
+      httpTarget('customer-support-ab', 'CustomerSupportAB'),
+    ]);
+    expect(resolveRuntimeTargetNames('my-gw', 'CustomerSupportAB', spec)).toEqual(['customer-support-ab']);
+  });
+
+  it('returns every matching target, in spec order, when several front one runtime', () => {
+    const spec = specWithTargets([
+      httpTarget('customer-support-ab', 'CustomerSupportAB'),
+      httpTarget('customer-support-canary', 'CustomerSupportAB'),
+    ]);
+    expect(resolveRuntimeTargetNames('my-gw', 'CustomerSupportAB', spec)).toEqual([
+      'customer-support-ab',
+      'customer-support-canary',
+    ]);
+  });
+
+  it('returns [] when no target routes to the runtime', () => {
+    const spec = specWithTargets([httpTarget('orders', 'OrdersAgent')]);
+    expect(resolveRuntimeTargetNames('my-gw', 'CustomerSupportAB', spec)).toEqual([]);
+  });
+
+  it('returns [] for a gateway with no targets', () => {
+    expect(resolveRuntimeTargetNames('my-gw', 'CustomerSupportAB', specWithTargets([]))).toEqual([]);
+  });
+
+  // Only httpRuntime targets front a runtime; a same-named lambda/mcpServer target is not a route to it.
+  it('ignores targets that are not httpRuntime', () => {
+    const spec = specWithTargets([
+      { name: 'customer-support-ab', targetType: 'lambda', httpRuntime: { runtime: 'CustomerSupportAB' } },
+    ]);
+    expect(resolveRuntimeTargetNames('my-gw', 'CustomerSupportAB', spec)).toEqual([]);
+  });
+
+  it('returns [] for an unknown gateway name', () => {
+    const spec = specWithTargets([httpTarget('customer-support-ab', 'CustomerSupportAB')]);
+    expect(resolveRuntimeTargetNames('other-gw', 'CustomerSupportAB', spec)).toEqual([]);
+  });
+
+  it('returns [] when the gateway or runtime is unset', () => {
+    const spec = specWithTargets([httpTarget('customer-support-ab', 'CustomerSupportAB')]);
+    expect(resolveRuntimeTargetNames(undefined, 'CustomerSupportAB', spec)).toEqual([]);
+    expect(resolveRuntimeTargetNames('my-gw', undefined, spec)).toEqual([]);
+  });
+
+  it('returns [] when the project declares no gateways', () => {
+    expect(resolveRuntimeTargetNames('my-gw', 'CustomerSupportAB', {} as GatewaysOnly)).toEqual([]);
+  });
+});

src/cli/operations/jobs/ab-test/format.ts

@@ -3,24 +3,56 @@ import { dnsSuffix } from '../../../aws/partition';
 import { formatJobDate } from '../shared/format';
 import type { ABTestJobRecord } from '../shared/types';
 
-/**
- * Derive the gateway invocation URL from the stored gateway ARN.
- * Target-based: `https://{gateway}/{control-target-name}/invocations`.
- * Config-bundle: `https://{gateway}/{agent-name}/invocations`.
- */
-export function getInvocationUrl(record: ABTestJobRecord): string | undefined {
+/** Gateway base URL (no path) from the stored gateway ARN, or undefined if the ARN can't be parsed. */
+function gatewayBaseUrl(record: ABTestJobRecord): string | undefined {
   const parts = record.gatewayArn.split(':');
   const region = parts[3];
   const gatewayId = parts[5]?.split('/')[1];
   if (!region || !gatewayId) return undefined;
-  const baseUrl = `https://${gatewayId}.gateway.bedrock-agentcore.${region}.${dnsSuffix(region)}`;
-  if (record.mode === 'target-based') {
-    const targetName = record.variants[0]?.targetName;
-    return targetName ? `${baseUrl}/${targetName}/invocations` : undefined;
-  }
-  return record.agent ? `${baseUrl}/${record.agent}/invocations` : undefined;
+  return `https://${gatewayId}.gateway.bedrock-agentcore.${region}.${dnsSuffix(region)}`;
+}
+
+/** The gateway target name that uniquely identifies this test's invocation path, if there is exactly one. */
+function uniqueTargetName(record: ABTestJobRecord): string | undefined {
+  // Target-based: the control variant's target. Config-bundle: the target resolved at create time,
+  // set only when exactly one gateway target routed to the runtime.
+  return record.mode === 'target-based' ? record.variants[0]?.targetName : record.targetName;
+}
+
+/**
+ * Derive the complete invocation URL: `https://{gateway}/{target}/invocations`.
+ *
+ * The path segment must be a gateway TARGET name. Config-bundle records store the target resolved from
+ * the runtime at create time (`targetName`); target-based records carry it on the control variant.
+ * Returns undefined when no single target is known — either the gateway has none fronting the runtime,
+ * or several do (see getInvocationUrlCandidates). Substituting the runtime name here produced URLs that
+ * 404'd with "No Target found for Target name: <runtime>" (issue #1854), so it is deliberately not done.
+ */
+export function getInvocationUrl(record: ABTestJobRecord): string | undefined {
+  const baseUrl = gatewayBaseUrl(record);
+  const targetName = uniqueTargetName(record);
+  return baseUrl && targetName ? `${baseUrl}/${targetName}/invocations` : undefined;
 }
 
+/**
+ * Candidate invocation URLs when several gateway targets route to the runtime (config-bundle only).
+ * Each is a valid path; only the user can say which should receive traffic. Empty when a single URL
+ * was resolvable (use getInvocationUrl) or when no target matched.
+ */
+export function getInvocationUrlCandidates(record: ABTestJobRecord): string[] {
+  const baseUrl = gatewayBaseUrl(record);
+  if (!baseUrl || !record.targetCandidates?.length) return [];
+  return record.targetCandidates.map(t => `${baseUrl}/${t}/invocations`);
+}
+
+/** Gateway base URL to show when no invocation path could be determined, so the user can build one. */
+export function getGatewayBaseUrl(record: ABTestJobRecord): string | undefined {
+  return gatewayBaseUrl(record);
+}
+
+/** Names what the caller must append to a gateway base URL to reach a variant. */
+export const INVOCATION_PATH_HINT = 'append /<gateway-target>/invocations (see `agentcore status --json`)';
+
 export function printABTestHistory(records: ABTestJobRecord[]): void {
   if (records.length === 0) {
     console.log('No A/B test jobs found. Run `agentcore run ab-test` to create one.');
@@ -47,7 +79,19 @@ export function printABTestDetail(record: ABTestJobRecord): void {
   console.log(`Gateway: ${record.gatewayArn}`);
   console.log(`Gateway filter: ${record.gatewayFilter?.targetPaths?.[0] ?? 'none'}`);
   const invocationUrl = getInvocationUrl(record);
-  if (invocationUrl) console.log(`Invocation URL: ${invocationUrl}`);
+  const candidates = getInvocationUrlCandidates(record);
+  if (invocationUrl) {
+    console.log(`Invocation URL: ${invocationUrl}`);
+  } else if (candidates.length) {
+    console.log('Invocation URLs (one per matching gateway target — pick the one to send traffic to):');
+    for (const url of candidates) console.log(`  ${url}`);
+  } else {
+    const baseUrl = getGatewayBaseUrl(record);
+    if (baseUrl) {
+      console.log(`Gateway URL: ${baseUrl}`);
+      console.log(`  → ${INVOCATION_PATH_HINT}`);
+    }
+  }
   console.log(`Started: ${formatJobDate(record.createdAt)}`);
   if (record.completedAt) console.log(`Stopped: ${formatJobDate(record.completedAt)}`);
   if (record.maxDurationExpiresAt) console.log(`Max duration expires: ${formatJobDate(record.maxDurationExpiresAt)}`);

src/cli/operations/jobs/ab-test/handler.ts

@@ -25,7 +25,7 @@ import { regionFromArn, resolveJobRegion } from '../shared/region';
 import type { ABTestHandler, ABTestJobRecord, DebugCheckResult, StartABTestJobOptions } from '../shared/types';
 import { buildABTestRequest } from './build-options';
 import { promoteABTestConfig } from './promote';
-import { deleteABTestRole, getOrCreateABTestRole, resolveGatewayArn } from './resolve';
+import { deleteABTestRole, getOrCreateABTestRole, resolveGatewayArn, resolveRuntimeTargetNames } from './resolve';
 import { CloudWatchLogsClient, FilterLogEventsCommand } from '@aws-sdk/client-cloudwatch-logs';
 
 /** AB-test create retries while the freshly-created IAM role propagates (gateway/eval AccessDenied). */
@@ -190,6 +190,14 @@ export const abTestHandler: ABTestHandler = {
       opts.onProgress?.('started', `A/B test created: ${createResult.abTestId} (${createResult.executionStatus})`);
       logger?.finalize(true);
 
+      // Config-bundle tests carry no target in their variants; resolve the gateway target(s) routing to
+      // the runtime so `view` can print a complete invocation URL (a single match) or list candidates
+      // (several). Target-based tests already carry the target in variantSummaries.
+      const targetNames =
+        opts.mode === 'target-based'
+          ? []
+          : resolveRuntimeTargetNames(opts.gateway, opts.runtime ?? opts.agent, projectSpec);
+
       const record: ABTestJobRecord = {
         type: 'ab-test',
         id: createResult.abTestId,
@@ -203,6 +211,8 @@ export const abTestHandler: ABTestHandler = {
         mode: opts.mode,
         gatewayArn,
         gatewayName: opts.gateway,
+        targetName: targetNames.length === 1 ? targetNames[0] : undefined,
+        targetCandidates: targetNames.length > 1 ? targetNames : undefined,
         roleArn,
         roleCreatedByCli,
         variants: built.variantSummaries,