feat(ci): add prompt and managed agent cloud e2e pipeline

.azure-pipelines/README.md

@@ -8,6 +8,7 @@ or authorize an Azure DevOps pipeline.
 |---|---|---|
 | [hosted-agents-samples-ci.yml](hosted-agents-samples-ci.yml) | Python/C# hosted agents and their test fixtures | Changed samples on PRs; full discovery on relevant `main` pushes, daily at 09:00 UTC, and manual runs |
 | [private-bicep-pr-ci.yml](private-bicep-pr-ci.yml) | Bicep infrastructure templates and diagnostic tooling | Changed templates on PRs and `main` pushes; manual `samplePath` or `validateAll` |
+| [prompt-agents-samples-ci.yml](prompt-agents-samples-ci.yml) | CLI prompt/managed azd agents and their contracts | Changed samples on PRs; all eligible samples on relevant `main` pushes, daily at 09:00 UTC, and manual runs |
 
 The GitHub [policy checks](../.github/workflows/hosted-agent-policies.yml) remain
 credential-free. The public `trusted` check and
@@ -179,3 +180,66 @@ python .azure-pipelines/scripts/bicep/test_bicep_pipeline.py
 Live coverage requires an authorized Azure DevOps run after setup. Successful
 offline tests do not claim that service connections, cloud quota, or deployments
 have been validated.
+
+## Prompt and managed agent E2E
+
+Register [prompt-agents-samples-ci.yml](prompt-agents-samples-ci.yml) as a separate
+Azure DevOps pipeline connected to this repository. Like the hosted-agent E2E
+pipeline, it runs for changes on PRs targeting `main`, relevant `main` pushes,
+and a daily full run at 09:00 UTC; it also supports manual runs. It does not
+post per-sample GitHub statuses or affect the public `trusted` check. Require
+maintainer approval before fork PR builds run, disable fork secrets and
+full-access tokens, and enforce resource approvals/checks on the service
+connection and variable group: PRs can edit pipeline YAML. Authorize a dedicated
+service connection and variable group `samples-prompt-agents-ci`:
+
+| Variable | Purpose |
+|---|---|
+| `AZURE_SERVICE_CONNECTION` | Service connection for the dedicated test subscription |
+| `SKIP_PROVISION` | Required: `true` borrows the existing project/model; `false` provisions a disposable project/model per cell |
+| `AZURE_AI_PROJECT_ID`, `AZURE_AI_PROJECT_ENDPOINT`, `AZURE_AI_MODEL_DEPLOYMENT_NAME` | Required when `SKIP_PROVISION=true`: existing Foundry project and deployment |
+| `AZURE_SUBSCRIPTION_ID`, `AZURE_LOCATION` | Required when `SKIP_PROVISION=false`: subscription and region for disposable project/model |
+
+The connection needs project data-plane rights to create, invoke, inspect,
+and delete agents/toolboxes, plus management-plane access for project/model
+resolution. `AzureCLI@2` authenticates azd through the Azure CLI session. Like
+hosted E2E, `SKIP_PROVISION=true` uses an existing project/model; the runner
+creates and deletes only its temporary agents (and owned toolboxes for managed
+samples). With `SKIP_PROVISION=false`, each cell creates a fresh resource group,
+provisions a project and the sample's declared model deployment with `azd`,
+then tears down the cell-owned resource group after agent cleanup. The latter
+requires permission to create/delete resource groups and model deployments,
+available quota in `AZURE_LOCATION`, and a dedicated disposable subscription.
+It fails closed if ownership or resource-group deletion cannot be verified.
+Registration, variable-group and service-connection authorization, resource
+approvals, and any required-check settings must be configured outside the YAML.
+
+By default, PRs run changed samples, while pushes, schedules, and manual runs
+run all eligible samples. `sampleFilter` defaults to `*` (all); a comma-separated
+list narrows the selection using case-insensitive OR substrings. `discoveryMode`
+defaults to `auto` and can be overridden with `all` or `changed` for manual
+runs; `baseRef` defaults to `origin/main`. Changed mode compares committed HEAD
+with the merge base; the PR target branch is explicitly fetched.
+Discovery scans prompt-agent `azure.yaml` manifests, applies
+[`blacklist.json`](scripts/prompt-agents/blacklist.json), and validates all
+non-skipped contracts before filtering. The prompt and managed matrices run one
+cell at a time per profile, not under a project-wide lock across pipeline runs.
+
+Each cell hydrates its record from `PromptAgentSamplesMatrix`, runs the azd
+lifecycle, and attempts recovery cleanup in an authenticated `always()` task.
+Provisioned cells run an additional authenticated `always()` teardown step;
+their summaries require verified resource-group deletion.
+Worker loss or late remote completion can still leave resources to reclaim.
+The matrix, per-cell `PromptAgentResult-<comboId>`, and `PromptAgentSummary`
+artifacts contain prompts, model output, project identifiers, and diagnostics;
+restrict access and do not publish azd authentication/config directories.
+No matches produce a `NoSamples` stage instead of a live job. See the
+[runner documentation](scripts/prompt-agents/README.md) for supported manifests,
+contract capabilities, and credential-free local commands.
+
+The published `01-basic` and `03-mcp-tools` manifests have contracts; the
+private `04-foundry-toolbox` contract is not included because that sample has
+no published manifest. Contract discovery fails closed on missing sample
+manifests or unsupported fixture paths, even with a manual filter. No managed
+sample manifest is currently published, so managed live
+coverage awaits a compatible public sample and contract.

.azure-pipelines/prompt-agent-tests/cli/01-basic/test-spec.yml

@@ -0,0 +1,17 @@
+version: 1
+sample:
+  owner: zhihuan
+  experiences: [azd]
+tests:
+  - name: arithmetic-and-conversation
+    turns:
+      - input: "What is two plus two? Answer with just the number."
+        assertions:
+          - source: assistant_text
+            type: equals
+            value: '4'
+      - input: "Add three to your previous answer. Answer with just the number."
+        assertions:
+          - source: assistant_text
+            type: equals
+            value: '7'

.azure-pipelines/prompt-agent-tests/cli/03-mcp-tools/test-spec.yml

@@ -0,0 +1,18 @@
+version: 1
+sample:
+  owner: zhihuan
+  experiences: [azd]
+tests:
+  - name: microsoft-learn-documentation-answer
+    turns:
+      - input: >-
+          Use the Microsoft Learn MCP server to find the official documentation
+          for Azure Developer CLI. Explain briefly what azd does and include
+          the official documentation URL.
+        assertions:
+          - source: assistant_text
+            type: contains
+            value: Azure Developer CLI
+          - source: assistant_text
+            type: regex
+            value: 'https://learn\.microsoft\.com/(?:[a-z]{2}-[a-z]{2}/)?azure/developer/azure-developer-cli(?:/|[?#\s)]|$)'