Rename client-side "preflight" to "provision validation" (#7113)#8844
Rename client-side "preflight" to "provision validation" (#7113)#8844vhvb1989 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Renames azd’s local client-side “preflight” provisioning checks to “provision validation”, splits the existing config gate so local validation and ARM ValidatePreflight can be disabled independently, and updates related telemetry/docs/tests to reduce terminology confusion.
Changes:
- Split the previous
provision.preflightgate intovalidation.provision(local checks) vsprovision.preflight(server-side ARM ValidatePreflight), and update the Bicep provider flow accordingly. - Rename telemetry event/fields from
validation.preflight*tovalidation.provision*and outcomes fromaborted_by_*tocanceled_by_*, updating metrics-audit docs and public telemetry reference. - Rename/refresh UX report + snapshots and functional/unit tests to match the new “provision validation” terminology.
Reviewed changes
Copilot reviewed 34 out of 46 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/specs/metrics-audit/telemetry-schema.md | Renames the validation event/fields and updates outcomes in the schema doc. |
| docs/specs/metrics-audit/feature-telemetry-matrix.md | Updates provision command telemetry inventory to the new event/field names. |
| docs/specs/exegraph/spec.md | Updates terminology in exegraph spec (“validation-cancel” wording). |
| docs/reference/telemetry-data.md | Updates public telemetry reference for the renamed validation event/fields/outcomes. |
| docs/concepts/glossary.md | Renames glossary entry to “Provision Validation” and updates the disable config key. |
| docs/architecture/system-overview.md | Updates high-level provisioning pipeline terminology. |
| docs/architecture/provisioning-pipeline.md | Updates pipeline stage naming and documents the new config split. |
| cli/azd/test/functional/testdata/samples/ai-quota/sub-deployment/infra/main.bicep | Updates sample comment to “validation” terminology. |
| cli/azd/test/functional/testdata/samples/ai-quota/rg-deployment/infra/main.bicep | Updates sample comment to “validation” terminology. |
| cli/azd/test/functional/testdata/samples/ai-quota/README.md | Updates sample README references and test filename reference. |
| cli/azd/test/functional/provision_validation_quota_test.go | Renames and updates functional tests to “provision validation” naming and stdin helpers. |
| cli/azd/resources/config_options.yaml | Documents the new validation.provision config option and clarifies provision.preflight. |
| cli/azd/pkg/output/ux/testdata/TestProvisionValidationReport_Snapshot_RoleAssignmentMissing.snap | New/updated snapshot for renamed UX report output. |
| cli/azd/pkg/output/ux/testdata/TestProvisionValidationReport_Snapshot_RoleAssignmentConditional.snap | New/updated snapshot for renamed UX report output. |
| cli/azd/pkg/output/ux/testdata/TestProvisionValidationReport_Snapshot_ReservedResourceName.snap | New/updated snapshot for renamed UX report output. |
| cli/azd/pkg/output/ux/testdata/TestProvisionValidationReport_Snapshot_AllWarningsCombined.snap | New/updated snapshot for renamed UX report output. |
| cli/azd/pkg/output/ux/testdata/TestProvisionValidationReport_Snapshot_AiModelQuotaExceeded.snap | New/updated snapshot for renamed UX report output. |
| cli/azd/pkg/output/ux/testdata/TestProvisionValidationReport_Snapshot_AiModelNotFound.snap | New/updated snapshot for renamed UX report output. |
| cli/azd/pkg/output/ux/provision_validation_report.go | Renames preflight report UX types to provision validation report equivalents. |
| cli/azd/pkg/output/ux/provision_validation_report_test.go | Updates tests and snapshots for the renamed UX report. |
| cli/azd/pkg/infra/provisioning/provider.go | Lowercases DeploymentStateSkipped value and renames the “aborted” skip reason. |
| cli/azd/pkg/infra/provisioning/manager.go | Updates skipped-reason handling to the new “validation canceled” constant. |
| cli/azd/pkg/infra/provisioning/bicep/role_assignment_check_test.go | Renames test helpers/types to provision validation equivalents. |
| cli/azd/pkg/infra/provisioning/bicep/provision_validation.go | Renames and refactors local validation pipeline types and temp file naming. |
| cli/azd/pkg/infra/provisioning/bicep/provision_validation_test.go | Updates unit tests for the renamed validation pipeline. |
| cli/azd/pkg/infra/provisioning/bicep/bicep_provider.go | Splits config gates, renames telemetry + outcomes, and separates local vs ARM preflight steps. |
| cli/azd/pkg/infra/provisioning/bicep/bicep_provider_test.go | Updates tests to assert renamed telemetry keys/outcomes and method names. |
| cli/azd/pkg/infra/provisioning/bicep/bicep_provider_reserved_names_test.go | Updates severity enum name in reserved-names validation tests. |
| cli/azd/pkg/infra/provisioning/bicep/bicep_provider_coverage_test.go | Updates coverage tests for renamed constructor/types. |
| cli/azd/magefile.go | Updates excluded playback test names to new provision validation test names. |
| cli/azd/internal/tracing/fields/fields.go | Renames tracing attribute keys for provision validation. |
| cli/azd/internal/tracing/events/events.go | Renames validation event constant to validation.provision. |
| cli/azd/internal/errors.go | Updates comment to reflect provision validation decline behavior. |
| cli/azd/internal/cmd/provision_test.go | Renames regression test and updates skipped reason constant. |
| cli/azd/internal/cmd/provision_graph.go | Updates the graph path sentinel/error translation for validation-canceled behavior. |
| cli/azd/docs/environment-variables.md | Updates AZD_DEPLOYMENT_ID_FILE docs to reference “canceled by provision validation”. |
| cli/azd/docs/design/provision-validation.md | Renames and updates design doc for the new feature naming and config split. |
| cli/azd/cmd/middleware/error_test.go | Updates test text for wrapped ErrAbortedByUser case. |
| cli/azd/CHANGELOG.md | Adds breaking change entry describing rename + config split + telemetry rename. |
| cli/azd/.vscode/cspell.yaml | Updates cspell override path to the renamed Go file. |
Comments suppressed due to low confidence (2)
cli/azd/docs/design/provision-validation.md:104
- azd-code-reviewer: This section documents check return values as
*ProvisionValidationCheckResult, but the implementation returns a slice ([]ProvisionValidationCheckResult). The docs should match the actual API so new checks are written correctly.
cli/azd/docs/design/provision-validation.md:114 - azd-code-reviewer: The "Adding a New Check" snippet uses
validator.AddCheck(func(...) (*ProvisionValidationCheckResult, error) { ... }), but the code registers checks viaProvisionValidationCheck{RuleID, Fn}whereFnreturns([]ProvisionValidationCheckResult, error). The snippet should be updated to prevent copy/paste errors.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 36 out of 48 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
cli/azd/pkg/output/ux/provision_validation_report.go:112
- azd-code-reviewer: The JSON envelope message here still says "validation: ..." even though the feature is now consistently called "provision validation" elsewhere. This string can show up in JSON output/logging, so keeping the label consistent makes it easier to interpret and search.
cli/azd/docs/design/provision-validation.md:200 - azd-code-reviewer: This design doc describes check functions returning
*ProvisionValidationCheckResult, but the implementation now returns[]ProvisionValidationCheckResult(a slice) from each check function. Updating the type in the doc keeps the design reference accurate for anyone adding new validation rules.
vhvb1989
requested review from
JeffreyCA,
RickWinter,
hemarina and
tg-msft
as code owners
jongio
left a comment
jongio
left a comment
There was a problem hiding this comment.
The rename is well-structured and the config gate split into two independent keys (validation.provision vs provision.preflight) is a solid design choice. The function decomposition into traceLocalProvisionValidation + runLocalProvisionValidation cleanly separates tracing concerns from validation logic.
One issue worth addressing: after this PR, both ErrOperationCancelled and ErrAbortedByUser have the identical message "operation canceled by user". These sentinels serve different semantic roles (the first is for Ctrl+C / explicit cancellation, the second is for the user declining to proceed with exit code 0). Identical messages make log analysis and debugging harder. See inline comment for details.
hemarina
left a comment
hemarina
left a comment
There was a problem hiding this comment.
Nice cleanup — disambiguating "preflight" into three distinct meanings has been a long-standing source of confusion, and splitting the local vs. server-side gates is the right model. The telemetry rename + downstream notifications (#1790/#1792) and the docs sync are thorough.
One thing I'd like to confirm before this ships — the config split has no migration path, and the silent re-enablement is a behavior change with real blast radius:
Today on main, a single provision.preflight off skips the entire validatePreflight block — i.e., both the local client-side checks (newLocalArmPreflight: role-assignment, AI-model quota, reserved-names) and the server-side target.ValidatePreflight call. After this PR, provision.preflight off gates only the ARM call, and the local checks key off the new validation.provision. So any user who set provision.preflight off will have local validation silently re-enabled on upgrade.
That matters because the local checks auto-abort on errors without prompting:
if report.HasErrors() {
p.console.Message(ctx, "... deployment aborted.")
return true, nil // skips deploy, no prompt — fires even under --no-prompt/CI
}So someone who set provision.preflight off to get past a false-positive local error (or in a CI pipeline) can see a previously-green azd provision/azd up start failing after an azd upgrade, until they discover and set validation.provision off. The CHANGELOG flags this as breaking, but the failure mode is silent — the old key just quietly stops affecting local validation.
Could we soften this with a back-compat shim — e.g., if validation.provision is unset and provision.preflight=off, also skip local validation, emitting a one-time deprecation notice pointing at the new key? That preserves existing users' intent through the rename. If the silent re-enablement is intentional (e.g., we want those users back on local validation), that's a defensible call — just want to confirm it's deliberate rather than an artifact of the gate split.
vhvb1989
force-pushed
the
rename-validate-from-localPreflight
branch
from
7877350 to
1bf6a0a
Compare
📋 Prioritization NoteThanks for the contribution! The linked issue isn't in the current milestone yet. |
vhvb1989
force-pushed
the
rename-validate-from-localPreflight
branch
from
1bf6a0a to
bd608aa
Compare
vhvb1989
requested review from
glharper,
huimiu,
therealjohn,
trangevi and
trrwilson
as code owners
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 49 out of 62 changed files in this pull request and generated 4 comments.
Files not reviewed (1)
- cli/azd/pkg/azdext/validation.pb.go: Generated file
Comments suppressed due to low confidence (1)
cli/azd/docs/design/provision-validation.md:345
- azd-code-reviewer: This doc string still uses British spelling "cancelled"; elsewhere the PR standardizes on "canceled" for provision validation terminology. Aligning the spelling avoids confusing readers and keeps text searchable.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 50 out of 63 changed files in this pull request and generated 3 comments.
Files not reviewed (1)
- cli/azd/pkg/azdext/validation.pb.go: Generated file
Comments suppressed due to low confidence (1)
cli/azd/docs/design/provision-validation.md:344
- azd-code-reviewer: This paragraph still quotes the old British-spelling UX text, but
wrapProvisionErrornow emits “Provisioning was canceled.” Keep the design reference synchronized with the actual user-facing message.
| if val, exists := userConfig.GetString("validation.provision"); exists && val == "off" { | ||
| skipValidation = true | ||
| } | ||
| if val, exists := userConfig.GetString("provision.preflight"); exists && val == "off" { | ||
| skipPreflight = true | ||
| skipArmPreflight = true |
| val, exists := userConfig.GetString("validation.provision") | ||
| return exists && val == "off" |
| ProvisionValidationOutcomeKey = AttributeKey{ | ||
| Key: attribute.Key("validation.provision.outcome"), |
jongio
left a comment
jongio
left a comment
There was a problem hiding this comment.
Re-reviewed against the latest commit. The earlier concern is resolved: ErrAbortedByUser now carries a distinct message ("operation declined by user") instead of colliding with ErrOperationCancelled's "operation canceled by user", and every caller dispatches through errors.Is, so the string change carries no behavioral risk. The cancel/decline spelling is normalized to "canceled" throughout, and the no-findings telemetry path now emits the same empty attribute shape as the other paths.
One non-code heads up: the branch is currently in conflict with main and needs a rebase before it can merge.
RickWinter
left a comment
RickWinter
left a comment
There was a problem hiding this comment.
This renames the client-side provisioning validation surface while leaving ARM preflight terminology intact. The extension protocol compatibility issue below must be addressed before merge.
| // carries ARM-specific data (template, parameters, resource snapshot) and | ||
| // therefore only runs for Bicep-provisioned deployments. | ||
| ValidationCheckTypeLocalPreflight = "local-preflight" | ||
| ValidationCheckTypeArmProvision = "arm-provision" |
There was a problem hiding this comment.
azd-code-reviewer: I think we need to keep accepting local-preflight and map it to arm-provision for at least one release. Existing installed extensions register the former value over gRPC, and registration still succeeds, but their checks will no longer be dispatched after an azd upgrade.
There was a problem hiding this comment.
Confirmed this is a real break. ValidationCheckTypeLocalPreflight = "local-preflight" shipped in the released azdext SDK (1.27.1 on 2026-07-09, still present on main / 1.28.0), so it's a public gRPC contract value, not just an internal name. Extensions built against that SDK register their check with local-preflight, and the dispatcher matches registered checks by exact checkType string, so after this rename azd core only dispatches arm-provision. Those extensions still register successfully but never get invoked, and the failure is silent.
The bundled azure.ai.agents extension already moved to ValidationCheckTypeProvision, so it's unaffected, but any externally distributed extension pinned to 1.27.x is. Accepting local-preflight as an alias for arm-provision for at least one release, as you suggest, keeps those working. A CHANGELOG breaking-change note for the eventual removal would help consumers plan the migration.
jongio
left a comment
jongio
left a comment
There was a problem hiding this comment.
Following up on my review: my earlier approval predates confirmation of a breaking change, and I don't think this is mergeable as-is. Two things are still open.
The local-preflight -> arm-provision rename in validation_provider.go breaks the released azdext gRPC contract. ValidationCheckTypeLocalPreflight = "local-preflight" shipped in the 1.27.1 SDK and is still on main, so extensions built against it register local-preflight over gRPC; registration keeps succeeding, but core will only dispatch arm-provision, so those checks silently stop running. As RickWinter called out, core should keep accepting local-preflight as an alias for arm-provision for at least one release. See the open thread on validation_provider.go.
Separately, the branch is currently conflicting with main and needs a rebase, and the gate-split test gaps flagged on bicep_provider.go and provision_validation.go are still open.
Once the alias is in place and the conflicts are resolved I'm happy to re-approve.
jongio
left a comment
jongio
left a comment
There was a problem hiding this comment.
Re-reviewed after the rebase. My earlier note about the two cancellation sentinels is addressed - ErrAbortedByUser now reads "operation declined by user", distinct from ErrOperationCancelled.
Two things still stand between this and merge:
The check type rename local-preflight to arm-provision in validation_provider.go remains a breaking change to a released gRPC contract. ValidationCheckTypeLocalPreflight = "local-preflight" shipped in azdext 1.27.1, so extensions pinned to that SDK register their check with local-preflight, and the dispatcher matches by exact checkType string. After this rename core dispatches only arm-provision, so those extension checks register fine but silently stop running. Accepting local-preflight as an alias for arm-provision for one release keeps them working. This is tracked in the open thread on validation_provider.go.
The branch also conflicts with main now (merge state is dirty), so it needs a conflict resolution before it can go in.
…ion" (#7113) Rename the client-side, pre-deployment check feature from "local preflight" to "provision validation" to avoid confusion with the Azure ARM Preflight API. User-facing changes: - Split config gates: new `validation.provision` key gates the client-side (local) checks; existing `provision.preflight` now gates ONLY the server-side ARM preflight call (previously a single key disabled both). - Rename the Bicep-only extension check type `local-preflight` -> `arm-provision`; the provider-agnostic `provision` check type (added in #9019) is unchanged. - Telemetry event `validation.preflight` -> `validation.provision`, fields `validation.preflight.*` -> `validation.provision.*`. - Outcome vocabulary `aborted_by_*` -> `canceled_by_*`; "abort" UI language replaced with "canceled". - `DeploymentStateSkipped` value corrected to lowercase "deployment state". Internal renames: PreflightReport -> ProvisionValidationReport, local_preflight.go -> provision_validation.go, PreflightCheck* -> ValidationCheck*, newLocalArmPreflight -> newProvisionValidator, PreflightValidationEvent -> ProvisionValidationEvent, Preflight*Key -> ProvisionValidation*Key, ErrProvisionValidationAborted -> ErrProvisionValidationCanceled, and related test/recording/snapshot renames. Server-side ARM preflight (`ValidatePreflight`), `mage preflight` dev checks, and generic third-party "preflight" usages are intentionally left unchanged. Closes #7113 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- errors.go: give ErrAbortedByUser a distinct message ("operation declined
by user") so it no longer collides with ErrOperationCancelled in logs/output
(jongio).
- provision_graph.go / up_graph.go: standardize user-facing message and
comments on American spelling "canceled" (was "cancelled").
- provision_validation.go: emit the empty/zero validation.provision.* field
shape (diagnostics + warning/error counts) in the no-results branch so
telemetry is consistent with the Bicep "arm-provision" dispatch.
- bicep_provider_test.go: rename table-test cases "aborted by *" ->
"canceled by *" to match the canceled_by_* outcomes.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989
force-pushed
the
rename-validate-from-localPreflight
branch
from
a52274b to
ce01ed5
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 49 out of 62 changed files in this pull request and generated 1 comment.
Files not reviewed (1)
- cli/azd/pkg/azdext/validation.pb.go: Generated file
Comments suppressed due to low confidence (1)
cli/azd/docs/design/provision-validation.md:344
- azd-code-reviewer: This doc still says "cancelled" (British spelling), but the codebase/user-facing UX in this PR standardizes on "canceled". Aligning this avoids confusion and makes log/output searches consistent.
| span.SetAttributes(fields.ProvisionValidationCheckTypeKey.String(azdext.ValidationCheckTypeProvision)) | ||
|
|
||
| checkContext := m.provisionValidationContext() |
jongio
left a comment
jongio
left a comment
There was a problem hiding this comment.
The check type rename in validation_provider.go is still a breaking change to the released azdext contract on this commit, so I can't move back to approve yet. Details inline. That alias is the one thing left blocking merge from my side; the cancellation message and empty-results telemetry from my earlier notes are handled in the latest commit.
| // carries ARM-specific data (template, parameters, resource snapshot) and | ||
| // therefore only runs for Bicep-provisioned deployments. | ||
| ValidationCheckTypeLocalPreflight = "local-preflight" | ||
| ValidationCheckTypeArmProvision = "arm-provision" |
There was a problem hiding this comment.
ValidationCheckTypeLocalPreflight = "local-preflight" has shipped in every azdext release since 1.26.0 and is still on main, so extensions built against that SDK register their check with the string local-preflight. Core dispatches by exact checkType, so once this renames the value to arm-provision, those extensions keep registering without error but their checks silently stop running. Keep accepting local-preflight as an alias for arm-provision for at least one release so installed extensions don't break. Same point as the open thread on this line.
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
Fixes #7113
Summary
Renames azd's client-side "local preflight" provisioning check to "provision validation" to avoid confusion with the Azure ARM Preflight API. The term "preflight" was overloaded across three unrelated meanings; this PR renames only the local client-side feature and leaves the server-side ARM preflight call and the
mage preflightdev checks untouched.Behavioral change — config gate split
The single
provision.preflightgate is split into two independent keys:provision.preflight offtarget.ValidatePreflight)validation.provision off(new)Issue sub-tasks addressed
DeploymentStateSkippedvalue to"deployment state"aborted_by_*→canceled_by_*)Telemetry rename
validation.preflightvalidation.provisionvalidation.preflight.*fieldsvalidation.provision.*aborted_by_user/aborted_by_errorscanceled_by_user/canceled_by_errorsTelemetry docs (
telemetry-data.md, metrics-audit schema + matrix), the design doc, architecture docs, config option descriptions, and CHANGELOG are all updated in sync. Downstream analytics issues (Azure/azure-dev-pr#1790, Azure/azure-dev-pr#1792) have been notified with the field mapping and forward-compatible KQL.Renamed files
bicep/local_preflight.go→bicep/provision_validation.go(+ test)output/ux/preflight_report.go→output/ux/provision_validation_report.go(+ test, + 6 snapshots)test/functional/preflight_quota_test.go→provision_validation_quota_test.go(+ 6 recordings)docs/design/local-preflight-validation.md→docs/design/provision-validation.mdValidation
go build ./..., targeted unit tests, andcmdfig/usage snapshot tests passgolangci-lint run— 0 issuescspell,gofmt -s, copyright-check — cleango.mod/go.sumchanges