Publish the mxc-sdk crate closure to crates.io - #716
Publish the mxc-sdk crate closure to crates.io#716Darren Hoehna (dhoehna) wants to merge 25 commits into
Conversation
Make the mxc-sdk crate closure publishable and add official-build packaging plus a guarded ESRP release flow with dependency ordering, checksum verification, propagation waits, and safe retries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f199a77f-51ec-4441-baa5-b87df8177b83
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f199a77f-51ec-4441-baa5-b87df8177b83
There was a problem hiding this comment.
Pull request overview
Adds opt-in packaging and ESRP publication of the mxc-sdk crate dependency closure to crates.io.
Changes:
- Adds publishable metadata and versioned internal dependencies for 18 crates.
- Adds packaging, dependency-order validation, checksum verification, and retry-safe publishing.
- Keeps crate packaging disabled pending Cargo #17196 resolution.
Reviewed changes
Copilot reviewed 24 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
src/Cargo.toml |
Adds repository metadata and dependency versions. |
src/mxc_telemetry/Cargo.toml |
Adds repository metadata. |
src/core/wxc_common/Cargo.toml |
Adds package metadata and NanVix version. |
src/core/mxc-sdk/Cargo.toml |
Adds repository metadata. |
src/core/mxc_pty/Cargo.toml |
Adds publishable metadata. |
src/core/mxc_engine/Cargo.toml |
Adds repository metadata. |
src/core/generated/base_container_specification/Cargo.toml |
Makes generated bindings publishable. |
src/backends/wslc/common/Cargo.toml |
Adds publishable metadata. |
src/backends/windows_sandbox/lifecycle/Cargo.toml |
Adds license and package metadata. |
src/backends/windows_sandbox/common/Cargo.toml |
Adds publishable metadata. |
src/backends/seatbelt/common/Cargo.toml |
Adds publishable metadata. |
src/backends/nanvix/runner/Cargo.toml |
Adds metadata and dependency version. |
src/backends/nanvix/common/Cargo.toml |
Adds repository metadata. |
src/backends/lxc/common/Cargo.toml |
Adds publishable metadata. |
src/backends/isolation_session/common/Cargo.toml |
Adds publishable metadata. |
src/backends/isolation_session/bindings/Cargo.toml |
Makes bindings publishable. |
src/backends/hyperlight/common/Cargo.toml |
Adds publishable metadata. |
src/backends/bubblewrap/common/Cargo.toml |
Adds publishable metadata. |
src/backends/appcontainer/common/Cargo.toml |
Adds publishable metadata. |
.azure-pipelines/templates/Publish.CratesIo.Job.yml |
Defines ordered ESRP publication workflow. |
.azure-pipelines/templates/Package.Crates.Job.yml |
Packages the crate closure as an artifact. |
.azure-pipelines/templates/1ES.Build.Stages.yml |
Adds the opt-in packaging stage. |
.azure-pipelines/scripts/crates_release.py |
Validates, stages, and verifies crate releases. |
.azure-pipelines/1ES.Release.yml |
Adds selectable npm and crates.io release stages. |
.azure-pipelines/1ES.Build.Official.yml |
Exposes the crate-packaging parameter. |
| default: false | ||
|
|
||
| jobs: | ||
| - deployment: publish_crates |
There was a problem hiding this comment.
Good catch - this was correct and would have failed validation before any step ran.
Confirmed against the 1ES release-job docs, whose deployment-job example annotates the field as required verbatim: environment: MyEnvironment # Required, name of the environment.
Fixed in d408846 by taking your second suggestion: converted to a standard job with templateContext.type: releaseJob and flattened strategy.runOnce.deploy.steps into plain steps:. That matches the NPM release job already working in 1ES.Release.yml, and it avoids provisioning an ADO Environment (and the approval gates that come with it) purely to satisfy the schema. The 1ES docs show the standard-job form for release jobs in four of their five examples, so it is the better-trodden path here.
Two independent blockers prevented the crates.io release path from working. B1 - packaging. With `[source.crates-io] replace-with = "Mxc-Azure-Feed"` active, `cargo package` resolves a just-packaged sibling through the replacement feed instead of its own in-memory overlay, so the sibling 404s (rust-lang/cargo#17196): cargo registers the overlay under the pre-replacement SourceId but looks it up under the post-replacement one. Pass `--registry Mxc-Azure-Feed` so the source override is not consulted. The resulting .crate is byte-identical to a no-replacement baseline, so no feed URL leaks into the published artifact. `--registry` defaults to `crates-io`, leaving local and developer builds unchanged. B2 - releasing. The release job made four reads of index.crates.io (probe, status, verify-dependencies, wait). The 1ES CFSClean isolation policy denies that host, so the job could not succeed on the pool regardless of the artifacts. Those reads are removed; ESRP publishes server-side and is unaffected. Correctness is preserved by three existing layers: 1. sibling version-req match is enforced by `cargo package` itself (exit 101) 2. leaf-first ordering is validated offline by _validate_release_graph 3. dependency name existence is enforced server-side by crates.io via a Postgres lookup on the crates table, so index propagation never gates sequential publishing `verify-order` now accepts a correctly ordered subset of the packaged order, so a partially failed release can be resumed by narrowing `crateOrder`; it emits warnings naming every dependency it therefore assumes is already published. release-order.json records sha256 per crate for out-of-band auditing. packageCrates and publishCrates still default to false. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f199a77f-51ec-4441-baa5-b87df8177b83
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 24 out of 25 changed files in this pull request and generated 1 comment.
Suppressed comments (2)
.azure-pipelines/templates/Publish.CratesIo.Job.yml:27
- The PR description promises automatic crates.io checksum/existence checks and checksum-aware reruns, but this implementation explicitly performs no registry reads and requires operators to remove crates manually. That means a resumed run can publish dependents based only on an unchecked operator assertion, and an unchanged rerun is not safe as described. Either restore the advertised preflight/ledger behavior in a network-capable stage or update the release design and PR description to make this manual safety boundary explicit.
# RESUMING A FAILED RELEASE: crates.io rejects a duplicate version, so simply
# re-running this job unchanged fails on the first crate that already landed.
# The pool cannot ask crates.io what landed, and 1ES forbids `checkout` in a
# release job, so there is no automated ledger. Instead the operator re-queues
# the release with the already-published crates removed from `crateOrder`.
# `verify-order` accepts that subset only if it is still in packaged leaf-first
# order, and logs every dependency it is consequently assuming is already live.
.azure-pipelines/scripts/crates_release.py:229
- This disables Cargo's only check that each generated archive can compile after workspace path dependencies are rewritten to registry dependencies. A successful workspace build does not cover missing packaged files or normalized-manifest problems, so the release can publish crates that consumers cannot build. Remove
--no-verify(the private-registry workaround should also support verification), or add an equivalent build-from-archive gate before this stage is enabled.
"--no-verify",
| source = _crate_file(args.order_file, entry) | ||
| if not os.path.isfile(source): | ||
| print(f"Crate file not found: {source}") | ||
| return 1 | ||
| shutil.copy2(source, os.path.join(out_dir, entry["file"])) |
The crates release job was a `deployment` job with no `environment:`. 1ES
documents `environment` as required for the deployment form of a release job
("environment: MyEnvironment # Required, name of the environment"), so the
stage would have failed YAML validation before any step ran.
Convert it to a standard `job` with `templateContext.type: releaseJob` and
flatten the `strategy.runOnce.deploy.steps` block into plain `steps:`. This
matches the NPM release job already working in 1ES.Release.yml, and avoids
provisioning an ADO Environment (and its approval gates) purely to satisfy the
schema.
Reported by the Copilot PR reviewer on microsoft#716.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f199a77f-51ec-4441-baa5-b87df8177b83
Two outstanding items that need a decisionBoth are external to this PR - no amount of code review resolves them, and both have lead time, so they are worth starting now rather than at release time. 1.
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 24 out of 25 changed files in this pull request and generated no new comments.
Suppressed comments (2)
.azure-pipelines/templates/Publish.CratesIo.Job.yml:94
- The release starts the next dependent crate immediately after ESRP reports completion, but crates.io index updates are asynchronous; the new dependency may not yet be resolvable, causing the next publish to fail with “dependency does not exist.” The PR’s Safety section says propagation is awaited, but this loop has no delay or retry. Add a supported propagation gate after each publish (preferably polling the sparse index from a network-enabled job, or a conservative wait if this pool must remain isolated).
- task: EsrpRelease@12
.azure-pipelines/templates/Publish.CratesIo.Job.yml:30
- This explicitly contradicts the PR’s Safety claim that reruns skip versions already published with the matching checksum. The implementation instead fails on duplicates and asks an operator to remove entries without verifying the published checksum, so a partial release is neither automatically resumable nor checksum-safe as described. Restore the live checksum/idempotency check (for example in a permitted network-enabled gate or trusted publication ledger), or revise the stated release guarantees before relying on this workflow.
# RESUMING A FAILED RELEASE: crates.io rejects a duplicate version, so simply
# re-running this job unchanged fails on the first crate that already landed.
# The pool cannot ask crates.io what landed, and 1ES forbids `checkout` in a
# release job, so there is no automated ledger. Instead the operator re-queues
# the release with the already-published crates removed from `crateOrder`.
Releasing crates previously took two queued runs: MXC-Official-Build with packageCrates to produce the artifact, then MXC-Official-Release with publishCrates to consume it. The intent is that releasing is one click. 1ES forbids `checkout` in a release JOB, but a regular job in the same PIPELINE may check out and build - 1ES documents this pattern explicitly, wiring a release job to a sibling build job's artifact via `input: pipelineArtifact` with `artifact:` rather than a cross-pipeline `pipeline:`/`artifactName:` pair. Add a Package_Crates_For_Release stage to 1ES.Release.yml that runs Package.Crates.Job.yml, and make Publish_to_CratesIo depend on it. The release job's artifact input switches from the cross-pipeline form to the same-run form. Queueing 1ES.Release.yml with publishCrates now packages the closure and then publishes it in one run. cratesDryRun packages and verifies while skipping ESRP, which is the cheap way to exercise the full 18-crate closure without publishing. The npm release is unchanged and still consumes mxc-npm-sdk-package from MXC-Official-Build. packageCrates on the official build is retained as a CI canary: it detects packaging breakage at merge time without requiring a release run. Note: the packaging job now runs CargoAuthenticate against Mxc-Azure-Feed from the release pipeline rather than the build pipeline. Both run in the same ADO project and so should share a build service identity, but that is unverified - a cratesDryRun run exercises exactly this path and publishes nothing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f199a77f-51ec-4441-baa5-b87df8177b83
| package_args = [ | ||
| "cargo", | ||
| "package", | ||
| "--no-verify", | ||
| "--allow-dirty", | ||
| "--registry", | ||
| args.registry, | ||
| "--manifest-path", | ||
| manifest, | ||
| ] |
| pipeline: MXC | ||
| targetPath: '$(Pipeline.Workspace)/packages' | ||
| artifactName: mxc-npm-sdk-package | ||
| - ${{ if parameters.publishNpm }}: |
Select the release in the Run dialog's "Pipeline version" picker instead of typing a tag into a parameter. The ref the run is queued against now supplies both the pipeline definition and the packaged source, so a release is immutable: changing anything about how a release is built, including this pipeline, requires cutting a new tag. A new Validate_Release_Ref stage fails the run unless Build.SourceBranch matches refs/tags/release/*, and the package and publish stages depend on it, so nothing is checked out or published from a branch, a bare version tag, or a raw commit. The check has to run at runtime because Build.SourceBranch is not available to compile-time template expressions. The stage is marked isSkippable: false so it cannot be deselected in the Run dialog's "Stages to run" panel, which would otherwise make the guard advisory. Re-enable cargo package verification. Passing --registry resolves workspace siblings against the temporary package registry, so rust-lang/cargo#17196 does not apply and --no-verify is unnecessary. --allow-dirty is likewise dropped: the only file the pipeline modifies is the workspace .cargo/config.toml, which lies outside every package directory, so a dirty-tree failure now means a crate source really was modified. The script's --registry default moves to Mxc-Azure-Feed to match, since the old crates-io default was only harmless while verification was off. Verify each staged .crate against the sha256 recorded at packaging time before handing it to ESRP, failing closed on a missing or mismatched digest. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7cf82d0d-a130-47ee-8e7c-409209a3b4f9
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 26 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (2)
docs/crates-release.md:67
- The release procedure will break on the next version bump. These newly versioned path dependencies keep literal
0.7.0requirements (including the two directnanvix_commondependencies), so changing only[workspace.package].versionmakes the local 0.8.0 siblings fail the 0.7.0 requirements;scripts/check-version-sync.jsdoes not update or validate them. The instructions must require updating every first-party dependency requirement in lockstep (and the SDK versions checked by the existing sync gate).
1. Bump the version in `src/Cargo.toml`.
.azure-pipelines/1ES.Release.Crates.yml:80
- The release entry point advertised in the PR description is not present: this new pipeline exposes
cratesDryRun, while the existing1ES.Release.ymlhas nopublishCratesparameter or crates stages. QueueingMXC-Official-ReleasewithpublishCrates: truetherefore cannot perform the documented release. Either integrate these stages into that pipeline or update the PR description to direct operators to the separately registered1ES.Release.Crates.ymlpipeline.
parameters:
- name: cratesDryRun
displayName: 'Dry run — package and verify order but skip ESRP publish'
type: boolean
default: false
Resolves a textual conflict in src/Cargo.toml. Upstream added the learning_mode_core workspace member and its path dependency while this branch added version fields to the dependencies the crates.io release publishes. Both sides are kept. learning_mode_core stays unversioned because it is not in the current 18-crate publish list.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 26 out of 27 changed files in this pull request and generated 2 comments.
Suppressed comments (2)
docs/crates-release.md:156
- This resume procedure repackages the remaining crates from a new commit rather than the original release tag/artifact. If
mainhas advanced, Cargo verifies against the new local sibling sources while consumers resolve already-published siblings from the old tag, allowing an incompatible mixed closure under one version; even a pipeline-only commit changes Cargo's packaged VCS provenance/checksum. Resume from the original artifact/tag while supplying only the remaining order, rather than tagging a newmaincommit.
2. Edit the `crateOrder` default in `.azure-pipelines/1ES.Release.Crates.yml`,
removing the already-published crates, and merge that change to `main`.
`crateOrder` is an `object` parameter; changing it means editing the
pipeline file.
3. Cut a **new** release tag containing that edit (for example
docs/crates-release.md:62
- This release step is incomplete now that the PR adds literal
version = "0.7.0"requirements for first-party dependencies in the workspace and crate-local manifests. Changing onlyworkspace.package.versionto0.8.0leaves packages requiring^0.7.0siblings, socargo packagewill reject the next release. Include those dependency requirements in the version bump instructions.
1. Bump the version in `src/Cargo.toml`.
| "bwrap_common", | ||
| "seatbelt_common", | ||
| "sandbox_spec", | ||
| "appcontainer_common", |
| - name: cratesDryRun | ||
| displayName: 'Dry run — package and verify order but skip ESRP publish' | ||
| type: boolean | ||
| default: false |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 28 out of 29 changed files in this pull request and generated no new comments.
Suppressed comments (4)
docs/crates-release.md:256
- This resume procedure no longer builds from the failed run's exact commit: the
crateOrderedit must be committed before the branch is pushed, andcargo packageembeds that new HEAD in each archive's.cargo_vcs_info.json. The resumed crates therefore have different provenance from the first part of the release. Reuse the original package artifact, or make the verified publish subset a queue-time input while running the original immutable ref.
3. On that branch, edit the `crateOrder` default in
`.azure-pipelines/templates/Publish.CratesIo.Job.yml`, removing the
already-published crates. Keep the remaining entries in their existing
relative order, and **do not** rerun `crates_release.py order` — that
regenerates the full closure and would put the already-published crates
docs/crates-release.md:174
- These defaults contradict both the pipeline definition and the ESRP identity section below: the parameters default to
$(Build.RequestedForEmail), not this individual address. This would give operators incorrect instructions about which identity is submitted.
| **ESRP release owners email** (`esrpOwnersEmail`) | Defaults to `Darren.Hoehna@microsoft.com`; see [ESRP identity](#esrp-identity) below. |
| **ESRP release approvers email** (`esrpApproversEmail`) | Defaults to `Darren.Hoehna@microsoft.com`; see [ESRP identity](#esrp-identity) below. |
docs/crates-release.md:321
- This prerequisite is stale relative to the PR description: definition 198954 is already registered, and the variable group and service connection are authorized. The unresolved authorization is for the
Azure-Pipelines-1ESPT-ExDSharedagent pool (queue 250521), so the current runbook directs operators to fix the wrong resource.
5. **Pipeline registration** — `.azure-pipelines/1ES.Release.Crates.yml` is new
and must be registered as a pipeline in Azure DevOps, and that pipeline must
be authorized to use the `MXC-ESRP-Signing` variable group.
.azure-pipelines/templates/1ES.Build.Stages.yml:100
- This comment incorrectly says the official-build artifact is consumed by the release pipeline.
1ES.Release.Crates.ymlpackages its own ref and consumes only the artifact from that same run; this stage is an independent canary. Keeping the distinction explicit avoids accidentally coupling release operations to the wrong build artifact.
# Source-only (no binaries), so it does not depend on Build_Binaries. Produces
# the `mxc-crates-package` artifact consumed by 1ES.Release.Crates.yml for ESRP-based
# crates.io publishing. Isolated in its own stage so it never gates npm
# packaging or the integration tests that depend on Package_MXC.
Two Run-dialog inputs asked a human for something the machine already
knows, and one silent failure mode had no gate at all.
ESRP owner and approver were string parameters defaulting to
$(Build.RequestedForEmail). Anything under `parameters:` renders as an
editable field in the Run dialog, so both showed up as boxes to fill in --
but an email address is not a decision the person clicking Run should make.
There is no right answer for them to know, and a typo submits a release to
the wrong owner. Both are now fixed to $(Build.RequestedForEmail) in the
YAML, which is the value a default run already used: this removes the
prompt, not the behavior.
The two compile-time ${{ if }} emptiness guards went with them. They could
only ever fire when an operator cleared the box, and nothing can clear it
now. They also could not have caught the real risk: $(Build.RequestedForEmail)
is a non-empty literal token at compile time, so a ${{ }} guard never sees
the resolved value. A runtime guard in the gate stage checks the resolved
value instead, and fails before anything is checked out or packaged.
Separately, `crateOrder` in the publish template is a pasted copy of
`crates_release.py order` output, because ${{ each }} expands before any
script runs. Forgetting to regenerate it failed OPEN: `verify-order`
accepts an ordered subset on purpose, since that is how a part-published
release resumes, so a template missing a newly-added crate PASSES and the
crate is silently never published. The only trace is a PARTIAL RELEASE
warning in a log read afterwards, and it is not repairable by re-running --
the crates that did land cannot be republished.
`check-template` recomputes the order and asserts the template matches it
exactly, wired into the Versioning Checks workflow next to the existing
codegen drift checks. Drift is now a red build at PR time. It does not
block the resume procedure: Build.yml triggers on main, feature/*, and
user/* only, so a trimmed crateOrder on a release/* branch is never checked.
Verified: clean tree passes; a template missing mxc_pty fails naming it as
"would NOT be published"; a swapped mxc_engine/mxc-sdk pair fails as a
reordering. All three YAML files parse and the Run dialog now renders only
dryRun.
Also corrects the operator doc, which claimed the emails defaulted to a
hardcoded address. That was untrue before this change as well.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab22cd6f-f3d9-4325-8397-9e998aa058c3
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 29 out of 30 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
src/Cargo.toml:113
- These hard-coded local dependency requirements become additional version sources, but the existing version-sync check only compares the workspace, npm, and C# package versions. Following the new release guide's instruction to bump
src/Cargo.tomlto 0.8.0 leaves these requirements at^0.7.0, so workspace resolution/cargo packagefails before that documented release can run. Add a CI check that keeps every publishable path dependency requirement synchronized with the workspace version, and include those pins in the version-bump procedure.
wxc_common = { path = "core/wxc_common", version = "0.7.0" }
mxc_engine = { path = "core/mxc_engine", version = "0.7.0" }
mxc-sdk = { path = "core/mxc-sdk" }
appcontainer_common = { path = "backends/appcontainer/common", version = "0.7.0" }
windows_sandbox_common = { path = "backends/windows_sandbox/common", version = "0.7.0" }
| - ${{ else }}: | ||
| - task: EsrpRelease@12 |
An independent review found the previous commit left the irreversible path
open. check-template ran only from Build.yml, which triggers on pushes to
main and on pull requests into main, feature/*, and user/*. A release ref
can be pushed straight to microsoft/mxc without ever opening a pull request,
so a release could be built from a commit the gate never saw. verify-order
does not cover it either -- it accepts an ordered subset on purpose, since
that is how a part-published release resumes -- so a stale crateOrder would
package the full closure, publish a subset, and report success. That is the
exact silent partial publish the previous commit claimed to have closed, and
it cannot be repaired by re-running, because published versions cannot be
republished.
The packaging job now runs check-template itself, before producing any
artifact. A deliberate resume declares a RESUME-SUBSET marker above the
crateOrder default, so the intent travels with the release ref instead of
being typed into the Run dialog. A marked subset is still proved to be an
ordered subsequence of the computed order, and the run logs exactly which
crates it assumes are already live -- trimming is allowed, reordering is not.
The template parser is hardened. It refused to guess before only by
accident: the review showed a block scalar containing a decoy
"- name: crateOrder" could be validated in place of the real, stale
parameter. Two matches now raise rather than pick one. Comments, blank
lines, quoted names, and trailing comments inside the list are handled;
flow-sequence style raises instead of silently returning a wrong list. Every
unreadable shape fails closed, because a short list from a misparse is
indistinguishable from one that was deliberately trimmed.
Documentation corrections, all of which were false as written:
- "doing only the first is what verify-order exists to catch" -- it does
not; that is what check-template is for. The same false claim had
already been fixed in one passage and survived in another.
- The resume section said a release/* branch is exempt from the gate
because it appears in no trigger list. GitHub's pull_request branches
filter matches the BASE branch, so a pull request from release/* into
main does run the check. Only a direct push skips it.
- The subcommand table omitted `order` and `check-template`.
Verified across ten cases: clean tree passes; a missing crate fails naming
it; a swapped pair fails as a reordering; a marked resume subset passes and
lists the skipped crates; a marked but reordered subset fails; the decoy
block scalar fails loudly; comments, blank lines, quoted names, and trailing
comments all still parse; flow style fails closed. All four YAML files
parse.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab22cd6f-f3d9-4325-8397-9e998aa058c3
0f05fd7 to
55e8629
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 29 out of 30 changed files in this pull request and generated no new comments.
Suppressed comments (4)
.azure-pipelines/scripts/crates_release.py:601
check-templateaccepts aRESUME-SUBSETmarker in every invocation, including the GitHub versioning check. Therefore a resume branch merged intomainwith a shortened list and marker passes CI, contrary todocs/crates-release.md:304-308, and leaves future releases silently incomplete. Make resume acceptance explicit (for example, an--allow-resume-subsetflag used only by the dedicated release job); the default CI check must require the full order.
if resume_reason is not None:
src/Cargo.toml:110
- These literal first-party requirements add new version-bump targets, but the established process says to update only the workspace/npm/C# versions and the sync check validates only those (
.github/copilot-instructions.md:235,scripts/check-version-sync.js:15-57). For the documented 0.8.0 release, these^0.7.0requirements no longer match the 0.8.0 sibling crates, socargo packagefails. Extend the version-sync check and update the release/contributor guidance to keep all first-party requirements aligned.
wxc_common = { path = "core/wxc_common", version = "0.7.0" }
mxc_engine = { path = "core/mxc_engine", version = "0.7.0" }
.azure-pipelines/templates/Publish.CratesIo.Job.yml:146
- The real-publish branch is currently enabled even though this PR documents two guaranteed mid-release blockers: the unraised first-publish rate limit and the
hyperlight_commonname collision. SettingdryRun=falsecan permanently publish the early crates and then fail, leaving an avoidable partial release. Hard-disable or gate this branch until those blockers are resolved, rather than relying only on operator instructions.
- ${{ else }}:
- task: EsrpRelease@12
docs/crates-release.md:384
- This prerequisite is stale relative to the PR description:
MXC-Release-Cratesis already registered as definition 198954 and the signing variable group is authorized. The unresolved prerequisite is authorization on theAzure-Pipelines-1ESPT-ExDSharedagent pool, so the operator runbook currently directs readers to the wrong setup action.
5. **Pipeline registration** — `.azure-pipelines/1ES.Release.Crates.yml` is new
and must be registered as a pipeline in Azure DevOps, and that pipeline must
be authorized to use the `MXC-ESRP-Signing` variable group.
…f it
A dry run previously ran the publish stage to completion with the ESRP task
compiled out, so the stage reported green. That is what led to a real run
being cancelled by hand: a green stage named "Publish Crates.io Packages"
gives an operator no way to tell a rehearsal from a publish.
Gate the stage instead, so the run reports it as Skipped while it stays
visible in the UI:
condition: and(succeeded(), ${{ eq(parameters.dryRun, false) }})
The ${{ }} is evaluated at template expansion, so this compiles to a boolean
literal rather than a runtime string comparison. Verified against ADO's own
compiler through the pipeline preview API: dryRun=true compiles to
`and(succeeded(), False)` with 0 EsrpRelease@12 tasks, dryRun=false to
`and(succeeded(), True)` with all 20 emitted. succeeded() is required
because naming any custom condition replaces the implicit default; without
it the stage would publish even after packaging had failed.
No coverage is lost. verify-order and the per-crate stage rehearsal ran only
inside the publish job, and both are redundant with the check-template step
that already runs in the packaging job: all three derive from the same
_release_order(cargo metadata) at the same ref, and crateOrder is a fixed
template default that cannot diverge between them. On a real publish the
publish-job checks still run, unchanged.
The compile-time ${{ if parameters.dryRun }} guard inside the publish job is
kept as a backstop, so that deleting the stage condition still cannot cause a
publish.
Also fixes three statements this makes false: the dry-run description in the
pipeline header, the dryRun parameter comment in the publish job, and step 6
of the operator doc, which told operators to look for per-crate DRY RUN log
lines that a dry run no longer produces.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab22cd6f-f3d9-4325-8397-9e998aa058c3
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 29 out of 30 changed files in this pull request and generated no new comments.
Suppressed comments (3)
src/Cargo.toml:110
- These registry requirements create additional version sources that the documented release bump does not update. After changing
[workspace.package]to the documented next version (0.8.0), these^0.7.0requirements (and the two directnanvix_commonrequirements) reject the newly packaged0.8.0siblings, socargo packagecannot produce the release. Please extend the version-bump tooling/check to update and validate every first-party requirement, and update the release instructions and Copilot versioning convention accordingly.
wxc_common = { path = "core/wxc_common", version = "0.7.0" }
mxc_engine = { path = "core/mxc_engine", version = "0.7.0" }
.azure-pipelines/scripts/crates_release.py:601
- This success path also runs in the GitHub
Versioning Checksworkflow, so a PR that carries a short list plusRESUME-SUBSETreturns 0 and can merge tomain. That contradicts the documented backstop that main must reject resume subsets. Require an explicit--allow-resume-subset(or equivalent) only from the release-ref packaging call, while keeping the CI invocation strict.
if resume_reason is not None:
docs/crates-release.md:388
- The known-blocker list omits the currently unresolved agent-pool authorization from the PR description. Until definition 198954 is authorized for
Azure-Pipelines-1ESPT-ExDShared(queue 250521), even a dry run cannot acquire an agent; include that authorization alongside the variable group/service connection prerequisites.
5. **Pipeline registration** — `.azure-pipelines/1ES.Release.Crates.yml` is new
and must be registered as a pipeline in Azure DevOps, and that pipeline must
be authorized to use the `MXC-ESRP-Signing` variable group.
An independent review of 33276e0 found two defects in it. First, its commit message claimed "no coverage is lost" because verify-order, the per-crate stage step, and check-template "all three derive from the same _release_order(cargo metadata)". That is false. Only package and check-template call _release_order (crates_release.py:286, :598). verify-order loads the packaged release-order.json instead (crates_release.py:355), and stage does work nothing else does at all: it checks each archive exists, verifies its recorded SHA-256, and fails closed when the digest is missing (crates_release.py:430-464). The narrow ordering claim does hold -- verify-order compares crateOrder against a manifest that package wrote from _release_order at the same ref, so check-template catches the same drift earlier. But archive integrity, the pipeline artifact round-trip, and ESRP staging are genuinely no longer exercised by a dry run. That does not change the decision to gate the stage, since none of those checks protect a run that publishes nothing. It does change what a green dry run entitles anyone to conclude, so say so where operators read it rather than leaving the overclaim standing. Second, the operator doc told readers a real release was the "same steps as above" while step 6 had just been rewritten to say the publish stage should report Skipped -- exactly backwards for a real release, and the one signal that distinguishes "I published" from "I ran a dry run that succeeded". Give the real-release path its own check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ab22cd6f-f3d9-4325-8397-9e998aa058c3
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 29 out of 30 changed files in this pull request and generated no new comments.
Suppressed comments (1)
docs/crates-release.md:109
- The release procedure is incomplete for the next minor version. The new first-party path dependencies carry literal
version = "0.7.0"requirements (for examplesrc/Cargo.toml:109-133), so bumping only[workspace.package].versionto0.8.0makes those requirements reject the local 0.8.0 crates andcargo metadata/cargo packagefails. Document updating all first-party dependency requirements as part of this step (and preferably provide a sync command), otherwise the documented release cannot be performed.
1. Bump the version in `src/Cargo.toml`.
| The npm SDK release (`.azure-pipelines/1ES.Release.yml`) consumes artifacts from | ||
| a **separate** official build pipeline (`MXC-Official-Build`) and publishes a | ||
| single `@microsoft/mxc-sdk` package. The crates release is self-contained: it | ||
| checks out the release ref, packages, and publishes in one run. This design | ||
| exists because 1ES forbids `checkout` in a release *job*, but a normal job in | ||
| the same *pipeline* may check out and build. |
There was a problem hiding this comment.
question: I'm not sure I understand why we need the release ref thing? For every release I've done so far I've just created a tag in github https://github.com/microsoft/mxc/tags, then used that here
So I'm a bit confused why we'd need this release ref.
In terms of using the MXC-Official-Build I assumed you can use that as well just like we do with the npm release. I just kick off a new official build using the same refs/tags/ first, then I run the official release with the same refs/tags/.
Packaging ran on a Linux pool while four crates in the closure are Windows-only, so `cargo package` failed in its verification build. The pipeline now runs entirely on Windows and verifies every crate against all three triples we ship, which turns a publish-time surprise into a package-time failure. Two independent causes made the Windows-only crates fail off Windows. First, `isolation_session_bindings` and `wslc_common` declared Windows dependencies in an ungated `[dependencies]` table. The `windows-future` 0.3.2 crate is the one that actually breaks: it references `windows_threading::submit` and `windows_core::imp::IMarshal` unconditionally while both sit behind a crate-wide `cfg(windows)`, which is rust-lang/windows-rs#4738. That fix is unreleased, so the dependency is gated here instead. Second, `windows_sandbox_common` reached base64 helpers through `wxc_common::string_util`, a Windows-gated module, though the helpers themselves are portable and also live in the ungated `wxc_common::encoding`. Verifying macOS surfaced two failures that no pipeline covered before: `nanvix_common` gated its constants to Linux specifically rather than to non-Windows, and `nanvix_runner` calls a module that only exists on Windows and Linux. `cargo package` now receives all three triples in a single invocation, and the publish stage still depends on that job succeeding, so a crate that fails to compile for any of the three cannot reach crates.io. The tarball is unaffected by which targets are verified -- it is created before the verification build is entered, and independent Windows and Linux runs produce a byte-identical archive -- so the crates published from the Windows agent are the same artifacts regardless. Packaging steps move from `bash` to `pwsh` because the agent is now Windows. This repository had no `bash` steps at all, and `python3` is not reliably present on a Windows agent, so the interpreter is resolved explicitly and the job fails loudly when it is missing. Target installation stays in the pipeline; the script only verifies the targets are present, asking rustc rather than rustup so it keeps working under the internal toolchain. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ab22cd6f-f3d9-4325-8397-9e998aa058c3
The bash `case` gate this replaced matched case-sensitively. PowerShell's -like does not: it accepted refs/heads/Release/v0.8.0 and refs/tags/V1, which are different refs from the lowercase ones the release/* ruleset freezes. A publish is irreversible, so the gate must not be wider than the rule that protects the refs it admits. -clike restores the original semantics; verified against 10 refs, including the three case variants that -like wrongly allowed. Also correct two comments that overstated what the code does. The packaging call is one cargo invocation for the whole closure, not one per crate. And nanvix_runner's crate-level cfg now cites the pre-existing gate on wxc_common::microvm_staging that it mirrors, so the empty-library-on-macOS result reads as a propagated platform constraint rather than a convenience. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ab22cd6f-f3d9-4325-8397-9e998aa058c3
What this does
Adds a dedicated pipeline that packages the
mxc-sdkcrate closure — 20 crates — and publishes them to crates.io through ESRP Release, one crate at a time in leaf-first order. Publishing is manual and gated on a release ref, so nothing reaches crates.io until a release is deliberately cut.Running a release
The pipeline is MXC-Release-Crates, definition 198954: https://microsoft.visualstudio.com/Dart/_build?definitionId=198954
Push a release ref to
microsoft/mxcand pick it in the Run dialog's branch/tag selector.Validate_Release_Refaccepts exactly two shapes and fails the run on anything else:refs/heads/release/*— for examplerelease/v0.8.0refs/tags/v*— for examplev0.8.0The gate carries
isSkippable: false, so it cannot be switched off in the "Stages to run" panel, and bothPackage_Crates_For_ReleaseandPublish_to_CratesIodepend on it. The definition carries no triggers, so a pipeline that publishes to a public registry cannot fire on a push.Two things worth knowing before the first run. The ref must exist on
microsoft/mxc— the Run dialog enumerates that repository only, so a branch pushed to a fork will not appear. And a release branch is not immutable on its own: the gate proves where a run came from, not that the ref cannot move. Freeze it with a GitHub ruleset (Settings → Rules → Rulesets) blocking pushes, force-pushes, and deletion, or use a tag.dryRundefaults to true: it packages the closure and checks the publish order, then skips the publish stage entirely — the run reports that stage asSkippedrather than green, so a rehearsal is distinguishable from a publish at a glance. Set it to false to publish for real. It is the only field in the Run dialog. The ESRP owner and approver are not fields: ADO renders every top-levelparameters:entry as an editable box, and an email is not a decision the person clicking Run should make — there is no right answer for them to know, and a typo submits the release to the wrong owner. Both are fixed in the YAML to$(Build.RequestedForEmail), which is what a default run already used, so this removed the prompt and not the behavior. A runtime guard in the gate stage fails the run if that variable resolves empty, which it can for a run queued by a service identity rather than a person.The pipeline needs no crates.io network access: the 1ES pool's CFSClean isolation blocks crates.io, dependency resolution stays on the private
Mxc-Azure-Feed, and ESRP owns the publishing credentials. Operator steps are documented indocs/crates-release.md.How it works
cargo packagerun with one-pflag per crate against--registry Mxc-Azure-Feed. That override works around A bug aboutcargo publish --workspace --registry crates-iorust-lang/cargo#17196, under which just-packaged workspace siblings are otherwise looked up in the wrong source and not found. Cargo's verification build stays on, so a mismatched sibling version requirement fails packaging.cargo packagecall carries three--targetflags —x86_64-pc-windows-msvc,x86_64-unknown-linux-gnu, andaarch64-apple-darwin— which cargo honors together and builds in parallel. A crate that no longer compiles on Linux or macOS therefore fails at package time, before anything is published, instead of after. The publish stage depends on that job, so any red triple stops the release. Verifying on Windows costs nothing in fidelity: the.cratetarball is written before the verification build starts, and independent Windows and Linux runs produce a byte-identical archive, so the artifacts published from the Windows agent are the same bytes either way. The three triples are installed by the job;crates_release.pyindependently confirms each one is present and fails the run if it is not, so a missing target can never quietly reduce what was checked.CRATESincrates_release.pyis an unordered set of package names; the order is a leaf-first topological sort of the workspace dependency graph derived fromcargo metadata, with alphabetical tie-breaking. Packaging derives it live; the template'scrateOrderis a pasted copy of the same computation, because${{ each }}expands before any script runs. A pasted copy can drift, and drifting failed silently:verify-orderaccepts an ordered subset on purpose, since that is how a part-published release resumes, so a template missing a newly-added crate passed and the crate was simply never published.check-templatenow recomputes the order on every PR and fails the build unless the template matches exactly. Adding a crate is: add the name, runcrates_release.py order, paste the output — and CI fails if you forget the last step.release-order.json. Each staged.crateis re-checked against that digest before ESRP submission, and staging fails closed if the digest is missing or disagrees.verify-orderaccepts an ordered subset of the packaged closure, which is what makes resuming a partly-published release possible.packageCratesparameter (default false) that runs the same packaging as a merge-time canary; it produces the artifact but never publishes.src/Cargo.tomlsetsrepositoryunder[workspace.package]and pinsversion = "0.7.0"on the first-party path dependencies so they are publishable.Validation
1ES.Release.Crates.ymlagainst definition 198954 — HTTP 200, expanding to the three stages and 20EsrpRelease@12tasks.release/*branches andv*tags, rejectsmain,user/*, bare commits, and the case variantsRelease/xandV1. The gate uses PowerShell-clikerather than-like, because-likeis case-insensitive and would admit refs therelease/*ruleset does not freeze.verify-orderexercised against the generated order — accepts the full 20-crate list and an 8-crate resume subset, rejects an out-of-order pair and an unknown crate name.cargo packageacross all 20 crates in a single invocation against all three target triples, on a clean tree — exit 0, 20.cratefiles produced, and per-triple.rlibartifacts present for 20 of 20 crates underx86_64-pc-windows-msvc,x86_64-unknown-linux-gnu, andaarch64-apple-darwin. Sixty of sixty cells built. Twelve of those cells are deliberately near-empty libraries, per the Windows-only note above; the check is that the closure compiles on every platform, not that every crate carries code on every platform.Still open
The pipeline is not yet authorized on the— resolved. Runs 154036650 and 154039902 both allocated agents on queue 250521 and executed. TheAzure-Pipelines-1ESPT-ExDSharedagent poolMXC-ESRP-Signingvariable group and theMXC-ESRP-Authservice connection are both authorized.hyperlight_commoncannot publish under that name. It normalizes tohyperlight-common, already taken on crates.io byhyperlight-dev(v0.16.0), and crates.io treats-and_as equivalent. A broadermxc_prefix policy for the whole closure is under discussion, and would supersede a single rename.Rustcontent-type enablement for the MXC client ID, OSPO open-source release registration, and a crates.ioPublishNewrate-limit override all remain to be requested. ESRP does supportRustand crates.io publishing; what is missing is per-client-ID enablement.— resolved, and the original diagnosis was incomplete. Run 154039902 packaged all 20 crates and then failed verifyingcargo package --verifyfails on the Linux agent.isolation_session_bindings, becausePackage.Crates.Job.ymlpinnedubuntu-latestwhile four crates in the closure are Windows-only. There were two independent causes, not one. Ungatedwindows-rsdependencies inisolation_session_bindingsandwslc_commonwere the first; the crate that actually breaks iswindows-future0.3.2, which referenceswindows_threading::submitandwindows_core::imp::IMarshalunconditionally while both sit behind a crate-widecfg(windows)— rust-lang/windows-rs#4738, whose fix is unreleased, so gating our own manifests is the only remedy. The second waswindows_sandbox_commonreaching base64 helpers throughwxc_common::string_util, a Windows-gated module, when the helpers are portable and already live in the ungatedwxc_common::encoding. Verifying macOS then surfaced two more that no pipeline had ever covered:nanvix_commongated its constants to Linux specifically rather than to non-Windows, andnanvix_runnercalls a module that exists only on Windows and Linux. Moving to Windows was necessary but not sufficient;--no-verifywas refused throughout, since it would delete the only compile check standing before an irreversible publish.isolation_session_bindings,isolation_session_common, andappcontainer_commonexpose nothing off Windows, andnanvix_runnerexposes nothing on macOS: itscfgdeliberately mirrors the pre-existing gate onwxc_common::microvm_staging, which its core logic depends on and which does not exist there, so it propagates an existing platform constraint rather than inventing one. That is the correct shape for crates.io, where a package must resolve on every platform even when it does nothing there, and it is what the verification proves: the closure builds everywhere, not that every crate is useful everywhere.mxc_telemetryis partially gated the same way, which predates this change.dryRun: truerun stops after packaging — it deliberately does not execute the publish job, so artifact download,verify-order, staging, and ESRP submission are all still unexercised.