Skip to content

README.md

Latest commit

 

History

History
118 lines (93 loc) · 5.39 KB

README.md

File metadata and controls

118 lines (93 loc) · 5.39 KB
Raw

build/ — Pipeline definitions

This directory contains Azure DevOps pipeline YAML files and the pod-scheduler configs that generate them.

Generated files (do not edit by hand)

The CI pipelines that run the daily benchmark sweep are generated by the pod-scheduler. Each pipeline maps to a JSON config that lists the available pods (machine groups) and scenarios.

Generated YAML Source config Notes
benchmarks-ci-01.yml, benchmarks-ci-02.yml benchmarks_ci_pods.json On-prem fleet (Gold); split across two YAMLs to spread load
benchmarks-ci-azure.yml benchmarks_ci_azure_pods.json Azure cloud pods (includes the merged eastus2 cobalt cloud pods)
benchmarks-ci-cobalt.yml benchmarks_ci_cobalt_pods.json Cobalt-hosted pods

To regenerate, from the repo root:

python ./scripts/pod-scheduler/main.py \
    --config ./build/benchmarks_ci_pods.json \
    --yaml-output ./build

python ./scripts/pod-scheduler/main.py \
    --config ./build/benchmarks_ci_azure_pods.json \
    --base-name benchmarks-ci-azure \
    --yaml-output ./build

python ./scripts/pod-scheduler/main.py \
    --config ./build/benchmarks_ci_cobalt_pods.json \
    --base-name benchmarks-ci-cobalt \
    --yaml-output ./build

The header of each generated YAML embeds the exact regen command for that file.

The citrine1, citrine2, and citrine3 names in the on-prem config are controller queues, not retired benchmark machines or relay targets.

Hand-maintained scenario templates

Everything else in this directory is hand-maintained. The most important groups:

  • *-scenarios.yml — AzDO job templates that the generated pipelines invoke (e.g. proxies-scenarios.yml, grpc-scenarios.yml). One per benchmark family. The pod-scheduler picks which template to run via the template field on each scenario in the JSON config.
  • prbenchmarks.yml — scheduled PR-benchmark bot pipeline.
  • prbenchmarks.*.config.yml — components, scenarios, and machine choices for individual PR-benchmark requests.
  • job-variables.yml — shared variable definitions consumed by the scenario templates.

When adding a new benchmark family, drop a *-scenarios.yml template in this directory and reference it from the relevant benchmarks_ci_*_pods.json.

PR benchmarks

prbenchmarks.yml polls benchmark requests on dotnet/aspnetcore, dotnet/efcore, and dotnet/runtime every 30 minutes. The controller image determines where PR components are built, not which machine runs the benchmark. The first profile in each bot config is the default machine selection.

Config Controller Supported benchmark machines (default first)
prbenchmarks.aspnetcore.config.yml Windows aspnet-gold-lin, aspnet-gold-win (x64, 56 cores)
prbenchmarks.efcore.config.yml Windows aspnet-gold-win (x64, 56 cores)
prbenchmarks.runtime.windows_x64.config.yml Windows aspnet-gold-win (x64, 56 cores)
prbenchmarks.runtime.linux_x64.config.yml Linux aspnet-gold-lin (x64, 56 cores)

These selections use the existing aspnet-gold-*-relay profiles imported by the benchmark scenarios. Gold Linux uses goldlin, goldload, and golddb; Gold Windows uses goldwin, goldload2, and golddb. The disconnected Perf and Citrine choices are no longer offered. Runtime ARM64 jobs/configs have been retired because their relay machines are unavailable; Gold is x64 and does not replace ARM64 coverage.

The ASP.NET Core, EF Core, and runtime scenario/component definitions are unchanged. This migration does not change the polling schedule, controller images, or Crank SDK/build configuration.

Editing the schedule

Common edits, all done in JSON:

  • Add a scenario: append to scenarios[] in the appropriate benchmarks_ci_*_pods.json.
  • Add a pod / machine class: append to pods[] and reference its name from any scenario you want to target it.
  • Change cadence or splits: edit metadata.schedule (cron), metadata.yaml_generation.target_yaml_count, or metadata.yaml_generation.schedule_offset_hours.
  • Override a job's timeout: add "timeout": <minutes> to the scenario.

After any edit, regenerate the YAML and commit both the JSON and the YAML together. The snapshot tests in scripts/pod-scheduler/tests/test_snapshots.py will fail in CI if the generated YAML drifts from the configs.

Minimal Todo

minimal-todo-scenarios.yml dispatches scenarios/minimal-todo/minimal-todo.benchmarks.yml (1 CPU, 536,870,912 bytes / 512 MiB, 25200 req/s, enabled) for the gold-lin and Azure azure-arm64/cobalt-cloud-lin pods. C# runs on every regular scheduled slot using the direct (non-Docker) csharp-project build; Go and Rust run the Docker go/rust builds but only on a shared weekly slot (condition: Math.round(Date.now() / 43200000) % 14 == 0, combined with the pod's own condition, matching the containers-scenarios.yml idiom). The Docker csharp/go/rust builds (floating, multi-arch base images, no pinned digests, no requiredArchitecture gating) remain the canonical generation/update/admission/pre-promotion comparison path and are unaffected -- see scenarios/minimal-todo/README.md for the full execution-path mapping. Generation, admission, and calibration tooling for this cohort live in the separate dotnet-performance-tools repository; this directory only carries the scheduled dispatch template.