Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
47 changes: 47 additions & 0 deletions .azure-pipelines/Fuzz.Build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
#
# Daily fuzzing pipeline for MXC.
#
# Builds the mxc_fuzz crate with AddressSanitizer and submits the resulting
# drop directory to OneFuzz. Bugs found are filed against the SDL fuzzing
# work item 62294501 via the routing fields in src/fuzz/OneFuzzConfig.json.
#
# Schedule: daily at 00:00 UTC on `main`. PRs do not trigger this pipeline.

pr: none
trigger: none

schedules:
- cron: "0 0 * * *"
displayName: Daily fuzzing submission
Comment thread
richiemsft marked this conversation as resolved.
branches:
include:
- main
always: true

name: $(SourceBranchName)_$(Date:yyyyMMdd)$(Rev:.r)

resources:
repositories:
- repository: 1ESPipelineTemplates
type: git
name: 1ESPipelineTemplates/1ESPipelineTemplates
ref: refs/tags/release

extends:
template: v1/1ES.Unofficial.PipelineTemplate.yml@1ESPipelineTemplates
parameters:
pool:
name: Azure-Pipelines-1ESPT-ExDShared
image: windows-latest
os: windows

customBuildTags:
- ES365AIMigrationTooling

stages:
- stage: Fuzz
displayName: 'Build + submit fuzzers'
jobs:
- template: .azure-pipelines/templates/Fuzz.Build.Job.yml@self
112 changes: 112 additions & 0 deletions .azure-pipelines/templates/Fuzz.Build.Job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
#
# Build the mxc_fuzz crate (libFuzzer + AddressSanitizer) and submit the
# resulting drop directory to OneFuzz via `onefuzz-task@0`.
#
# Nightly Rust toolchain: cargo-fuzz requires `-Z sanitizer=address`, which
# only nightly accepts. The Mxc-Azure-Feed (`ms-prod-*`) toolchains are
# stable-only today, so we install nightly via `rustup-init` here.
# TODO(SFI): switch to a Microsoft-published nightly via msrustup once one
# is available in Mxc-Azure-Feed; until then keep cargo registry pointed at
# Mxc-Azure-Feed so all *crate* sources still come from the internal feed.
Comment on lines +7 to +12

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Is this something I need to add? I can check and get back to you

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unsure, haven't been able to test the azure side since in order to generate the pipeline the code has to be check in. I couldn't reference a non checked in file.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all good, lets check it in so you can try it out. There is a way to do it without having to check it in though if I recall but it's ok lets get it in.


jobs:
- job: build_fuzz_windows_x64
displayName: 'Build Fuzzers (Windows x64, ASAN)'

pool:
name: Azure-Pipelines-1ESPT-ExDShared
image: windows-latest
os: windows

variables:
triplet: x86_64-pc-windows-msvc
fuzzDir: $(Build.SourcesDirectory)/src/fuzz
fuzzTargetDir: $(Build.SourcesDirectory)/src/fuzz/target/$(triplet)/release
dropDir: $(Build.ArtifactStagingDirectory)/onefuzz-drop
# TODO: pin a specific nightly date (e.g. nightly-2026-05-01) for build
# reproducibility once we've validated cargo-fuzz against it.
rustNightly: nightly

steps:
- checkout: self

# Keep all crate fetches going through the internal feed.
- powershell: |
Add-Content -Path "$(Build.SourcesDirectory)/.cargo/config.toml" -Value ("`n" + (Get-Content -Raw "$(Build.SourcesDirectory)/.azure-pipelines/.cargo/config.toml"))
displayName: Setup Cargo Config

# Install nightly Rust via rustup-init. The toolchain itself runs on the
# hosted agent only -- crates come from Mxc-Azure-Feed (see above).
- powershell: |
$rustupInit = "$env:TEMP\rustup-init.exe"
Invoke-WebRequest -UseBasicParsing -Uri 'https://win.rustup.rs/x86_64' -OutFile $rustupInit
& $rustupInit -y --profile minimal --default-toolchain $(rustNightly) --default-host x86_64-pc-windows-msvc
if ($LASTEXITCODE -ne 0) { throw "rustup-init failed" }
Write-Host "##vso[task.prependpath]$env:USERPROFILE\.cargo\bin"
displayName: Install Rust nightly

- powershell: |
rustc --version
cargo --version
displayName: Verify toolchain

- powershell: cargo install cargo-fuzz --locked
displayName: Install cargo-fuzz

# Build all three fuzz targets with ASAN.
- powershell: |
cd $(fuzzDir)
foreach ($t in 'config_parser','base64_decode') {
Write-Host "===== building $t ====="
cargo fuzz build $t --release
if ($LASTEXITCODE -ne 0) { throw "fuzz build failed for $t" }
}
Write-Host "===== building validator ====="
cargo fuzz build validator --release --features hyperlight,isolation_session
if ($LASTEXITCODE -ne 0) { throw "fuzz build failed for validator" }
displayName: Build fuzz targets (ASAN)

# Stage the OneFuzz drop directory: one subdir per fuzzer with the .exe,
# the OneFuzzConfig.json subset for that fuzzer, the ASAN runtime DLL,
# and the seed corpus.
- powershell: |
$asanDir = (Get-ChildItem 'C:\Program Files\Microsoft Visual Studio' -Recurse -Filter 'clang_rt.asan_dynamic-x86_64.dll' -ErrorAction SilentlyContinue | Where-Object { $_.FullName -match 'HostX64\\x64\\clang_rt' } | Select-Object -First 1).Directory.FullName
if (-not $asanDir) { throw "Could not locate clang_rt.asan_dynamic-x86_64.dll" }
Write-Host "ASAN runtime: $asanDir"

New-Item -ItemType Directory -Force -Path '$(dropDir)' | Out-Null
Copy-Item '$(fuzzDir)/OneFuzzConfig.json' '$(dropDir)/OneFuzzConfig.json'

foreach ($t in 'config_parser','base64_decode','validator') {
$sub = "$(dropDir)/$t"
New-Item -ItemType Directory -Force -Path $sub, "$sub/corpus" | Out-Null
Copy-Item "$(fuzzTargetDir)/$t.exe" $sub
Copy-Item "$asanDir/clang_rt.asan_dynamic-x86_64.dll" $sub
if ($t -eq 'base64_decode') {
Copy-Item "$(fuzzDir)/corpus/$t/*" "$sub/corpus/" -Recurse
} else {
Copy-Item "$(Build.SourcesDirectory)/test_configs/*.json" "$sub/corpus/"
}
}
Get-ChildItem -Recurse '$(dropDir)' | Select-Object FullName, Length
displayName: Stage OneFuzz drop directory

# Publish the drop dir as a pipeline artifact for visibility / debugging.
- task: 1ES.PublishPipelineArtifact@1
displayName: Publish OneFuzz drop
inputs:
path: '$(dropDir)'
artifactName: onefuzz-drop

# Submit to OneFuzz. Skipped on PR builds; only the scheduled daily run
# submits live jobs.
- task: onefuzz-task@0
displayName: Submit to OneFuzz
condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
inputs:
onefuzzOSes: Windows
env:
onefuzzDropDirectory: $(dropDir)
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
101 changes: 101 additions & 0 deletions docs/fuzzing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Fuzzing

MXC uses [cargo-fuzz](https://rust-fuzz.github.io/book/cargo-fuzz.html) for
local fuzzing harnesses and [OneFuzz](https://aka.ms/onefuzz) for continuous
fuzzing in CI.

## What we fuzz

The `mxc_fuzz` crate at `src/fuzz/` defines three libFuzzer targets, all
exercising the attacker-influenced config surface consumed by `wxc-exec` and
`lxc-exec`:

| Target | Entry point |
| ---------------- | -------------------------------------------------------- |
| `config_parser` | `load_mxc_request(s, .., is_base64 = false)` |
| `base64_decode` | `load_mxc_request(s, .., is_base64 = true)` (SDK wire format) |
| `validator` | parse + `validate_common` on a one-shot request |

Seed corpora for `config_parser` and `validator` targets come directly from
`test_configs/*.json`. The `base64_decode` target uses pre-encoded seeds in
`src/fuzz/corpus/base64_decode/`. OneFuzz dedups by coverage server-side and
grows the corpus across daily runs, so we keep the in-repo seeds small.

## Platform coverage

Targets are pure-Rust code in `wxc_common`, so they compile and run
identically on Windows, Linux, and macOS. We fuzz on **Windows only**
because:

- OneFuzz supports Windows, Ubuntu, AzureLinux3, and TKO β€” not macOS.
- For these parser targets the bugs are platform-independent; one OS gives
full coverage of the relevant code paths.

## Running locally (Windows)

```pwsh
# One-time setup
rustup toolchain install nightly --profile minimal
cargo +nightly install cargo-fuzz

# Put the MSVC ASAN runtime DLL on PATH for this shell
$asanDir = (Get-ChildItem 'C:\Program Files\Microsoft Visual Studio' -Recurse `
-Filter 'clang_rt.asan_dynamic-x86_64.dll' -ErrorAction SilentlyContinue `
| Where-Object FullName -Match 'HostX64\\x64\\clang_rt' | Select-Object -First 1).Directory.FullName
$env:PATH = "$asanDir;$env:PATH"

# Run a target for 30 seconds (uses test_configs/ as the seed corpus)
cd src\fuzz
cargo +nightly fuzz run config_parser ..\..\test_configs -- -max_total_time=30
```

Discovered crashes are written to `artifacts/<target>/` (relative to `src/fuzz/`)
and printed to the console. Re-run a single input with:

```pwsh
cargo +nightly fuzz run config_parser artifacts\config_parser\crash-<hash>
```

## Minimizing the seed corpus

libFuzzer auto-saves any new-coverage input into the corpus dir during a
run, which can bloat the commit. Before committing seed-corpus updates:

```pwsh
cargo +nightly fuzz cmin <target>
```

`cmin` keeps the smallest set that retains all coverage.

## Continuous fuzzing pipeline

`.azure-pipelines/Fuzz.Build.yml` runs daily at 00:00 UTC on `main`. The job
template (`.azure-pipelines/templates/Fuzz.Build.Job.yml`):

1. Installs nightly Rust on the agent (cargo registry stays pointed at
`Mxc-Azure-Feed`, so all crate sources still come from the internal feed).
2. Installs `cargo-fuzz`.
3. Builds the three fuzz targets with `-Z sanitizer=address`.
4. Stages an OneFuzz drop directory: one subdir per fuzzer with the `.exe`,
the ASAN runtime DLL, and the seed corpus.
5. Publishes the drop dir as a pipeline artifact (for debugging).
6. Submits via `onefuzz-task@0` (skipped on PR builds).

## Bug triage

When OneFuzz files a bug via the routing configured in `OneFuzzConfig.json`,
triage steps:

1. **Reproduce locally.** Download the offending input from the fuzz job
page and run `cargo +nightly fuzz run <target> <crash-file>` (see
"Running locally"). If it reproduces against `main`, the bug is real.
2. **Classify.** AddressSanitizer findings (heap overflow, use-after-free,
etc.) are security-relevant and should be handled through the project's
security response process. Plain panics in parsers are correctness bugs
and can be fixed in-band.
3. **Add a regression test.** Drop the minimized crash input into the
appropriate corpus subdir so `cmin` keeps it. If the bug fits the unit
test pattern, add a dedicated `#[test]` in `wxc_common` too.
4. **Fix + verify.** After the fix lands, re-run the fuzz target locally
against the original crash to confirm. Once the daily pipeline runs
again with the fix, the fuzz job should mark the bug as resolved.
1 change: 1 addition & 0 deletions src/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ members = [
"generated/base_container_specification",
"mxc_diagnostic_console",
]
exclude = ["fuzz"]
resolver = "3"

# Full debug info so we can analyse customer crash dumps in WinDbg until we
Expand Down
4 changes: 4 additions & 0 deletions src/fuzz/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
target/
Cargo.lock
artifacts/
coverage/
44 changes: 44 additions & 0 deletions src/fuzz/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
[package]
name = "mxc_fuzz"
version = "0.0.0"
publish = false
edition = "2021"

[package.metadata]
cargo-fuzz = true

# Excluded from the parent workspace (see ../Cargo.toml `exclude`) so that
# `cargo-fuzz`-managed Cargo.lock / target directories don't interfere with
# normal builds and so the nightly-only `libfuzzer-sys` dependency is only
# pulled in when explicitly fuzzing.

[features]
# Enable runner-specific validation in the `validator` fuzz target.
# These pull in heavier deps so they're opt-in for local dev.
hyperlight = ["wxc_common/hyperlight"]
isolation_session = ["wxc_common/isolation_session"]

[dependencies]
libfuzzer-sys = "0.4"
wxc_common = { path = "../wxc_common" }

[[bin]]
name = "config_parser"
path = "fuzz_targets/config_parser.rs"
test = false
doc = false
bench = false

[[bin]]
name = "base64_decode"
path = "fuzz_targets/base64_decode.rs"
test = false
doc = false
bench = false

[[bin]]
name = "validator"
path = "fuzz_targets/validator.rs"
test = false
doc = false
bench = false
Loading
Loading