feat: add OpenShell-compatible diagnostics

[RamonArjona4]

Description

Adds OpenShell-compatible local structured diagnostics and audit records for MXC. Records cover policy identity, sandbox identity, enforcement degradation, network policy, process outcomes, teardown, timeout/kill failures, and configuration rejection while remaining local-only and excluded from SDK stdout/captured output.

The change also hardens diagnostic IPC and redaction, including per-session pipe tokens, user-scoped pipe access, control-character sanitization, and removal of isolation-session credentials from diagnostic output.

Intentional behavior changes:

• IsolationSession forced termination waits up to five seconds.
• Diagnostic pipe access is user-scoped rather than unrestricted.
• File audit records use the documented timestamp-prefixed format.

Documentation was kept at the public product/technology level; newly added documentation does not expose targeted Windows-internal implementation names or private telemetry values.

References

No issue linked.

Validation

• cargo fmt --all -- --check
• cargo test --workspace
• cargo clippy --workspace --all-targets -- -D warnings
• cargo test -p wxc_e2e_tests
• cargo test -p wxc_e2e_tests -- --ignored (13 Windows E2E tests passed)
• Windows debug Rust build completed successfully.
• Security, adversarial, functionality, cross-platform, documentation, test-coverage, and regression reviews completed.

Known environment limitations:

• The build.bat --debug SDK packaging phase could not resolve ip-address@10.4.0 from the configured npm feed and lacked tsc; no SDK files were changed.
• One standalone basic-process script encountered 0xC0000135, while the native process-container E2E tests passed.
• src/Cargo.lock adds sha2; the dependency-feed check should be run by CI/ADO before merge.

Checklist

• Signed the Contributor License Agreement
• Linked to an issue
• Updated documentation (if applicable)
• Updated Copilot instructions (if build, architecture, or conventions changed)
• If this PR changes Cargo.lock, the dependency-feed-check check passes (see docs/pull-requests.md)

Issue Type

• Bug fix
• Feature
• Task

Microsoft Reviewers: Open in CodeFlow

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Not ready to approve

Several paths leak unsanitized identities or emit inaccurate process, firewall, and teardown outcomes.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

Adds local structured diagnostics and audit records across MXC execution paths, with hardened Windows diagnostic IPC and redaction.

Changes:

• Adds structured audit events for policy, lifecycle, network, process, and rejection outcomes.
• Introduces canonical policy hashing and identity sanitization.
• Hardens diagnostic pipes and documents the local audit format.

File summaries

File	Description
src/tools/mxc_diagnostic_console/src/main.rs	Secures pipe creation and sanitizes displayed records.
src/core/wxc/src/main.rs	Emits rejection, policy, and sandbox identity records.
src/core/wxc_common/src/policy_identity.rs	Implements canonical policy hashing and redaction.
src/core/wxc_common/src/models.rs	Adds bounded network value renderers.
src/core/wxc_common/src/logger.rs	Adds structured diagnostic sinks and pipe verification.
src/core/wxc_common/src/lib.rs	Exports audit and policy identity modules.
src/core/wxc_common/src/diagnostic.rs	Adds session tokens and credential redaction.
src/core/wxc_common/src/config_parser.rs	Classifies malformed one-shot requests.
src/core/wxc_common/src/audit.rs	Defines audit records and sanitization.
src/core/wxc_common/Cargo.toml	Adds SHA-256 dependency.
src/core/mxc_engine/src/state_aware.rs	Handles the new parse-error variant.
src/core/mxc_engine/src/run.rs	Emits policy and degradation records.
src/core/mxc_engine/src/lib.rs	Exports policy-hash logging.
src/core/mxc_engine/src/dispatch.rs	Adds policy records to streaming launches.
src/Cargo.toml	Declares workspace SHA-256 dependency.
src/Cargo.lock	Locks the added dependency.
src/backends/isolation_session/common/src/state_aware.rs	Connects diagnostics during state-aware execution.
src/backends/isolation_session/common/src/one_shot.rs	Passes diagnostics into process execution.
src/backends/isolation_session/common/src/manager.rs	Records timeouts and termination failures.
src/backends/appcontainer/common/src/network_manager.rs	Reports network setup and teardown outcomes.
src/backends/appcontainer/common/src/job_object.rs	Surfaces termination errors.
src/backends/appcontainer/common/src/fallback_detector.rs	Adds bounded degradation reasons.
src/backends/appcontainer/common/src/dispatcher.rs	Emits enforcement degradation records.
src/backends/appcontainer/common/src/base_container_runner.rs	Adds process and teardown auditing.
src/backends/appcontainer/common/src/appcontainer_runner.rs	Adds network, process, and teardown auditing.
docs/telemetry/telemetry.md	Documents local audit records.
docs/diagnostics.md	Documents tokens, sinks, and audit scope.

Review details

Suppressed comments (1)

src/backends/appcontainer/common/src/appcontainer_runner.rs:1591

• This bfs_removed value is only the cleanup predicate; the boolean returned by remove_configuration is ignored. If bfscfg --clearpolicy fails, the record still says bfs_removed=true, and status remains success because it considers only firewall removal. Use the actual return value and include a requested-but-failed BFS cleanup in the teardown status.

        let bfs_removed = self.filesystem_mode == FilesystemMode::Bfs
            && self.prepared.bfs_manager.configured()
            && !self.preserve_policy;
        if bfs_removed {
            self.prepared.bfs_manager.remove_configuration(&mut logger);

• Files reviewed: 26/27 changed files
• Comments generated: 10
• Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[Copilot]

🟡 Not ready to approve

State-aware hashing, rejection coverage, identity redaction, and diagnostic documentation contain unresolved correctness and security issues.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (9)

docs/telemetry/telemetry.md:448

• This paragraph describes the pre-change behavior. state_aware.rs now inherits the driver's diagnostic sink and passes the logger to create_process, so state-aware IsolationSession exec is recorded; leaving this text in place directly contradicts the implementation.

The isolation-session `ProcessTimedOut` and `ProcessKillFailed` records are
emitted by the one-shot runner, where `wxc-exec` supplies the local diagnostic
logger to the backend. The state-aware backend trait does not currently carry a
logger into its `exec` method, so state-aware isolation-session exec remains
unrecorded by these MXC-local process-boundary events.

docs/telemetry/telemetry.md:454

• This coverage row also says state-aware exec is unrecorded, but the new inherited logger path records it. Update the local-coverage statement so consumers do not incorrectly treat these records as unavailable.

| M-ETW-1 process outcome | Existing OS process-lifecycle records cover normal exit. The OS does not provide a verified timeout or kill-failure record for this requirement. | `mxc.ProcessTimedOut` and `mxc.ProcessKillFailed` cover the one-shot MXC boundary; state-aware exec is not currently logger-backed. | Join the OS lifecycle identity to the MXC sandbox identity where available; use the process ID for process records. |

src/core/wxc/src/main.rs:512

• State-aware policy hashes are computed before the phase config is deserialized or validated, and ParsedStateAwareRequest::request intentionally excludes every per-backend phase block (config_parser.rs:1409-1413). Consequently, provision requests with different filesystem/configuration settings can emit the same hash, and malformed requests can emit a PolicyHash even though no effective policy was resolved. Move this emission after typed phase validation and include a credential-redacted projection of the selected phase config.

    mxc_engine::log_policy_hash(&parsed.request, logger);

src/core/wxc/src/main.rs:521

• Errors returned here include phase-config deserialization and backend validate_* failures, but the later error path only calls log_state_aware_dispatch_error, so these rejected requests never produce the new machine-readable mxc.ConfigRejected record. Emit that record for validation-class error codes before finalizing the error envelope, while keeping runtime backend failures distinct.

    let mut outcome = mxc_engine::run_state_aware(parsed, dry_run);

docs/telemetry/telemetry.md:434

• The table is stale: this PR now installs the thread diagnostic sink for state-aware dispatch and passes the inherited logger into IsolationSession create_process, so state-aware exec does emit these process-boundary records. Mark the Windows state-aware column accordingly.

This issue also appears in the following locations of the same file:

• line 444
• line 454

| `mxc.ProcessExited` / `TimedOut` / `KillFailed` | ✅ (including isolation-session one-shot) | — | — | — |

src/core/wxc_common/src/audit.rs:45

• Remove the stray /// embedded in this sentence.

/// Closed set of audit record names. The `mxc.` prefix namespaces the record/// against unrelated lines sharing the same sink.

src/core/wxc/src/main.rs:1163

• ParseError::Decode also covers a missing/unreadable config file (decode_request_input_without_logging, lines 280-289), not just invalid base64/JSON. Those operational input failures are therefore recorded as malformed_json, which is inaccurate for machine consumers. Preserve a distinct decode/input-error category (for example input_unavailable) instead of mapping every Decode error to malformed JSON.

                RejectionReason::MalformedJson,

docs/diagnostics.md:20

• This quick start no longer tells users to elevate the console, but Logger::verify_server_integrity still rejects any pipe server below High integrity. Following these commands in ordinary terminals therefore leaves wxc-exec unable to connect. Either restore the administrator requirement here (and the matching console text) or intentionally relax the client check.

# Terminal 1: choose one token and start the diagnostic console
$env:MXC_DIAG_PIPE_TOKEN = [guid]::NewGuid().ToString("N")
$env:MXC_DIAG_PIPE_TOKEN
mxc-diagnostic-console.exe

docs/telemetry/telemetry.md:268

• These instructions also start the pipe server unelevated, while the client explicitly requires the server token to be High integrity. Add the elevation step here so this documented named-pipe setup can actually connect.

# Named-pipe sink (Windows): use the same high-entropy token in both processes.
$env:MXC_DIAG_PIPE_TOKEN = [guid]::NewGuid().ToString('N')
$env:MXC_DIAG_PIPE_TOKEN
# Terminal 1: start the console with the shared token.
mxc-diagnostic-console.exe

• Files reviewed: 26/27 changed files
• Comments generated: 2
• Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[Copilot]

🟡 Changes recommended

Raw credentials can still reach diagnostics, while state-aware policy hashes and partial firewall success records can be inaccurate.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (8)

docs/telemetry/telemetry.md:452

• This paragraph describes the pre-change behavior. The state-aware exec path now uses Logger::inherit_thread_diagnostic_sink() and passes that logger to create_process, so its process outcome records reach both --log-file and the diagnostic pipe. Update the scope statement accordingly.

The isolation-session `ProcessTimedOut` and `ProcessKillFailed` records are
emitted by the one-shot runner, where `wxc-exec` supplies the local diagnostic
logger to the backend. The state-aware backend trait does not currently carry a
logger into its `exec` method, so state-aware isolation-session exec remains
unrecorded by these MXC-local process-boundary events.

docs/telemetry/telemetry.md:458

• The coverage row still says state-aware execution is not logger-backed, but this PR explicitly adds inherited diagnostic sinks to that path. Record both one-shot and state-aware IsolationSession coverage here.

| M-ETW-1 process outcome | Existing OS process-lifecycle records cover normal exit. The OS does not provide a verified timeout or kill-failure record for this requirement. | `mxc.ProcessTimedOut` and `mxc.ProcessKillFailed` cover the one-shot MXC boundary; state-aware exec is not currently logger-backed. | Join the OS lifecycle identity to the MXC sandbox identity where available; use the process ID for process records. |

src/backends/appcontainer/common/src/appcontainer_runner.rs:1784

• ProcessExited is emitted only from blocking wait(). SandboxProcess::try_wait() is a supported completion path (also exposed through the Rust SDK and C FFI), and a caller may poll until Some(code) and then drop/free the handle; that path never reaches this block, so a normal process outcome is missing from the documented audit stream. Emit the terminal event from both try_wait and wait behind a shared once-only flag to avoid duplicates.

                    let exit_code = code as i32;
                    if self.audit_enabled() && !self.kill_requested {
                        let record = self
                            .audit(AuditEventName::ProcessExited)
                            .i64("exit_code", exit_code as i64);
                        self.audit_logger.log_audit_event(&record);
                    }

src/backends/appcontainer/common/src/base_container_runner.rs:2189

• ProcessExited is emitted only from blocking wait(). A valid streaming caller can observe completion through SandboxProcess::try_wait() (including the Rust SDK/C FFI) and then drop/free the handle, which skips this record entirely. Route both try_wait and wait through a once-only terminal-event helper so polling completion is covered without duplicate records.

                    let exit_code = code as i32;
                    if self.audit_enabled() && !self.kill_requested {
                        let record = self
                            .audit(AuditEventName::ProcessExited)
                            .i64("exit_code", exit_code as i64);
                        self.audit_logger.log_audit_event(&record);
                    }

src/core/wxc/src/main.rs:512

• This generic hash call receives an incomplete and incorrectly attributed request for non-provision state-aware phases. Those requests omit containment, so convert_wire_config defaults parsed.request.containment to ProcessContainer even though resolve_backend above resolves iso:/wsb: correctly; moreover, backend phase config remains only in experimental_raw (for example, IsolationSession start.configurationId). The resulting record can say backend=processcontainer, and two different effective start policies can produce the same hash. Build the state-aware policy identity from the resolved backend plus a credential-redacted typed phase config, or relay the provision-time effective-policy identity through the lifecycle.

    // State-aware dispatch bypasses the one-shot runner funnel, so anchor the
    // effective lifecycle policy here before the request is consumed.
    mxc_engine::log_policy_hash(&parsed.request, logger);

docs/telemetry/telemetry.md:438

• State-aware IsolationSession exec now inherits the driver's logger and passes it into create_process, so the state-aware column is no longer entirely unsupported for these process-boundary records. Mark IsolationSession state-aware execution as covered while retaining the gap for Windows Sandbox.

This issue also appears in the following locations of the same file:

• line 448
• line 458

| `mxc.ProcessExited` / `TimedOut` / `KillFailed` | ✅ (including isolation-session one-shot) | — | — | — |

src/core/wxc_common/src/audit.rs:45

• This rustdoc line accidentally concatenates a second doc-comment marker into the sentence, so generated documentation reads record/// against.

/// Closed set of audit record names. The `mxc.` prefix namespaces the record/// against unrelated lines sharing the same sink.

src/backends/appcontainer/common/src/network_manager.rs:286

• outcome.is_ok() is not the actual aggregate apply result: process_host_list still converts hostname-resolution failures and every create_rule Ok(false)/Err(_) into continue, then returns Ok(()) (lines 375-398). If the primary rule succeeds but a requested allow/block rule does not land, this sets firewall_apply_ok = Some(true), and NetworkPolicyApplied reports status=success/firewall_applied=true for incomplete enforcement (notably a failed block-host rule weakens an allow-default policy). Propagate these failures while retaining successfully created rules for cleanup.

    ) -> Result<(), WxcError> {
        let outcome = self.apply_firewall_rules_inner(principal_id, policy, logger);
        self.firewall_apply_ok = Some(outcome.is_ok());
        outcome

• Files reviewed: 26/27 changed files
• Comments generated: 1
• Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[Copilot]

🟡 Changes recommended

Identity sanitization can expose caller-controlled identifiers, and policy hashing does not consistently represent effective enforcement.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (7)

src/core/wxc_common/src/audit.rs:436

• sanitize_identity cannot prove minting provenance from the string shape. The AppContainer call sites pass caller-controlled containerId, so values such as iso:alice, wsb:ticket-1234, or a correctly shaped sandbox-<hex> pass this check and are written verbatim, contradicting the stated no-caller-identifiers guarantee. Split sanitization by provenance (or pass a typed provenance): ProcessContainer caller IDs must redact, while only validated state-aware or internally minted IDs may pass.

    let mxc_opaque = identity
        .split_once(':')
        .map(|(prefix, token)| {
            matches!(prefix, "iso" | "wsb")
                && !token.is_empty()

docs/telemetry/telemetry.md:452

• This paragraph describes the pre-fix implementation. State-aware IsolationSession exec now receives both --log-file and named-pipe sinks through Logger::inherit_thread_diagnostic_sink, so saying it remains unrecorded directly contradicts the code.

The isolation-session `ProcessTimedOut` and `ProcessKillFailed` records are
emitted by the one-shot runner, where `wxc-exec` supplies the local diagnostic
logger to the backend. The state-aware backend trait does not currently carry a
logger into its `exec` method, so state-aware isolation-session exec remains
unrecorded by these MXC-local process-boundary events.

docs/telemetry/telemetry.md:458

• This coverage row also still says state-aware exec is not logger-backed, but the new thread-local sink inheritance makes these process-boundary records observable for state-aware IsolationSession execution. Update the MXC-local coverage cell to match the implementation and platform table.

| M-ETW-1 process outcome | Existing OS process-lifecycle records cover normal exit. The OS does not provide a verified timeout or kill-failure record for this requirement. | `mxc.ProcessTimedOut` and `mxc.ProcessKillFailed` cover the one-shot MXC boundary; state-aware exec is not currently logger-backed. | Join the OS lifecycle identity to the MXC sandbox identity where available; use the process ID for process records. |

src/core/wxc_common/src/policy_identity.rs:225

• Removing user entirely makes a local IsolationSession policy and an Entra-backed policy hash identically, even though presence of the bundle selects different provisioning/start APIs and a different sandbox identity model. Strip the UPN/token values, but retain a bounded user_present/entra boolean; tests should verify different credentials hash alike while absent versus present does not.

    // `IsolationSessionConfig::user` carries a WAM bearer token and a UPN, so
    // the section is serialized and then the credential key is stripped rather
    // than being passed through.
    let mut iso = serde_json::to_value(isolation_session).unwrap_or(Value::Null);
    strip_keys(&mut iso, &["user"]);

src/core/wxc_common/src/policy_identity.rs:216

• This serializes WindowsSandboxConfig.idle_timeout_ms and daemon_pipe_name into the policy hash, but mxc_engine::run explicitly treats both as ignored legacy one-shot settings (run.rs:253-263). Changing either therefore changes the identity without changing enforcement, violating this module's effective-policy contract. Exclude these fields or project only Windows Sandbox settings that are actually enforced.

    out.insert(
        "windows_sandbox".into(),
        serde_json::to_value(windows_sandbox).unwrap_or(Value::Null),
    );

src/core/wxc_common/src/policy_identity.rs:320

• Filesystem path lists, capabilities, and host lists are set-like enforcement inputs; their consumers install/check every entry without positional semantics (for example, filesystem conflict handling in config_parser.rs:369-420). Preserving their input order makes reordered but otherwise identical effective policies produce different hashes. Canonicalize each set-like list in the projection (sort and, where semantics permit, deduplicate) while preserving order only for genuinely ordered arrays.

        Value::Array(items) => {
            out.push('[');
            for (i, item) in items.iter().enumerate() {
                if i > 0 {
                    out.push(',');
                }
                write_canonical(item, out);

docs/telemetry/telemetry.md:438

• The platform matrix is stale: state-aware IsolationSession exec now inherits the driver's diagnostic sinks and passes that logger into create_process, so it emits ProcessExited, ProcessTimedOut, and ProcessKillFailed. Mark the Windows state-aware column accordingly.

This issue also appears in the following locations of the same file:

• line 448
• line 458

| `mxc.ProcessExited` / `TimedOut` / `KillFailed` | ✅ (including isolation-session one-shot) | — | — | — |

• Files reviewed: 27/28 changed files
• Comments generated: 0 new
• Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.