test(ci): publish raw Claude hook latency reports with runner metadata

[gnanirahulnutakki]

Parent: #364

Depends on: #378

Problem

The informational Claude hook latency job currently prints only median/p95/p99 and assertion output. It does not preserve the raw sample distribution or a machine-readable report, and functional transport failures are mixed with statistical threshold failures in pytest output.

Without a durable artifact, reviewers cannot independently recompute percentiles, compare runner classes, or distinguish a native read failure from hosted-runner tail jitter.

Primary references, rechecked 2026-07-23:

• GitHub workflow artifacts
• GitHub-hosted runners
• GitHub Actions variables and runner metadata

Scope

In scope:

• Produce versioned machine-readable latency evidence for subprocess cold, native daemon-client, shell-wrapper telemetry, and in-process daemon hot paths.
• Record every raw duration sample in deterministic ordinal order.
• Record the percentile method, configured thresholds, and recomputable median/p95/p99.
• Record bounded runner metadata: source SHA, event, run ID/attempt, runner OS/architecture, ImageOS, ImageVersion, and exact Python implementation/version.
• Represent functional failures separately from statistical threshold violations.
• Preserve partial evidence when a warmup or measured call fails.
• Write reports atomically under ${{ runner.temp }}.
• Upload the report directory with if: always() using the repository's digest-pinned actions/upload-artifact.
• Keep the job informational.

Out of scope:

• changing native retry behavior beyond dependency fix(claude): preserve native response-read errno and bound EINTR retries #378;
• choosing the final benchmark execution model;
• voting across multiple runners;
• changing latency thresholds;
• deduplicating workflow events; and
• making latency a required branch-protection context.

Acceptance criteria

• Every benchmark path emits a versioned JSON report or a versioned entry in an artifact manifest.
• Reports contain the complete raw sample distribution, not only summaries.
• Reported median/p95/p99 can be exactly recomputed from raw samples using the declared method.
• Non-finite, negative, missing, duplicated, or out-of-order samples are rejected by deterministic validation.
• Functional outcomes identify the stage and stable native exit/errno classification without request data.
• Functional failures and threshold violations are separate fields and separate human-readable pytest messages.
• A report is preserved when a warmup, native call, or threshold assertion fails.
• Metadata is populated from an explicit allowlist; no environment dump is recorded.
• Artifacts are uploaded on success and failure with a bounded retention period.
• Missing report output is visible rather than silently accepted.
• The job remains continue-on-error and excluded from the blocking tests aggregate.

Targeted tests

• Report schema/version and required-field tests.
• Raw-sample ordering, count, finiteness, and percentile recomputation tests.
• Functional-error fixture distinct from threshold-violation fixture.
• Partial-report persistence after a simulated native failure.
• Metadata allowlist and host-path/token redaction tests.
• Workflow contract test proving if: always(), digest-pinned upload action, ${{ runner.temp }} output, bounded retention, and no silent missing-file behavior.
• Focused latency-report tests, Ruff, pre-commit, check-local.sh --quick, and YAML validation.

A base-revision red proof should show that raw-report and artifact-contract tests fail on unmodified dev.

Security and cost

Do not serialize environment variables wholesale. Reports must exclude tokens, mission passports, request bodies, tool arguments, usernames, absolute executable paths, socket paths, and temporary-directory paths.

This child must not add benchmark repetitions or runner jobs. Raw JSON for existing bounded samples should be small. Retention must remain explicit and bounded because artifact storage accrues cost.

Dependencies and overlap

• Land fix(claude): preserve native response-read errno and bound EINTR retries #378 first so native failure reports have a stable errno contract.
• Open Dependabot PRs ci(deps): bump actions/setup-python from 6.3.0 to 7.0.0 #358 and ci(deps): bump actions/setup-go from 6.5.0 to 7.0.0 #360 both modify .github/workflows/tests.yml. PR ci(deps): bump actions/setup-python from 6.3.0 to 7.0.0 #358 edits the latency job's setup block directly; this child must wait, coordinate, or rebase cleanly and must not overwrite either PR.
• Use the repository's existing actions/upload-artifact pin and artifact conventions.
• No external credentials, model access, Kubernetes workload, or Docker registry is required.