Skip to content

[NO-REVIEW] Add diagnostics for HTTP/3 cookie redirect failures - #134591

Open
rzikm wants to merge 1 commit into
dotnet:mainfrom
rzikm:rzikm/http3-cookie-redirect-investigation
Open

rzikm wants to merge 1 commit into
dotnet:mainfrom
rzikm:rzikm/http3-cookie-redirect-investigation

Conversation

@rzikm

@rzikm rzikm commented Sep 24, 2026

Copy link
Copy Markdown
Member

Purpose

Experimental diagnostics-only PR for #133842, intended for CI evidence gathering, not a fix or a merge candidate. Please apply/retain NO-REVIEW.

The reported cookie-redirect failures have five distinct final throw paths. This captures enough chronology to distinguish repeated rejected connection reuse from a pool-publication/shutdown race without assuming either hypothesis is the cause.

Diagnostics

  • Instrument the existing cookie-redirect test, preserving both cookie assertions, request/redirect behavior, GOAWAY ordering, retry limits and existing client/server timeout budgets.
  • Run its HTTP/3 variant in an isolated RemoteExecutor child to capture private HTTP/QUIC events without enabling process-wide tracing alongside unrelated tests. Unsupported platforms still execute the original scenario with phase-only logging; they are not skipped.
  • Capture initial versus redirected request phases, request/pool/managed-connection identities, native handles and stream IDs, waiter publication, pool insertion/invalidation outcomes, and GOAWAY sent/received/applied boundaries.
  • Label each retry path separately: peer H3_REQUEST_REJECTED, no outbound request stream (including any suppressed exception), stream at/above the GOAWAY boundary, local OperationAborted, and internal cancellation. Preserve retry-count and exception evidence rather than inferring repeated 267 errors from the final message.
  • Log client faults before disposal and fault aggregation, server phases, final task states, runtime/assembly information and native MsQuic initialization. The combinator's up-to-three-second post-fault grace is explicitly identified.
  • Bound capture to 4,096 events with an overflow count. Filter HTTP lifecycle messages; do not emit raw request/response/header dumps. Redirect events retain identity, not URI contents.

The two opt-in loopback helper changes are identical to the published helper changes in #134413 at ebcfc82faa036155193841484e2f9eae8ee8fb56. Its request-reuse test is untouched; overlap is deliberate and no common root cause is asserted.

Validation

Windows x64, CoreCLR Release/libraries Debug, local .NET 12.0.0-dev:

  • Clean short-path build.cmd clr+libs -rc release baseline and System.Net.Http rebuild: passed, no warnings/errors.
  • Primary cookie-redirect test: 4 passed, none skipped, including HTTP/3.
  • Cookie and HTTP/3 protocol/diagnostics/eviction/cancellation coverage: 519 passed, 1 existing skip.
  • System.Net.Http unit tests: 2,677 passed, 4 existing skips.
  • Standalone traced and phase-only probes verified success/task markers and header-value exclusion. A synthetic client fault verified original exception propagation and failure-before-disposal/factory ordering; completion took 3.037 seconds, consistent with the combinator grace. This is a diagnostic control, not a reproduction of the CI failure.
  • Passing XML includes the redirect marker, pool/GOAWAY chronology and local MsQuic 2.5.10.154561281 (9ff06b71fd4b4d5258361598ada5b24cbc1beb20).

A small BenchmarkDotNet observer-overhead check on the same Debug library binaries measured 28.57 ms with phase-only logging and 29.60 ms with client tracing (5 iterations, 16 invocations/iteration, 2 warmups; in-process, existing power plan). This includes test setup/exchange/cleanup but excludes child startup/stdout forwarding. It is not a baseline-versus-patch production performance comparison.

Limitations

Tracing and process isolation perturb scheduling. Local Windows passes do not reproduce or exonerate the Linux CI failures. A hard child-process kill/crash before finally loses buffered events; ordinary failures and managed factory timeouts flush them. Synthetic cookie assertion failures may themselves include expected/actual test-cookie strings. No retry/suppression workaround or CI rerun is included.

Related investigation: #133842.

Note

This PR description and implementation were generated with GitHub Copilot.

Capture isolated request, connection, pool, GOAWAY and retry-path chronology without changing cookie assertions or retry semantics. Reuse opt-in HTTP/3 loopback logging from dotnet#134413.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 24, 2026 13:54
@rzikm rzikm added the NO-REVIEW Experimental/testing PR, do NOT review it label Sep 24, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 4 pipeline(s).
12 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @karelz, @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved diagnostic correctness issues could omit or misorder failure evidence.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 Medium severity

Open (2)
What changed in this PR

Adds opt-in, isolated HTTP/3 diagnostics for cookie redirect failures without changing functional behavior.

Changes:

  • Adds retry, pooling, GOAWAY, QUIC, and lifecycle tracing.
  • Runs diagnostics in an isolated process with bounded event capture.
  • Adds loopback server and client phase logging.
File Summary
src/​libraries/​System.Net.Http/​tests/​FunctionalTests/​SocketsHttpHandlerTest.cs Captures and filters diagnostic events.
src/​libraries/​System.Net.Http/​src/​System/​Net/​Http/​SocketsHttpHandler/​Http3RequestStream.cs Logs retry paths.
src/​libraries/​System.Net.Http/​src/​System/​Net/​Http/​SocketsHttpHandler/​Http3Connection.cs Logs connection, GOAWAY, and shutdown state.
src/​libraries/​System.Net.Http/​src/​System/​Net/​Http/​SocketsHttpHandler/​ConnectionPool/​HttpConnectionPool.Http3.cs Logs pool publication and invalidation.
src/​libraries/​System.Net.Http/​src/​System/​Net/​Http/​SocketsHttpHandler/​ConnectionPool/​HttpConnectionPool.cs Adds retry correlation diagnostics.
src/​libraries/​Common/​tests/​System/​Net/​Http/​HttpClientHandlerTest.Cookies.cs Adds client and server phase logging.
src/​libraries/​Common/​tests/​System/​Net/​Http/​Http3LoopbackServer.cs Adds server lifecycle logging.
src/​libraries/​Common/​tests/​System/​Net/​Http/​Http3LoopbackConnection.cs Adds connection and stream diagnostics.

Comment on lines +6049 to +6059
var events = new ConcurrentQueue<string>();
long started = Stopwatch.GetTimestamp();
int eventCount = 0;
void Log(string message)
{
int sequence = Interlocked.Increment(ref eventCount);
if (sequence <= MaximumEvents)
{
events.Enqueue($"{sequence}: {Stopwatch.GetElapsedTime(started).TotalMilliseconds:F3}ms thread={Environment.CurrentManagedThreadId} {message}");
}
}
Comment on lines +6081 to +6085
foreach (string entry in events.ToArray())
{
_output.WriteLine(entry);
}
_output.WriteLine($"Diagnostic events beyond limit: {Math.Max(0, Volatile.Read(ref eventCount) - MaximumEvents)}");

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-System.Net.Http NO-REVIEW Experimental/testing PR, do NOT review it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants