CSHARP-3550: CSOT: Server Selection #1705

sanych-sun · 2025-06-06T01:11:08Z

No description provided.

rstam · 2025-06-06T18:53:45Z

src/MongoDB.Driver/OperationCancellationContext.cs

+
+        public TimeSpan Elapsed => _stopwatch.Elapsed;
+
+        public TimeSpan Timeout { get; }


I think we need TWO properties:

public TimeSpan OperationTimeout { get; } public TimeSpan StepTimeout { get; }

to keep track of BOTH kinds of timeouts.

That probably also means we need two stopwatches. One for the operation as a whole, and the other for whatever sub step we are in (server selection, etc..).

The reason I say this is that when a timeout occurs we should be able to say in the message whether the whole operation timed out or whether a sub step timed out.

Interesting idea. What about having single timeout as is and ParentContext property instead? Somehow I would like to keep the Context kind of immutable.

Having a ParentContext might be a good idea in its own right.

You still need a way to detect when a timeout occurs whether it was the OperationTimeout or the StepTimeout that timed out. Does a ParentContext help you with that?

Pseudo-code to check which the timeout occurred:

if (operationContext.IsTimedOut()) { var isLocalTimeout = operationContext?.ParentContext.IsTimedOut() == false; }

So the logic is: timeout is "local" when there is no parent context or parent context is not timed out yet.

This code works, because child context is always more strict then parent. It means there should not be a case when ParentContext is timed out, but child is not.

In a way we are talkinig about the same thing, since a context represents (among other things) a timeout.

d) whole operation is timed out, but server selection is not - this is impossible state as per spec.

I still think this is possible. I just have to configure the operation timeout to be smaller than the server selection timeout.

Spec says we should use more strict timeout in such situation:

If timeoutMS is set, drivers MUST use min(serverSelectionTimeoutMS, remaining timeoutMS), referred to as computedServerSelectionTimeout as the timeout for server selection and connection checkout.

https://github.com/mongodb/specifications/blob/master/source/client-side-operations-timeout/client-side-operations-timeout.md#server-selection

I think we aren't talking about the same thing again.

I'm saying that the operation timeout can be less than the server selection timeout. Of course I can do that.

That's not the same thing as the "computedServerSelectionTimeout".

When the "computedServerSelectionTimeout" (NOT a configured value, but a value that is computed and is different evry time) times out, we probably need to know WHY it timed out. Did it timeout because the operation timed out or because server selection timed out? I think the error message should say why. Seems like the user would want to know which timeout timed out because they may want to know which timeout value to reconfigure if they actually didn't want to time out so quickly.

d) whole operation is timed out, but server selection is not - this is impossible state as per spec.

I don't think the spec says this is impossible. The spec describes how to calculate the "computedServerSelectionTimeout", but if the configured operation timeout is less than the configured server selection timeout and we timeout during server selection it is NOT the server selection that timed out, but the WHOLE operation.

As it was discussed offline:
We both were talking about the same thing, but there were small confusion regarding "configured" timeout and "computed" timeout. We confirmed that we DO need to distinguish what exactly timeout was occurred: was it "local" timeout or whole operation timeout. Also we decided to go with ParentContext approach as it seems to cover our needs and also let us have more the 2 level of nesting. To check whether it was local timeout occurred or whole operation - we have to examine the ParentContext property and see if it was timed out or not. If there is no ParentContext and the context is timed out - it means whole operation was timed out.

rstam · 2025-06-06T18:54:40Z

src/MongoDB.Driver/OperationCancellationContext.cs

+        {
+            _stopwatch = stopwatch;
+            Timeout = timeout;
+            CancellationToken = cancellationToken;


I recommend not mixing backing fields and auto-properties in the same class.

rstam · 2025-06-06T18:55:19Z

src/MongoDB.Driver/OperationCancellationContext.cs

+            return remainingTimeout < TimeSpan.Zero;
+        }
+
+        public OperationCancellationContext WithTimeout(TimeSpan timeout)


Maybe should be called WithStepTimeout?

As it was discussed we will not introduce additional properties for now, so will leave the method name as is to match the property name.

rstam · 2025-06-06T19:01:43Z

src/MongoDB.Driver/OperationCancellationContext.cs

+
+namespace MongoDB.Driver
+{
+    internal sealed class OperationCancellationContext


OperationContext?

rstam · 2025-06-06T20:52:58Z

src/MongoDB.Driver/OperationCancellationContext.cs

+
+        public TimeSpan Elapsed => _stopwatch.Elapsed;
+
+        public TimeSpan Timeout { get; }


Having a ParentContext might be a good idea in its own right.

You still need a way to detect when a timeout occurs whether it was the OperationTimeout or the StepTimeout that timed out. Does a ParentContext help you with that?

BorisDog · 2025-06-06T22:11:44Z

src/MongoDB.Driver/Core/ConnectionPools/ExclusiveConnectionPool.Helpers.cs

@@ -47,7 +47,7 @@ private Exception CreateTimeoutException(Stopwatch stopwatch, string message)
                var checkOutsForOtherCount = checkOutsCount - checkOutsForCursorCount - checkOutsForTransactionCount;

                message =
-                    $"Timed out after {stopwatch.ElapsedMilliseconds}ms waiting for a connection from the connection pool. " +
+                    $"Timed out after {operationContext.Elapsed.TotalMilliseconds}ms waiting for a connection from the connection pool. " +


minor: Looks like Elapsed.TotalMilliseconds is frequently used, does this award a shortcut property:
ElapsedMilleseconds?

BorisDog · 2025-06-06T22:14:07Z

src/MongoDB.Driver/OperationContext.cs

+        // TODO: this static field is temporary here and will be removed in a future PRs in scope of CSOT.
+        public static readonly OperationContext NoTimeout = new(System.Threading.Timeout.InfiniteTimeSpan, CancellationToken.None);
+
+        private readonly Stopwatch _stopwatch;


So we are saving the multiple stopwatches creation on operation execution path, it's minor but still nice.

src/MongoDB.Driver/Core/ConnectionPools/ExclusiveConnectionPool.Helpers.cs

sanych-sun · 2025-06-06T23:48:47Z

tests/MongoDB.Driver.Tests/Core/Clusters/ClusterTests.cs

            [Values(false, true)]
            bool async)
        {
-            var subject = CreateSubject();
+            var subject = CreateSubject(serverSelectionTimeout: TimeSpan.FromMilliseconds(10));


This adjusted serverSelectionTimeout is an optimization of the test: we probably do not have to wait whole 2 seconds here (the default server selection for this test class, see line 49).

sanych-sun · 2025-06-07T00:05:06Z

tests/MongoDB.Driver.Tests/OperationExecutorTests.cs

@@ -134,23 +134,13 @@ await Record.ExceptionAsync(() => subject.ExecuteWriteOperationAsync(null, opera
                .Subject.ParamName.Should().Be("session");
        }

-        private OperationExecutor CreateSubject(out Mock<IClusterInternal> clusterMock, out Mock<ICoreSessionHandle> implicitSessionMock)


Leftovers of the implicit session creation, that was factored out in one of the latest commits related to OperationExecutor refactoing.

Copilot

Pull Request Overview

This PR refactors various server selection and connection pool APIs to use a new OperationContext abstraction instead of separate CancellationToken and timeout TimeSpan parameters. Key changes include:

Replaced all CancellationToken (and internal Stopwatch timeout logic) with OperationContext throughout connection pool helpers, cluster selection, and binding classes.
Updated CreateTimeoutException methods to accept a TimeSpan elapsed rather than a Stopwatch.
Removed legacy overloads and helper classes associated with CancellationToken/TimeSpan and simplified server selection wait logic using OperationContext.

Reviewed Changes

Copilot reviewed 141 out of 141 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/MongoDB.Driver/Core/ConnectionPools/ExclusiveConnectionPool.Helpers.cs	Swapped `Stopwatch` usage and token/timeouts for `OperationContext`
src/MongoDB.Driver/Core/Clusters/LoadBalancedCluster.cs	Updated server selection to use `OperationContext` and `WithTimeout`
src/MongoDB.Driver/Core/Clusters/IClusterExtensions.cs	Changed extension methods to accept `OperationContext`
src/MongoDB.Driver/Core/Clusters/ICluster.cs	Updated interface signatures to use `OperationContext`
src/MongoDB.Driver/Core/Clusters/Cluster.cs	Refactored internal select/wait loops to data in `OperationContext`
src/MongoDB.Driver/Core/Bindings/*/.cs	Replaced all `CancellationToken` parameters in binding APIs with `OperationContext`

Comments suppressed due to low confidence (3)

src/MongoDB.Driver/Core/ConnectionPools/ExclusiveConnectionPool.Helpers.cs:50

[nitpick] Using TotalMilliseconds can include fractional values (e.g., 12.345ms). Consider formatting with a fixed precision or rounding (e.g., {elapsed.TotalMilliseconds:F0}) to produce a cleaner integer-based message.

message = $"Timed out after {elapsed.TotalMilliseconds}ms waiting for a connection from the connection pool. " +

src/MongoDB.Driver/Core/Clusters/LoadBalancedCluster.cs:179

The new WithTimeout logic and WaitTask behavior are significant changes to server selection. Please add unit tests that simulate both timeout and cancellation scenarios for SelectServer/SelectServerAsync using mocks/stubs for _serverReadyTaskCompletionSource.

var serverSelectionOperationContext = operationContext.WithTimeout(_settings.ServerSelectionTimeout);

src/MongoDB.Driver/Core/Clusters/ICluster.cs:64

Removing the old CancellationToken overloads is a breaking API change. Consider retaining the old overloads (marked obsolete) to preserve backward compatibility, or provide clear upgrade guidance in release notes.

IServer SelectServer(IServerSelector selector, OperationContext operationContext);

Copilot · 2025-06-09T21:17:04Z

src/MongoDB.Driver/Core/Clusters/LoadBalancedCluster.cs

-            if (index != 0)
+            try
+            {
+                serverSelectionOperationContext.WaitTask(_serverReadyTaskCompletionSource.Task);


The original code explicitly threw on cancellation (cancellationToken.ThrowIfCancellationRequested()), but this method only catches TimeoutException. Consider catching OperationCanceledException from WaitTask or explicitly checking operationContext.CancellationToken to ensure cancellations are surfaced correctly.

Copilot · 2025-06-09T21:17:05Z

src/MongoDB.Driver/Core/Clusters/IClusterExtensions.cs

            // should be pinned.
-            var server = cluster.SelectServer(selector, cancellationToken);
+            var server = cluster.SelectServer(selector, operationContext);


[nitpick] We added non-null checks for selector and operationContext in core cluster methods; consider adding Ensure.IsNotNull(operationContext, nameof(operationContext)); here as well to maintain consistent validation.

Copilot · 2025-06-09T21:17:05Z

src/MongoDB.Driver/Core/Bindings/WritableServerBinding.cs

        {
-            return GetReadChannelSource(null, cancellationToken);
+            return GetReadChannelSource(null, operationContext);
        }

-        public Task<IChannelSourceHandle> GetReadChannelSourceAsync(CancellationToken cancellationToken)
+        public Task<IChannelSourceHandle> GetReadChannelSourceAsync(OperationContext operationContext)
        {


[nitpick] Most public API methods now guard against null operationContext. Add an Ensure.IsNotNull(operationContext, nameof(operationContext)); at the top of this method to match the pattern in cluster selection.

rstam · 2025-06-10T17:03:28Z

src/MongoDB.Driver/Core/Bindings/ChannelReadWriteBinding.cs

        {
            ThrowIfDisposed();
            return Task.FromResult(GetChannelSourceHelper());
        }

-        public IChannelSourceHandle GetReadChannelSource(IReadOnlyCollection<ServerDescription> deprioritizedServers, CancellationToken cancellationToken)
+        public IChannelSourceHandle GetReadChannelSource(IReadOnlyCollection<ServerDescription> deprioritizedServers, OperationContext operationContext)


I think operationContext should always be the first parameter.

Same everywhere. I stopped adding this comment after a while.

Done. Here and in all other places.

rstam · 2025-06-10T17:04:31Z

src/MongoDB.Driver/Core/Bindings/CoreSession.cs

@@ -145,6 +145,8 @@ public bool IsInTransaction
        {
            EnsureAbortTransactionCanBeCalled(nameof(AbortTransaction));

+            // TODO: CSOT implement proper way to obtain the operationCancellationContext
+            var operationCancellationContext = new OperationContext(Timeout.InfiniteTimeSpan, cancellationToken);


Variable name should be operationContext.

rstam · 2025-06-10T17:06:27Z

src/MongoDB.Driver/Core/Bindings/CoreSession.cs

@@ -537,21 +545,21 @@ private void EnsureTransactionsAreSupported()
            }
        }

-        private TResult ExecuteEndTransactionOnPrimary<TResult>(IReadOperation<TResult> operation, CancellationToken cancellationToken)
+        private TResult ExecuteEndTransactionOnPrimary<TResult>(IReadOperation<TResult> operation, OperationContext operationContext)


While it is true that cancellationToken is always the last parameter, I think that operationContext should always be the first parameter.

rstam · 2025-06-10T17:07:17Z

src/MongoDB.Driver/Core/Bindings/IBinding.cs


-        IChannelSourceHandle GetReadChannelSource(IReadOnlyCollection<ServerDescription> deprioritizedServers, CancellationToken cancellationToken);
-        Task<IChannelSourceHandle> GetReadChannelSourceAsync(IReadOnlyCollection<ServerDescription> deprioritizedServers, CancellationToken cancellationToken);
+        IChannelSourceHandle GetReadChannelSource(IReadOnlyCollection<ServerDescription> deprioritizedServers, OperationContext operationContext);


Seems like operationContext should always be the first parameter.

rstam · 2025-06-10T17:08:50Z

src/MongoDB.Driver/Core/Clusters/ICluster.cs

@@ -62,8 +61,8 @@ internal interface IClusterInternal : ICluster

        void Initialize();

-        IServer SelectServer(IServerSelector selector, CancellationToken cancellationToken);
-        Task<IServer> SelectServerAsync(IServerSelector selector, CancellationToken cancellationToken);
+        IServer SelectServer(IServerSelector selector, OperationContext operationContext);


Seems like operationContext should always be the first parameter.

rstam · 2025-06-10T17:13:25Z

src/MongoDB.Driver/Core/Clusters/Cluster.cs

                    }
                }
+                catch (TimeoutException)
+                {
+                    var message = BuildTimeoutExceptionMessage(_settings.ServerSelectionTimeout, selector, helper.Description);


BuildTimeoutExceptionMessage doesn't inform the user whether the operation timed out or the server selection timed out.

rstam · 2025-06-10T17:15:48Z

src/MongoDB.Driver/Core/Clusters/IClusterExtensions.cs

@@ -29,7 +28,7 @@ public static IServer SelectServerAndPinIfNeeded(
            ICoreSessionHandle session,
            IServerSelector selector,
            IReadOnlyCollection<ServerDescription> deprioritizedServers,
-            CancellationToken cancellationToken)
+            OperationContext operationContext)


Seems like operationContext should always be the "first" parameter. In the case of an extension method that would be the "second" parameter, which "looks" like the first parameter when called.

rstam · 2025-06-10T17:17:23Z

src/MongoDB.Driver/Core/Operations/AggregateOperation.cs

@@ -270,27 +269,27 @@ public bool? UseCursor

        // methods
        /// <inheritdoc/>
-        public IAsyncCursor<TResult> Execute(IReadBinding binding, CancellationToken cancellationToken)
+        public IAsyncCursor<TResult> Execute(IReadBinding binding, OperationContext operationContext)


Seems like the operationContext should always be the first parameter.

BorisDog · 2025-06-10T20:00:25Z

src/MongoDB.Driver/Core/ConnectionPools/MaintenanceHelper.cs

                    {
-                        var connection = connectionCreator.CreateOpened(cancellationToken);
+                        var connection = connectionCreator.CreateOpened(operationContext);


The reason for tiny 20ms timeout is just try to take turn in the connecting queue without blocking checkouts.
This timeout is applied to every step individually: MaxConnections and MaxConnecting.
Seems that this code changes this and allocates 20ms to

MaxConnections

MaxConnecting

Whole Connection creation operation (excluding open)

If this thought is correct, it's really surprising that this was not caught by tests.

BorisDog · 2025-06-10T20:15:36Z

src/MongoDB.Driver/Core/ConnectionPools/ExclusiveConnectionPool.Helpers.cs

                try
                {
-                    StartCheckingOut(stopwatch);
-                    _poolQueueWaitResult = _pool._maxConnectionsQueue.WaitSignaled(_timeout, cancellationToken);
+                    StartCheckingOut();


Looks like our previous ConnectionCheckedOutEvent duration calculation was a bit different from spec:
It didn't include emitting (and processing) the CheckoutStarted event.

Probably we should mention this nuance in release notes, as the duration will include client events processing code.

BorisDog · 2025-06-10T20:28:19Z

src/MongoDB.Driver/Core/ConnectionPools/ExclusiveConnectionPool.Helpers.cs

                _poolQueueWaitResult switch
                {
                    SemaphoreSlimSignalable.SemaphoreWaitResult.Signaled =>
                        MongoConnectionPoolPausedException.ForConnectionPool(_pool._endPoint),
                    SemaphoreSlimSignalable.SemaphoreWaitResult.TimedOut =>
-                        _pool.CreateTimeoutException(stopwatch, $"Timed out waiting for a connection after {stopwatch.ElapsedMilliseconds}ms."),
+                        _pool.CreateTimeoutException(operationContext.Elapsed, $"Timed out waiting for a connection after {operationContext.Elapsed.TotalMilliseconds}ms."),


Ideally we'd get the timeout closer to WaitSignaled result. But I don't see a very simple solution for this.

BorisDog · 2025-06-10T20:32:22Z

src/MongoDB.Driver/Core/ConnectionPools/ExclusiveConnectionPool.Helpers.cs

                    _pool._poolState.ThrowIfNotReady();

                    if (_connectingWaitStatus == SemaphoreSlimSignalable.SemaphoreWaitResult.TimedOut)
                    {
-                        _pool.CreateTimeoutException(stopwatch, $"Timed out waiting for in connecting queue after {stopwatch.ElapsedMilliseconds}ms.");
+                        _pool.CreateTimeoutException(operationContext.Elapsed, $"Timed out waiting for in connecting queue after {operationContext.Elapsed.TotalMilliseconds}ms.");


This one is not accurate, Elapsed includes more waits than just the maxConnecting queue.

BorisDog · 2025-06-10T20:33:59Z

src/MongoDB.Driver/Core/ConnectionPools/ExclusiveConnectionPool.Helpers.cs

-                            SemaphoreSlimSignalable.SemaphoreWaitResult.Entered => CreateOpenedInternal(cancellationToken),
-                            SemaphoreSlimSignalable.SemaphoreWaitResult.TimedOut => throw CreateTimeoutException(stopwatch),
+                            SemaphoreSlimSignalable.SemaphoreWaitResult.Entered => CreateOpenedInternal(operationContext),
+                            SemaphoreSlimSignalable.SemaphoreWaitResult.TimedOut => throw CreateTimeoutException(operationContext.Elapsed),


Same issue, the exception is just for maxConnecting queue accumulative waits.
Maybe we need to understand what is prescribed by spec here.

BorisDog · 2025-06-10T20:40:35Z

tests/MongoDB.Driver.Tests/OperationContextTests.cs

+        }
+
+        [Fact]
+        public void RenainingTimeout_could_be_negative()


type: remaining

CSHARP-3550: CSOT: Server Selection

de8c0ce

sanych-sun requested review from rstam, BorisDog and adelinowona June 6, 2025 01:11

sanych-sun requested a review from a team as a code owner June 6, 2025 01:11

sanych-sun added the feature label Jun 6, 2025

sanych-sun removed the request for review from a team June 6, 2025 01:16

rstam requested changes Jun 6, 2025

View reviewed changes

Rename OperationCancellationContext to OperationContext

6f38b2a

BorisDog reviewed Jun 6, 2025

View reviewed changes

sanych-sun commented Jun 6, 2025

View reviewed changes

PR

b819cc7

sanych-sun commented Jun 7, 2025

View reviewed changes

pr

fa4b6d3

sanych-sun requested a review from rstam June 7, 2025 00:55

pr

da1d212

sanych-sun requested a review from BorisDog June 9, 2025 20:48

BorisDog requested a review from Copilot June 9, 2025 21:15

Copilot AI reviewed Jun 9, 2025

View reviewed changes

rstam requested changes Jun 10, 2025

View reviewed changes

PR

d80fa50

sanych-sun requested a review from rstam June 10, 2025 19:09

BorisDog requested changes Jun 10, 2025

View reviewed changes


		public TimeSpan Elapsed => _stopwatch.Elapsed;

		public TimeSpan Timeout { get; }

CSHARP-3550: CSOT: Server Selection #1705

Are you sure you want to change the base?

CSHARP-3550: CSOT: Server Selection #1705

Conversation

sanych-sun commented Jun 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanych-sun Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanych-sun Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rstam Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

sanych-sun Jun 6, 2025 •

edited

Loading

sanych-sun Jun 6, 2025 •

edited

Loading

rstam Jun 6, 2025 •

edited

Loading