Remove Socket.SendFile 2GB limitation on Windows by partitioning large files #120634

Copilot · 2025-10-11T17:38:03Z

Summary: Remove Socket.SendFile 2GB limitation on Windows

Problem

On Windows, the Socket.SendFile API was limited to sending files up to int.MaxValue (2GB) due to the Win32 TransmitFile function's int numberOfBytesToWrite parameter. This limitation did not exist on Linux/macOS. The issue has been tracked as #42534.

Solution

Implemented automatic partitioning of large file sends into multiple TransmitFile calls when the file exceeds 2GB, as suggested by the TransmitFile Win32 API documentation.

Implementation Details

Modified: SocketPal.Windows.cs::SendFile

Detect files larger than int.MaxValue using RandomAccess.GetLength
For large files:
1. Separate behavior flags (WriteBehind, UseSystemThread, UseKernelApc) from terminal flags (Disconnect, ReuseSocket)
2. Apply behavior flags to all operations for consistent performance characteristics
3. Send preBuffer separately if present (with behavior flags)
4. Loop through file in int.MaxValue-sized chunks (with behavior flags):
  - Position file pointer using SetFilePointerEx
  - Call TransmitFile with exact chunk size
5. Send postBuffer with all flags (behavior + terminal flags apply to final operation)
For small files (≤ 2GB), use original single TransmitFile call

Modified: TransmitFileHelper

Added optional numberOfBytesToWrite parameter (defaults to 0 for "send entire file")
Preserves existing behavior for all current callers

Modified: System.Net.Sockets.csproj

Added Interop.SetFilePointerEx.cs to enable file positioning

Modified: SendFile.cs tests

Removed [ActiveIssue] attribute from GreaterThan2GBFile_SendsAllBytes test

Testing

✅ Code compiles successfully on all platforms
✅ All 154 existing SendFile tests pass on Linux (Unix implementation unchanged)
⏳ Windows CI will validate the fix for files > 2GB

Changes

Analyze the current implementation
Modify SocketPal.Windows.cs to partition large file sends
Update TransmitFileHelper to accept numberOfBytesToWrite parameter
Add loop to send file in chunks when size > int.MaxValue
Add SetFilePointerEx interop to System.Net.Sockets.csproj
Handle behavior and terminal flags correctly for large files
Remove ActiveIssue attribute from GreaterThan2GBFile test
Verify code compiles successfully
Validate existing tests pass on Linux

Original prompt

This section details on the original issue you should resolve

<issue_title>Consider removing Socket.SendFile 2GB limitation on Windows</issue_title>
<issue_description>Separated out of #41906

On Windows, the Socket.{Begin}SendFile API inherits the limitation of the Win32 TransmitFile function that constraints it to only being able to send int.MaxValue bytes; anything larger results in an error. This is not a regression, but we should consider supporting this by manually partitioning as the TransmitFile docs suggest. This already works on Linux and macOS.</issue_description>

Comments on the Issue (you are @copilot in this section)
@ Tagging subscribers to this area: @dotnet/ncl See info in area-owners.md if you want to be subscribed.

Fixes #42534

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Co-authored-by: stephentoub <[email protected]>

stephentoub · 2025-10-12T01:26:28Z

src/libraries/System.Net.Sockets/src/System/Net/Sockets/SocketPal.Windows.cs

+                    // Send postBuffer if present, or apply disconnect/reuse flags to last operation
+                    if (postBuffer.Length > 0)
+                    {
+                        bool success = TransmitFileHelper(handle, null, null, IntPtr.Zero, 0, (IntPtr)postPinnedBuffer, postBuffer.Length, flags);
+                        if (!success)
+                        {
+                            return GetLastSocketError();
+                        }
+                    }
+                    else if ((flags & (TransmitFileOptions.Disconnect | TransmitFileOptions.ReuseSocket)) != 0)
+                    {
+                        // If no postBuffer but disconnect/reuse flags are set, send an empty buffer with the flags
+                        bool success = TransmitFileHelper(handle, null, null, IntPtr.Zero, 0, IntPtr.Zero, 0, flags);
+                        if (!success)
+                        {
+                            return GetLastSocketError();
+                        }
+                    }


@copilot, what about the other TransmitFileFlags? If any are specified, might they need to be passed to the other TransmitFileHelper calls, not just the last one?

You're right - I've separated the flags into behavior flags (WriteBehind, UseSystemThread, UseKernelApc) and terminal flags (Disconnect, ReuseSocket). Behavior flags now apply to all TransmitFileHelper calls for consistent performance characteristics, while terminal flags only apply to the final operation. Fixed in commit e5b8a72.

stephentoub · 2025-10-12T01:27:54Z

src/libraries/System.Net.Sockets/src/System/Net/Sockets/SocketPal.Windows.cs

+                            return GetLastSocketError();
+                        }
+                    }
+                    else if ((flags & (TransmitFileOptions.Disconnect | TransmitFileOptions.ReuseSocket)) != 0)


@copilot, is splitting this if/else necessary? What would happen if you just always did the if block, regardless of whether postBuffer.Length > 0? Wouldn't that be the same thing?

Good catch - the if/else split was unnecessary. Now the postBuffer send always happens with the full flags, regardless of whether postBuffer is empty or not. This simplifies the logic. Fixed in commit e5b8a72.

Copilot

Pull Request Overview

This PR removes the 2GB file size limitation for Socket.SendFile on Windows by implementing automatic file partitioning. The Windows TransmitFile API is limited to int.MaxValue bytes, but the solution partitions large files into multiple chunks while maintaining proper handling of pre/post buffers and flags.

Implemented partitioning logic to handle files larger than 2GB by splitting them into manageable chunks
Enhanced the TransmitFileHelper method to accept explicit byte counts for precise chunk transmission
Enabled the previously skipped test case for files greater than 2GB on Windows

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
src/libraries/System.Net.Sockets/tests/FunctionalTests/SendFile.cs	Removed ActiveIssue attribute to enable >2GB file test on Windows
src/libraries/System.Net.Sockets/src/System/Net/Sockets/SocketPal.Windows.cs	Added file partitioning logic and updated TransmitFileHelper signature
src/libraries/System.Net.Sockets/src/System.Net.Sockets.csproj	Added reference to SetFilePointerEx interop for file positioning

Copilot · 2025-10-12T01:29:27Z

src/libraries/System.Net.Sockets/src/System/Net/Sockets/SocketPal.Windows.cs

+                        // Set the file pointer to the current offset
+                        if (!Interop.Kernel32.SetFilePointerEx(fileHandle!, offset, out _, 0 /* FILE_BEGIN */))
+                        {
+                            return GetLastSocketError();


When SetFilePointerEx fails, GetLastSocketError() returns socket-specific errors but SetFilePointerEx is a file operation. Consider using Marshal.GetLastWin32Error() or GetLastPInvokeError() to get the actual file operation error code.

Suggested change

return GetLastSocketError();

// Use Marshal.GetLastWin32Error() for file operation errors

return (SocketError)Marshal.GetLastWin32Error();

Copilot · 2025-10-12T01:29:27Z

src/libraries/System.Net.Sockets/src/System/Net/Sockets/SocketPal.Windows.cs

+                    // Send preBuffer if present
+                    if (preBuffer.Length > 0)
+                    {
+                        bool success = TransmitFileHelper(handle, null, null, (IntPtr)prePinnedBuffer, preBuffer.Length, IntPtr.Zero, 0, TransmitFileOptions.UseDefaultWorkerThread);


The variable name 'success' is reused in multiple scopes within the same method. Consider using more descriptive names like 'preBufferSent', 'postBufferSent', and 'flagsApplied' to improve code clarity.

Copilot · 2025-10-12T01:29:27Z

src/libraries/System.Net.Sockets/src/System/Net/Sockets/SocketPal.Windows.cs

+                    // Send postBuffer if present, or apply disconnect/reuse flags to last operation
+                    if (postBuffer.Length > 0)
+                    {
+                        bool success = TransmitFileHelper(handle, null, null, IntPtr.Zero, 0, (IntPtr)postPinnedBuffer, postBuffer.Length, flags);


The variable name 'success' is reused in multiple scopes within the same method. Consider using more descriptive names like 'preBufferSent', 'postBufferSent', and 'flagsApplied' to improve code clarity.

Copilot · 2025-10-12T01:29:28Z

src/libraries/System.Net.Sockets/src/System/Net/Sockets/SocketPal.Windows.cs

+                    else if ((flags & (TransmitFileOptions.Disconnect | TransmitFileOptions.ReuseSocket)) != 0)
+                    {
+                        // If no postBuffer but disconnect/reuse flags are set, send an empty buffer with the flags
+                        bool success = TransmitFileHelper(handle, null, null, IntPtr.Zero, 0, IntPtr.Zero, 0, flags);


The variable name 'success' is reused in multiple scopes within the same method. Consider using more descriptive names like 'preBufferSent', 'postBufferSent', and 'flagsApplied' to improve code clarity.

…operation Co-authored-by: stephentoub <[email protected]>

Initial plan

0fd2a96

Copilot AI assigned Copilot and stephentoub Oct 11, 2025

Copilot started work on behalf of stephentoub October 11, 2025 17:38 View session

Copilot AI and others added 4 commits October 11, 2025 18:05

Implement partitioning for files > 2GB in SendFile on Windows

6535829

Co-authored-by: stephentoub <[email protected]>

Fix build errors and add SetFilePointerEx interop

b5f6781

Co-authored-by: stephentoub <[email protected]>

Remove ActiveIssue attribute from GreaterThan2GBFile test

e6ef307

Co-authored-by: stephentoub <[email protected]>

Handle disconnect/reuse flags for large files without postBuffer

e22cf63

Co-authored-by: stephentoub <[email protected]>

Copilot AI changed the title ~~[WIP] Consider removing Socket.SendFile 2GB limitation on Windows~~ Remove Socket.SendFile 2GB limitation on Windows by partitioning large files Oct 11, 2025

Copilot AI requested a review from stephentoub October 11, 2025 18:18

Copilot finished work on behalf of stephentoub October 11, 2025 18:18

stephentoub added the area-System.Net.Sockets label Oct 11, 2025

stephentoub reviewed Oct 12, 2025

View reviewed changes

stephentoub marked this pull request as ready for review October 12, 2025 01:28

Copilot AI review requested due to automatic review settings October 12, 2025 01:28

Copilot started work on behalf of stephentoub October 12, 2025 01:28 View session

Copilot AI reviewed Oct 12, 2025

View reviewed changes

Apply behavior flags to all operations, terminal flags only to final …

036db39

…operation Co-authored-by: stephentoub <[email protected]>

Copilot AI requested a review from stephentoub October 12, 2025 01:53

Copilot finished work on behalf of stephentoub October 12, 2025 01:53

build-analysis bot mentioned this pull request Oct 12, 2025

[browser] HalfTests.ExplicitConversion_FromSingle failing due to NaN != NaN #103347

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Remove Socket.SendFile 2GB limitation on Windows by partitioning large files #120634

Remove Socket.SendFile 2GB limitation on Windows by partitioning large files #120634

Copilot AI commented Oct 11, 2025 •

edited

Loading

Uh oh!

stephentoub Oct 12, 2025

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

stephentoub Oct 12, 2025

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	return GetLastSocketError();
	// Use Marshal.GetLastWin32Error() for file operation errors
	return (SocketError)Marshal.GetLastWin32Error();

Remove Socket.SendFile 2GB limitation on Windows by partitioning large files #120634

Are you sure you want to change the base?

Remove Socket.SendFile 2GB limitation on Windows by partitioning large files #120634

Conversation

Copilot AI commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary: Remove Socket.SendFile 2GB limitation on Windows

Problem

Solution

Implementation Details

Testing

Changes

Comments on the Issue (you are @copilot in this section)

Uh oh!

stephentoub Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

stephentoub Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Oct 11, 2025 •

edited

Loading