Add UploadWithResponseAsync API #4062

GarrettBeatty · 2025-10-18T15:42:34Z

Stacked PRs:

Description

This change creates a new api UploadWithResponseAsync. It is the same as the existing UploadAsync api but it returns the TransferUtilityUploadResponse object.

How

In order to re-use as much as the existing code as possible, I made BaseCommand take in a generic which is the return type of the ExecuteAsync function. All commands UploadCommand, DownloadCommand, etc will pass in their expected return type TransferUtilityUploadResponse, TransferUTilityDownloadResponse, etc.

In order to make this change in one go, I had to create place holder classes for TransferUtilityDownloadDirectoryResponse and other responses. These will eventually be populated in future PRs

Motivation and Context

To adhere to the SEP

Testing

1.dry run - d704b36b-7329-42dc-80e0-525a2f82bbcc pass
2. Integration tests

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist

My code follows the code style of this project
My change requires a change to the documentation
I have updated the documentation accordingly
I have read the README document
I have added tests to cover my changes
All new and existing tests passed

License

I confirm that this pull request can be released under the Apache 2 license

stack-info: PR: #4062, branch: GarrettBeatty/stacked/4

Copilot

Pull Request Overview

Adds typed response support to S3 TransferUtility and introduces new UploadWithResponseAsync APIs that return upload metadata (e.g., ETag). Refactors internal command pipeline to a generic BaseCommand pattern and adds response types for upload/download/stream/directory/abort operations.

Add UploadWithResponseAsync overloads on TransferUtility returning TransferUtilityUploadResponse
Introduce BaseCommand and update internal commands to return typed responses
Add response DTOs for directory/download/open stream/abort operations and update dev config to minor

Reviewed Changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
sdk/src/Services/S3/Custom/Transfer/_async/TransferUtility.async.cs	Adds UploadWithResponseAsync overloads and wires typed command execution/return.
sdk/src/Services/S3/Custom/Transfer/TransferUtilityUploadDirectoryResponse.cs	Adds response DTO for upload directory (placeholder).
sdk/src/Services/S3/Custom/Transfer/TransferUtilityOpenStreamResponse.cs	Adds response DTO for open stream (currently empty).
sdk/src/Services/S3/Custom/Transfer/TransferUtilityDownloadResponse.cs	Adds response DTO for download (placeholder).
sdk/src/Services/S3/Custom/Transfer/TransferUtilityDownloadDirectoryResponse.cs	Adds response DTO for download directory (namespace differs from others).
sdk/src/Services/S3/Custom/Transfer/TransferUtilityAbortMultipartUploadsResponse.cs	Adds response DTO for abort multipart uploads (placeholder).
sdk/src/Services/S3/Custom/Transfer/TransferUtility.cs	Updates internal GetUploadCommand to typed BaseCommand.
sdk/src/Services/S3/Custom/Transfer/Internal/_bcl+netstandard/UploadDirectoryCommand.cs	Returns TransferUtilityUploadDirectoryResponse from ExecuteAsync.
sdk/src/Services/S3/Custom/Transfer/Internal/_bcl+netstandard/DownloadDirectoryCommand.cs	Returns TransferUtilityDownloadDirectoryResponse from ExecuteAsync.
sdk/src/Services/S3/Custom/Transfer/Internal/_async/SimpleUploadCommand.async.cs	Returns TransferUtilityUploadResponse and fires completed event.
sdk/src/Services/S3/Custom/Transfer/Internal/_async/OpenStreamCommand.async.cs	Returns TransferUtilityOpenStreamResponse (currently not carrying stream).
sdk/src/Services/S3/Custom/Transfer/Internal/_async/MultipartUploadCommand.async.cs	Returns TransferUtilityUploadResponse; unseekable path returns mapped response.
sdk/src/Services/S3/Custom/Transfer/Internal/_async/DownloadCommand.async.cs	Returns TransferUtilityDownloadResponse after download completes.
sdk/src/Services/S3/Custom/Transfer/Internal/_async/BaseCommand.async.cs	Makes BaseCommand generic and updates ExecuteCommandAsync signature.
sdk/src/Services/S3/Custom/Transfer/Internal/_async/AbortMultipartUploadsCommand.async.cs	Returns TransferUtilityAbortMultipartUploadsResponse from ExecuteAsync.
sdk/src/Services/S3/Custom/Transfer/Internal/UploadDirectoryCommand.cs	Type aligns with BaseCommand.
sdk/src/Services/S3/Custom/Transfer/Internal/SimpleUploadCommand.cs	Type aligns with BaseCommand.
sdk/src/Services/S3/Custom/Transfer/Internal/OpenStreamCommand.cs	Type aligns with BaseCommand; removes Return override.
sdk/src/Services/S3/Custom/Transfer/Internal/MultipartUploadCommand.cs	Type aligns with BaseCommand.
sdk/src/Services/S3/Custom/Transfer/Internal/DownloadDirectoryCommand.cs	Type aligns with BaseCommand.
sdk/src/Services/S3/Custom/Transfer/Internal/DownloadCommand.cs	Type aligns with BaseCommand.
sdk/src/Services/S3/Custom/Transfer/Internal/BaseCommand.cs	Converts BaseCommand to generic and removes Return property.
sdk/src/Services/S3/Custom/Transfer/Internal/AbortMultipartUploadsCommand.cs	Type aligns with BaseCommand.
generator/.DevConfigs/77d980ad-8f58-4f2e-97f8-d2c8c5ba3732.json	Adds dev config (minor) noting new UploadWithResponseAsync APIs and response-mapping enhancements.

sdk/src/Services/S3/Custom/Transfer/TransferUtilityDownloadDirectoryResponse.cs

sdk/src/Services/S3/Custom/Transfer/TransferUtilityOpenStreamResponse.cs

sdk/src/Services/S3/Custom/Transfer/Internal/_async/OpenStreamCommand.async.cs

sdk/src/Services/S3/Custom/Transfer/TransferUtilityOpenStreamResponse.cs

sdk/src/Services/S3/Custom/Transfer/_async/TransferUtility.async.cs

stack-info: PR: #4062, branch: GarrettBeatty/stacked/4

GarrettBeatty · 2025-10-20T14:11:08Z

@philasmar this pr is still in draft but wanted to get your feedback on this approach. seems to be the easiest/best way to add this new api and re-use the existing functionality but using generics. i havent filled in the details of each response object yet (they are placeholder classes). was going to do that after

GarrettBeatty · 2025-10-20T14:11:39Z

sdk/src/Services/S3/Custom/Transfer/Internal/BaseCommand.cs

+    /// <typeparam name="TResponse">Type of response returned by the command</typeparam>
+    internal abstract partial class BaseCommand<TResponse> where TResponse : class
    {
-        public virtual object Return


this is not being used anywhere so removing it to avoid confusion

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull Request Overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 5 comments.

Copilot · 2025-10-24T00:40:39Z

sdk/src/Services/S3/Custom/Transfer/_async/TransferUtility.async.cs

        ///     A cancellation token that can be used by other objects or threads to receive notice of cancellation.
        /// </param>
        /// <returns>The task object representing the asynchronous operation.</returns>
+        [Obsolete("Use UploadWithResponseAsync instead which allows you to upload files and also returns response metadata.")]


The Obsolete attribute should include the error: false parameter to maintain backward compatibility without breaking builds. Consider: [Obsolete(\"Use UploadWithResponseAsync...\", error: false)]

not sure if this is necessary because its a build time error. and i see we mark things with obsolete in other places without using error:false. is this needed/

actually i guess i may have to do error:false because i change https://github.com/aws/aws-sdk-net/pull/4062/files#r2461277562 back to uploadasync then it needs to be able to use it

sdk/src/Services/S3/Custom/Transfer/TransferUtilityUploadDirectoryResponse.cs

sdk/src/Services/S3/Custom/Transfer/TransferUtilityAbortMultipartUploadsResponse.cs

Copilot · 2025-10-24T00:40:39Z

sdk/src/Services/S3/Custom/Transfer/Internal/_async/OpenStreamCommand.async.cs

+            // TODO map and return response
+            return new TransferUtilityOpenStreamResponse();


The TODO comment indicates incomplete implementation. This should be tracked in a follow-up issue or completed in this PR to ensure the OpenStreamCommand returns properly mapped response data.

will be populated when doing SDK-228248

Copilot · 2025-10-24T00:40:40Z

sdk/src/Services/S3/Custom/Transfer/Internal/_async/DownloadCommand.async.cs

+            // TODO map and return response
+            return new TransferUtilityDownloadResponse();


The TODO comment indicates incomplete implementation. This should be tracked in a follow-up issue or completed in this PR to ensure the DownloadCommand returns properly mapped response data.

will be popuated for SDK-228248

GarrettBeatty · 2025-10-24T14:43:58Z

sdk/src/Services/S3/Custom/_async/AmazonS3Client.Extensions.cs

-            };
-            InternalSDKUtils.ApplyValuesV2(request, additionalProperties);
-            return transfer.UploadAsync(request, cancellationToken);
+            return UploadObjectFromStreamInternalAsync(bucketName, objectKey, stream, additionalProperties, cancellationToken);


made a helper function here so that we could use the new uploadwithresponseasync api but then still have the same function signature of returning Task only (not Task<TransferUtilityUploadResponse>

GarrettBeatty · 2025-10-24T14:45:18Z

sdk/src/Services/S3/Custom/Transfer/Internal/_async/AbortMultipartUploadsCommand.async.cs

                await WhenAllOrFirstExceptionAsync(pendingTasks,cancellationToken)
                    .ConfigureAwait(continueOnCapturedContext: false);
+
+                return new TransferUtilityAbortMultipartUploadsResponse();


i dont think we plan on populating this at all FYI

GarrettBeatty · 2025-10-24T16:53:44Z

sdk/src/Services/S3/Custom/Transfer/_bcl+netstandard/TransferUtility.sync.cs

            try
            {
-                UploadAsync(filePath, bucketName).Wait();
+                UploadWithResponseAsync(filePath, bucketName).Wait();


actually i just realized this should say as UploadAsync because otherwise this would eventually create a span with UploadAsyncWithResponse instead of UploadAsync. will update that in next revision. same in other places below

stack-info: PR: #4062, branch: GarrettBeatty/stacked/4 add obsolete functions

GarrettBeatty added a commit that referenced this pull request Oct 18, 2025

uploadwithresponse api

21a5f4f

stack-info: PR: #4062, branch: GarrettBeatty/stacked/4

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from 30b6e85 to 21a5f4f Compare October 18, 2025 15:42

This was referenced Oct 18, 2025

Add Progress listeners for initiated, complete, and failed for simple upload #4059

Merged

Add mapping of CompletemultipartUploadResponse to TransferUtilityUploadResponse #4060

Merged

Add multipartupload lifecycle tracking #4061

Merged

GarrettBeatty requested a review from Copilot October 18, 2025 19:01

Copilot AI reviewed Oct 18, 2025

View reviewed changes

GarrettBeatty changed the base branch from GarrettBeatty/stacked/3 to gcbeatty/tuuploadresponse October 18, 2025 19:54

GarrettBeatty added a commit that referenced this pull request Oct 18, 2025

uploadwithresponse api

e27d144

stack-info: PR: #4062, branch: GarrettBeatty/stacked/4

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from 21a5f4f to e27d144 Compare October 18, 2025 19:54

GarrettBeatty changed the base branch from gcbeatty/tuuploadresponse to GarrettBeatty/stacked/3 October 18, 2025 19:54

GarrettBeatty changed the base branch from GarrettBeatty/stacked/3 to gcbeatty/tuuploadresponse October 19, 2025 18:13

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from e27d144 to c89de03 Compare October 19, 2025 18:13

GarrettBeatty added a commit that referenced this pull request Oct 19, 2025

uploadwithresponse api

c89de03

stack-info: PR: #4062, branch: GarrettBeatty/stacked/4

GarrettBeatty changed the base branch from gcbeatty/tuuploadresponse to GarrettBeatty/stacked/3 October 19, 2025 18:13

GarrettBeatty changed the base branch from GarrettBeatty/stacked/3 to gcbeatty/tuuploadresponse October 19, 2025 18:15

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from c89de03 to 8d603df Compare October 19, 2025 18:15

GarrettBeatty added a commit that referenced this pull request Oct 19, 2025

uploadwithresponse api

8d603df

stack-info: PR: #4062, branch: GarrettBeatty/stacked/4

GarrettBeatty changed the base branch from gcbeatty/tuuploadresponse to GarrettBeatty/stacked/3 October 19, 2025 18:15

GarrettBeatty changed the base branch from GarrettBeatty/stacked/3 to gcbeatty/tuuploadresponse October 20, 2025 13:38

GarrettBeatty added a commit that referenced this pull request Oct 20, 2025

uploadwithresponse api

622b802

stack-info: PR: #4062, branch: GarrettBeatty/stacked/4

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from 8d603df to 622b802 Compare October 20, 2025 13:38

GarrettBeatty changed the base branch from gcbeatty/tuuploadresponse to GarrettBeatty/stacked/3 October 20, 2025 13:38

GarrettBeatty requested a review from philasmar October 20, 2025 13:43

GarrettBeatty changed the base branch from GarrettBeatty/stacked/3 to feature/transfermanager October 20, 2025 14:06

GarrettBeatty added a commit that referenced this pull request Oct 20, 2025

uploadwithresponse api

bb1413b

stack-info: PR: #4062, branch: GarrettBeatty/stacked/4

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from 622b802 to bb1413b Compare October 20, 2025 14:06

GarrettBeatty changed the base branch from feature/transfermanager to GarrettBeatty/stacked/3 October 20, 2025 14:06

GarrettBeatty commented Oct 20, 2025

View reviewed changes

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from 8254f43 to 17f2d2b Compare October 23, 2025 17:52

GarrettBeatty changed the base branch from feature/transfermanager to GarrettBeatty/stacked/9 October 23, 2025 17:52

GarrettBeatty requested a review from Copilot October 23, 2025 18:23

Copilot AI reviewed Oct 23, 2025

View reviewed changes

GarrettBeatty changed the base branch from GarrettBeatty/stacked/9 to feature/transfermanager October 23, 2025 23:53

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from 17f2d2b to a3dfaec Compare October 23, 2025 23:53

GarrettBeatty changed the base branch from feature/transfermanager to GarrettBeatty/stacked/9 October 23, 2025 23:54

GarrettBeatty mentioned this pull request Oct 23, 2025

Add DownloadInitiated, Failed and Completed events #4079

Open

10 tasks

GarrettBeatty requested a review from Copilot October 24, 2025 00:40

Copilot AI reviewed Oct 24, 2025

View reviewed changes

GarrettBeatty changed the base branch from GarrettBeatty/stacked/9 to feature/transfermanager October 24, 2025 13:48

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from a3dfaec to d3860eb Compare October 24, 2025 13:48

GarrettBeatty changed the base branch from feature/transfermanager to GarrettBeatty/stacked/9 October 24, 2025 13:49

GarrettBeatty commented Oct 24, 2025

View reviewed changes

GarrettBeatty changed the base branch from GarrettBeatty/stacked/9 to feature/transfermanager October 24, 2025 14:57

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from d3860eb to 6d18ba4 Compare October 24, 2025 14:57

GarrettBeatty changed the base branch from feature/transfermanager to GarrettBeatty/stacked/9 October 24, 2025 14:57

GarrettBeatty changed the base branch from GarrettBeatty/stacked/9 to feature/transfermanager October 24, 2025 15:17

GarrettBeatty changed the base branch from feature/transfermanager to GarrettBeatty/stacked/9 October 24, 2025 15:18

GarrettBeatty changed the base branch from GarrettBeatty/stacked/9 to feature/transfermanager October 24, 2025 15:46

GarrettBeatty changed the base branch from feature/transfermanager to GarrettBeatty/stacked/9 October 24, 2025 15:46

GarrettBeatty changed the base branch from GarrettBeatty/stacked/9 to feature/transfermanager October 24, 2025 16:19

GarrettBeatty changed the base branch from feature/transfermanager to GarrettBeatty/stacked/9 October 24, 2025 16:19

GarrettBeatty marked this pull request as ready for review October 24, 2025 16:38

GarrettBeatty commented Oct 24, 2025

View reviewed changes

Add UploadWithResponseAsync API

73b483f

stack-info: PR: #4062, branch: GarrettBeatty/stacked/4 add obsolete functions

GarrettBeatty changed the base branch from GarrettBeatty/stacked/9 to feature/transfermanager October 24, 2025 22:24

GarrettBeatty force-pushed the GarrettBeatty/stacked/4 branch from 6d18ba4 to 73b483f Compare October 24, 2025 22:24

GarrettBeatty changed the base branch from feature/transfermanager to GarrettBeatty/stacked/9 October 24, 2025 22:24

		// TODO map and return response
		return new TransferUtilityOpenStreamResponse();

		// TODO map and return response
		return new TransferUtilityDownloadResponse();

Add UploadWithResponseAsync API #4062

Are you sure you want to change the base?

Add UploadWithResponseAsync API #4062

Uh oh!

Conversation

GarrettBeatty commented Oct 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How

Motivation and Context

Testing

Types of changes

Checklist

License

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

GarrettBeatty commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GarrettBeatty Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

GarrettBeatty commented Oct 18, 2025 •

edited

Loading

GarrettBeatty commented Oct 20, 2025 •

edited

Loading

GarrettBeatty Oct 24, 2025 •

edited

Loading

GarrettBeatty Oct 24, 2025 •

edited

Loading

GarrettBeatty Oct 24, 2025 •

edited

Loading