Support ScannedCount on DynamoDb Document Model #3751
Conversation
| @@ -341,6 +360,7 @@ internal List<Document> GetNextSetHelper() | |||
| } | |||
| } | |||
| NextKey = scanResult.LastEvaluatedKey; | |||
| scannedCount = scanResult.ScannedCount.GetValueOrDefault(); | |||
There was a problem hiding this comment.
My understanding from our conversation is the scanned count on Search would always be an accumulation of every GetNextSetHelper call. The GetRemainingHelper implementation is doing an accumulation where if I call get next once that sounds the scan count to 100 and then call get remaining scans lets say another 300 through multiple get next calls the end value of Scanned count is 400. But as I read this if I only call get next always resets the scanned count what was scanned in that one.
I think for consistency sake we should have scanned count be an accumulation and be sure to document that in the docs for ScannedCount.
There was a problem hiding this comment.
after the last discussion, I've checked the behavior on Java SDK in order to have a similar approach, and there is no accumulation done on paginated search for scannedCount . https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/enhanced/dynamodb/model/Page.html#scannedCount(). Also there is no similar method in Java with GetRemaining, and this is why on this method the scannedCount is accumulating the values.
| @@ -533,10 +560,12 @@ internal async Task<List<Document>> GetRemainingHelperAsync(CancellationToken ca | |||
|
|
|||
| while (!IsDone) | |||
| { | |||
| var previousScannedCount = scannedCount; | |||
There was a problem hiding this comment.
Once GetNextSetHelper is updated to always do += for scanned count which makes it do the accumulation then I think the get remaining won't need any logic because it will all happen in GetNextSetHelper.
| @@ -18,6 +18,7 @@ | |||
| #else | |||
| #error Unknown platform constant - unable to set correct AssemblyDescription | |||
| #endif | |||
| [assembly: InternalsVisibleTo("AWSSDK.UnitTests.DynamoDBv2.NetFramework, PublicKey=0024000004800000940000000602000000240000525341310004000001000100db5f59f098d27276c7833875a6263a3cc74ab17ba9a9df0b52aedbe7252745db7274d5271fd79c1f08f668ecfa8eaab5626fa76adc811d3c8fc55859b0d09d3bc0a84eecd0ba891f2b8a2fc55141cdcc37c2053d53491e650a479967c3622762977900eddbf1252ed08a2413f00a28f3a0752a81203f03ccb7f684db373518b4")] | |||
There was a problem hiding this comment.
FYI this file is generated, it'll be overwritten by the generator the next time it runs.
I do see why you had to add it since a lot of the Search related types are internal, but we may need a different approach here (e.g. would it be better to add an integration test instead? Or revisit the access modifiers to make them public?)
There was a problem hiding this comment.
@dscpinheiro in this case is there any issue if I update the AssemblyInfo template and add assembly: InternalsVisibleTo for unit tests projects which looks like respects naming convention AWSSDK.UnitTests.{AssemblyTitle}.NetFramework ?
There was a problem hiding this comment.
That could work, but I think you'd also need to update the test projects to be signed... I'm honestly not a fan of making internals visible; I know we did that for Core but I wish we didn't (which is why I mentioned maybe using an integration test instead if possible).
We can until Norm's back to get his opinion.
Expose
ScannedCounton Amazon.DynamoDBv2.DocumentModel.SearchDescription
Add a new
ScannedCountproperty to Search class.Populate this value internally from the most recent
ScanResponseorQueryResponse.Accumulate
ScannedCountvalue forGetRemaining/GetRemainingAsyncMotivation and Context
#2018
Testing
Unit tests added for new property added
Screenshots (if appropriate)
Types of changes
Checklist
License