From a4c87ebf988bfa34082b5f61c34a3e94b1725a71 Mon Sep 17 00:00:00 2001 From: Valentin Kovalenko Date: Tue, 5 Nov 2024 15:56:44 -0700 Subject: [PATCH 1/6] Update the requirements regarding the size validation --- source/crud/bulk-write.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/source/crud/bulk-write.md b/source/crud/bulk-write.md index 6125254370..088696b122 100644 --- a/source/crud/bulk-write.md +++ b/source/crud/bulk-write.md @@ -638,15 +638,16 @@ write concern containing the following message: The server reports a `maxBsonObjectSize` in its `hello` response. This value defines the maximum size for documents that are inserted into the database. Documents that are sent to the server but are not intended to be inserted into the -database (e.g. command documents) have a size limit of `maxBsonObjectSize + 16KiB`. When an acknowledged write concern -is used, drivers MUST NOT perform any checks related to these size limits and MUST rely on the server to raise an error -if a limit is exceeded. However, when an unacknowledged write concern is used, drivers MUST raise an error if one of the -following limits is exceeded: +database (e.g. command documents) have a size limit of `maxBsonObjectSize + 16KiB`. The following table specifies the +size limits and the client behavior with regard to validating sizes depending on whether a server acknowledgement +is expected: -- The size of a document to be inserted MUST NOT exceed `maxBsonObjectSize`. This applies to the `document` field of an - `InsertOneModel` and the `replacement` field of a `ReplaceOneModel`. -- The size of an entry in the `ops` array MUST NOT exceed `maxBsonObjectSize + 16KiB`. -- The size of the `bulkWrite` command document MUST NOT exceed `maxBsonObjectSize + 16KiB`. +| Size limit | Acknowledgement expected | Acknowledgement not expected | +|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------| +| The size of a document to be inserted MUST NOT exceed `maxBsonObjectSize`. This applies to the `document` field of an `InsertOneModel` and the `replacement` field of a `ReplaceOneModel`. | MUST NOT validate, MUST rely on the server validation. This is to allow the server to proceed with the rest of the individual operations even if some of them violate the limit. | MUST validate. | +| The size of an entry in the `ops` array MUST NOT exceed `maxBsonObjectSize + 16KiB`. | MUST NOT validate, MUST rely on the server validation. This is to allow the server to proceed with the rest of the individual operations even if some of them violate the limit. | MUST validate. | +| The size of an entry in the `nsInfo` array MUST NOT exceed `maxBsonObjectSize + 16KiB`. | MAY rely on the server validation. | MUST validate. | +| The size of the `bulkWrite` command document MUST NOT exceed `maxBsonObjectSize + 16KiB`. | MAY rely on the server validation. | MUST validate. | See [SERVER-10643](https://jira.mongodb.org/browse/SERVER-10643) for more details on these size limits. @@ -924,6 +925,8 @@ batch-splitting to standardize implementations across drivers and simplify batch ## **Changelog** +- 2024-11-05: Updated the requirements regarding the size validation. + - 2024-10-07: Error if `w:0` is used with `ordered=true` or `verboseResults=true`. - 2024-10-01: Add sort option to `replaceOne` and `updateOne`. From 27d258eb13d8bb67563fbd5ecaa9756b2b680214 Mon Sep 17 00:00:00 2001 From: Valentin Kovalenko Date: Tue, 5 Nov 2024 18:00:47 -0700 Subject: [PATCH 2/6] Forbid size validation regardless of whether there is a server acknowledgement --- source/crud/bulk-write.md | 14 ++++------- source/crud/tests/README.md | 47 +++---------------------------------- 2 files changed, 7 insertions(+), 54 deletions(-) diff --git a/source/crud/bulk-write.md b/source/crud/bulk-write.md index 088696b122..d3734b1175 100644 --- a/source/crud/bulk-write.md +++ b/source/crud/bulk-write.md @@ -638,16 +638,10 @@ write concern containing the following message: The server reports a `maxBsonObjectSize` in its `hello` response. This value defines the maximum size for documents that are inserted into the database. Documents that are sent to the server but are not intended to be inserted into the -database (e.g. command documents) have a size limit of `maxBsonObjectSize + 16KiB`. The following table specifies the -size limits and the client behavior with regard to validating sizes depending on whether a server acknowledgement -is expected: - -| Size limit | Acknowledgement expected | Acknowledgement not expected | -|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------| -| The size of a document to be inserted MUST NOT exceed `maxBsonObjectSize`. This applies to the `document` field of an `InsertOneModel` and the `replacement` field of a `ReplaceOneModel`. | MUST NOT validate, MUST rely on the server validation. This is to allow the server to proceed with the rest of the individual operations even if some of them violate the limit. | MUST validate. | -| The size of an entry in the `ops` array MUST NOT exceed `maxBsonObjectSize + 16KiB`. | MUST NOT validate, MUST rely on the server validation. This is to allow the server to proceed with the rest of the individual operations even if some of them violate the limit. | MUST validate. | -| The size of an entry in the `nsInfo` array MUST NOT exceed `maxBsonObjectSize + 16KiB`. | MAY rely on the server validation. | MUST validate. | -| The size of the `bulkWrite` command document MUST NOT exceed `maxBsonObjectSize + 16KiB`. | MAY rely on the server validation. | MUST validate. | +database (e.g. command documents) have a size limit of `maxBsonObjectSize + 16KiB`. Clients MUST NOT validate the size +of BSON documents against these limits, and MUST rely on server validation, following the +["Where possible, depend on server to return errors"](https://github.com/mongodb/specifications/blob/f8dbd2469f18d093f917efa1f758024bca5d3aaa/source/driver-mantras.md#where-possible-depend-on-server-to-return-errors) +mantra. For simplicity, this requirement remains in force even when a server acknowledgement is not expected. See [SERVER-10643](https://jira.mongodb.org/browse/SERVER-10643) for more details on these size limits. diff --git a/source/crud/tests/README.md b/source/crud/tests/README.md index b169859c23..5983269683 100644 --- a/source/crud/tests/README.md +++ b/source/crud/tests/README.md @@ -370,56 +370,15 @@ Assert that a CommandStartedEvent was observed for the `killCursors` command. ### 10. `MongoClient.bulkWrite` returns error for unacknowledged too-large insert -This test must only be run on 8.0+ servers. This test must be skipped on Atlas Serverless. - -Construct a `MongoClient` (referred to as `client`). - -Perform a `hello` command using `client` and record the following values from the response: `maxBsonObjectSize`. - -Then, construct the following document (referred to as `document`): - -```javascript -{ - "a": "b".repeat(maxBsonObjectSize) -} -``` +Removed. #### With insert -Construct the following write model (referred to as `model`): - -```javascript -InsertOne: { - "namespace": "db.coll", - "document": document -} -``` - -Construct as list of write models (referred to as `models`) with the one `model`. - -Call `MongoClient.bulkWrite` with `models`. Pass `BulkWriteOptions` with `ordered` set to `false` and `writeConcern` set -to an unacknowledged write concern. - -Expect a client-side error due the size. +Removed. #### With replace -Construct the following write model (referred to as `model`): - -```javascript -ReplaceOne: { - "namespace": "db.coll", - "filter": {}, - "replacement": document -} -``` - -Construct as list of write models (referred to as `models`) with the one `model`. - -Call `MongoClient.bulkWrite` with `models`. Pass `BulkWriteOptions` with `ordered` set to `false` and `writeConcern` set -to an unacknowledged write concern. - -Expect a client-side error due the size. +Removed. ### 11. `MongoClient.bulkWrite` batch splits when the addition of a new namespace exceeds the maximum message size From 73ecd7c9d453bb2a479a8078ced66e4350c45cee Mon Sep 17 00:00:00 2001 From: Valentin Kovalenko Date: Tue, 5 Nov 2024 19:48:27 -0700 Subject: [PATCH 3/6] Simplify the description of BSON size limits --- source/crud/bulk-write.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/source/crud/bulk-write.md b/source/crud/bulk-write.md index d3734b1175..3219869746 100644 --- a/source/crud/bulk-write.md +++ b/source/crud/bulk-write.md @@ -634,12 +634,10 @@ write concern containing the following message: > Cannot request unacknowledged write concern and ordered writes -### Size Limits +### BSON Document Size Limit -The server reports a `maxBsonObjectSize` in its `hello` response. This value defines the maximum size for documents that -are inserted into the database. Documents that are sent to the server but are not intended to be inserted into the -database (e.g. command documents) have a size limit of `maxBsonObjectSize + 16KiB`. Clients MUST NOT validate the size -of BSON documents against these limits, and MUST rely on server validation, following the +The server limits the size of a BSON document. Clients MUST NOT validate the size +of a BSON document, and MUST rely on server validation, following the ["Where possible, depend on server to return errors"](https://github.com/mongodb/specifications/blob/f8dbd2469f18d093f917efa1f758024bca5d3aaa/source/driver-mantras.md#where-possible-depend-on-server-to-return-errors) mantra. For simplicity, this requirement remains in force even when a server acknowledgement is not expected. From 125d1921cda0acefb96f8160b42499037515e674 Mon Sep 17 00:00:00 2001 From: Valentin Kovalenko Date: Wed, 6 Nov 2024 16:45:33 -0700 Subject: [PATCH 4/6] Remove the section about the size validation requirements, and add a new entry to the Q&A section --- source/crud/bulk-write.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/source/crud/bulk-write.md b/source/crud/bulk-write.md index 3219869746..569e9a271f 100644 --- a/source/crud/bulk-write.md +++ b/source/crud/bulk-write.md @@ -634,15 +634,6 @@ write concern containing the following message: > Cannot request unacknowledged write concern and ordered writes -### BSON Document Size Limit - -The server limits the size of a BSON document. Clients MUST NOT validate the size -of a BSON document, and MUST rely on server validation, following the -["Where possible, depend on server to return errors"](https://github.com/mongodb/specifications/blob/f8dbd2469f18d093f917efa1f758024bca5d3aaa/source/driver-mantras.md#where-possible-depend-on-server-to-return-errors) -mantra. For simplicity, this requirement remains in force even when a server acknowledgement is not expected. - -See [SERVER-10643](https://jira.mongodb.org/browse/SERVER-10643) for more details on these size limits. - ## Auto-Encryption If `MongoClient.bulkWrite` is called on a `MongoClient` configured with `AutoEncryptionOpts`, drivers MUST return an @@ -915,6 +906,15 @@ number was determined by constructing `OP_MSG` messages with various fields atta Drivers are required to use this value even if they are capable of determining the exact size of the message prior to batch-splitting to standardize implementations across drivers and simplify batch-splitting testing. +### Why is there no requirement to validate the size of a BSON document? + +In accordance with the +["_Where possible, depend on server to return errors_"](https://github.com/mongodb/specifications/blob/f8dbd2469f18d093f917efa1f758024bca5d3aaa/source/driver-mantras.md#where-possible-depend-on-server-to-return-errors) +mantra, drivers should rely on the server validating the size. Such reliance is not possible when a server +acknowledgement is not expected, but for simplicity and given that unacknowledged writes does not provide a way to know +whether a write succeeded on the server, it does not seem necessary to help an application with that info +in the specific situation when the BSON document size limit is exceeded. + ## **Changelog** - 2024-11-05: Updated the requirements regarding the size validation. From 1bf26f275a4cbab86521cb40285d17f8209b99ee Mon Sep 17 00:00:00 2001 From: Valentin Kovalenko Date: Thu, 7 Nov 2024 08:04:09 -0700 Subject: [PATCH 5/6] Fix the mdformat pre-commit check --- source/crud/bulk-write.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/crud/bulk-write.md b/source/crud/bulk-write.md index 569e9a271f..872423271b 100644 --- a/source/crud/bulk-write.md +++ b/source/crud/bulk-write.md @@ -912,8 +912,8 @@ In accordance with the ["_Where possible, depend on server to return errors_"](https://github.com/mongodb/specifications/blob/f8dbd2469f18d093f917efa1f758024bca5d3aaa/source/driver-mantras.md#where-possible-depend-on-server-to-return-errors) mantra, drivers should rely on the server validating the size. Such reliance is not possible when a server acknowledgement is not expected, but for simplicity and given that unacknowledged writes does not provide a way to know -whether a write succeeded on the server, it does not seem necessary to help an application with that info -in the specific situation when the BSON document size limit is exceeded. +whether a write succeeded on the server, it does not seem necessary to help an application with that info in the +specific situation when the BSON document size limit is exceeded. ## **Changelog** From f90c80762b3a9a9ea4b98b36275b5d8f7f29147e Mon Sep 17 00:00:00 2001 From: Valentin Kovalenko Date: Thu, 7 Nov 2024 11:09:12 -0700 Subject: [PATCH 6/6] Mention that the size validation was requires previously --- source/crud/bulk-write.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/source/crud/bulk-write.md b/source/crud/bulk-write.md index 872423271b..285a5bb338 100644 --- a/source/crud/bulk-write.md +++ b/source/crud/bulk-write.md @@ -908,12 +908,12 @@ batch-splitting to standardize implementations across drivers and simplify batch ### Why is there no requirement to validate the size of a BSON document? -In accordance with the -["_Where possible, depend on server to return errors_"](https://github.com/mongodb/specifications/blob/f8dbd2469f18d093f917efa1f758024bca5d3aaa/source/driver-mantras.md#where-possible-depend-on-server-to-return-errors) -mantra, drivers should rely on the server validating the size. Such reliance is not possible when a server -acknowledgement is not expected, but for simplicity and given that unacknowledged writes does not provide a way to know -whether a write succeeded on the server, it does not seem necessary to help an application with that info in the -specific situation when the BSON document size limit is exceeded. +Following +["_Where possible, depend on server to return errors_"](https://github.com/mongodb/specifications/blob/f8dbd2469f18d093f917efa1f758024bca5d3aaa/source/driver-mantras.md#where-possible-depend-on-server-to-return-errors), +drivers should rely on the server to return errors about exceeded size limits. Such reliance is not possible for +unacknowledged writes. This specification previously required drivers to check size limits for unacknowledged writes. +The requirement has since been removed. Checking size limits complicates some driver implementations. Returning a driver +error in this specific situation does not seem helpful enough to require size checks. ## **Changelog**