You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 22e1e3f
Browse filesBrowse the repository at this point in the historyBrowse files
feat(extras): reject an oversized MicroVM input (#833)
microvm and vm.invoke now reject a job input whose JSON is over
MAX_JOB_INPUT_BYTES (6 MiB minus 8 KiB) with ValueError, before any
durable operation starts. The check measures the input with dumps, the
encoder of the job body. The fixed fields of the body take less than
3 KiB, so an accepted input always fits in the 6 MiB body.
The worker no longer handles Expect: 100-continue or reads and drops an
oversized body. It answers 400 without reading the body. Every SDK now
rejects an oversized input before it sends it: the JS SDK does so since
aws/aws-durable-execution-sdk-js#1088.
Copy file name to clipboardExpand all lines: packages/aws-durable-execution-sdk-python-extras/README.md
+6-5Lines changed: 6 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -123,7 +123,7 @@ except MicrovmError:
123
123
...# Any other MicroVM failure.
124
124
```
125
125
126
-
The class is the same on the first run and on every replay, so code that branches on it stays deterministic. A `microvm` call inside your own child context, or inside a `map` or `parallel` item, reaches the code around that context as the SDK's `ChildContextError`. Its `__cause__` is the MicroVM error, so check `isinstance(error.__cause__, MicrovmTimeoutError)` there. An invalid name or config raises the SDK's `ValidationError`, and an input that is not JSON-serializable raises `TypeError`, before any durable operation starts. `Duration` raises the same `ValidationError` for a negative value. The operation does not check `image_identifier` and `execution_role_arn` before it starts. boto3 checks them against the API model before it sends `RunMicrovm`, and the service checks them again. So an invalid value fails the launch step at once with `MicrovmLaunchError`, after the callback step.
126
+
The class is the same on the first run and on every replay, so code that branches on it stays deterministic. A `microvm` call inside your own child context, or inside a `map` or `parallel` item, reaches the code around that context as the SDK's `ChildContextError`. Its `__cause__` is the MicroVM error, so check `isinstance(error.__cause__, MicrovmTimeoutError)` there. The operation raises three errors before any durable operation starts. An invalid name or config raises the SDK's `ValidationError`. An input that is not JSON-serializable raises `TypeError`. An input over `MAX_JOB_INPUT_BYTES` raises `ValueError`. `Duration` raises the same `ValidationError` for a negative value. The operation does not check `image_identifier` and `execution_role_arn` before it starts. boto3 checks them against the API model before it sends `RunMicrovm`, and the service checks them again. So an invalid value fails the launch step at once with `MicrovmLaunchError`, after the callback step.
127
127
128
128
A failed AWS call keeps its request ID in the error message, in the form `<Operation> request ID: <id>.`, for example `RunMicrovm request ID: <id>.`. A support case can quote it. This applies to `RunMicrovm`, `GetMicrovm`, `ResumeMicrovm`, `CreateMicrovmAuthToken`, and `TerminateMicrovm`. The SDK checkpoints the message of a failed step, and not its `__cause__`. So the request ID is in the message, the handler sees it on the first run and on replay, and the execution history keeps it. A failed `TerminateMicrovm` call does not fail the operation. So a `TerminateMicrovm` request ID appears only in the terminate log entry and in the terminate step record. A message has no request ID when the answer carries none, for example after a connection failure. A message also has no request ID when the error is not the answer of the Lambda MicroVMs call. For example, a `client` with role-assuming credentials calls STS `AssumeRole` inside each call, and an STS error carries the request ID of STS. The operation recognizes such an error by its `operation_name`. The step raises a copy of the botocore error, of the same class, with the same `response`, and with the request ID in its message. So `default_microvm_retry_strategy` decides as before. A custom `retry_strategy` receives the error that botocore raised, without the request ID. So a strategy that matches the message, such as `create_retry_strategy(RetryStrategyConfig(retryable_errors=[...]))`, decides on the message of the service. A `ResourceNotFoundException` from `GetMicrovm`, `ResumeMicrovm`, or `CreateMicrovmAuthToken` is the exception. It means that the MicroVM no longer exists. So the operation raises `MicrovmStateError` instead, and the strategy receives that error. Its `__cause__` is the botocore error. An error that the operation creates itself, such as `MicrovmStateError` or "RunMicrovm returned no microvmId", has the request ID in its message when the strategy receives it. A custom client can raise one error object for several calls. Each step message then names only its own call, and the object keeps its message.
129
129
@@ -146,6 +146,7 @@ The 10-second reserve keeps time for the step to record its failure. A Lambda ti
146
146
147
147
-`timeout` must be between 1 second and 8 hours, because a MicroVM lives at most 8 hours. The MicroVM's `maximumDurationInSeconds` is `timeout` plus 5 minutes, at most 8 hours. So a timeout above 7 hours 55 minutes leaves the MicroVM less than 5 minutes beyond the callback's timeout.
148
148
-`job_input` is sent as JSON. So the MicroVM receives what `json.dumps` writes: a tuple arrives as a list, and a dict key that is not a string arrives as a string.
149
+
- The worker accepts a job request body of at most 6 MiB. The body holds the JSON of `job_input` and a few job fields. The job fields take less than 3 KiB. So `job_input` can be at most `MAX_JOB_INPUT_BYTES`, 6 MiB minus 8 KiB, as UTF-8 JSON. A larger input raises `ValueError` from `microvm` or `vm.invoke` before any durable operation starts.
149
150
- The MicroVM gets the `INTERNET_EGRESS` connector, because it needs egress to reach the Lambda callback APIs. For ingress, it gets `NO_INGRESS` with `run` hook delivery and `ALL_INGRESS` with HTTP delivery. If you set `ingress_network_connectors`, allow ingress for every job that goes over HTTP. A `microvm` job goes over HTTP when its `run` hook payload is over 4,096 code points, or when its config sets `request.path`. Every session job goes over HTTP. An empty tuple sends no connector.
150
151
-`heartbeat_timeout` starts when the callback is created, before the launch. The worker sends its first heartbeat when the job arrives. So the first period must cover the launch, the boot, and, for HTTP delivery, the request step, each with its retries. A throttled launch can take several minutes, and the request step retries for up to `retry_window` in each attempt. With a shorter heartbeat timeout, the operation fails with `MicrovmTimeoutError` before the MicroVM sends its first heartbeat.
151
152
- The request step needs at least 11 seconds left in the invocation when it starts. It keeps a 10-second reserve before the invocation times out, and it starts a call only with at least 1 second left before the reserve. The launch step can run earlier in the same invocation, so the function timeout must cover it too. A function timeout of `retry_window` plus 11 seconds, 71 seconds by default, leaves the first retry tier about its whole window. With less than 11 seconds left, every attempt fails with `MicrovmEndpointUnavailableError`, and the message of the final `MicrovmDeliveryError` names the function timeout. The Lambda default timeout is 3 seconds. With it, `microvm` runs small jobs in the `run` hook and fails every job that goes over HTTP: a job too large for the hook, and a job with `request.path`.
@@ -240,7 +241,7 @@ A failed `vm.invoke` raises its MicroVM error inside the handler. The handler ca
240
241
1. A MicroVM error that the handler raises itself reaches the caller with its own class, such as `MicrovmJobFailedError`.
241
242
2. Any other error reaches the caller as the SDK's `ChildContextError`, as from `run_in_child_context`. This includes a MicroVM error that leaves a child context that the handler created: that child context records `ChildContextError` as its error type. The caller then gets the MicroVM error's message, but not its class. Inside the handler, the `ChildContextError` of that child context has the MicroVM error as its `__cause__`. An SDK error that leaves the handler, such as the `StepError` of a failed step, also reaches the caller as `ChildContextError`, with the SDK error as its `__cause__`. This rule matches `run_in_child_context`. The JavaScript operation differs here: it passes an SDK error to the caller with its own class.
242
243
243
-
An invalid job raises `ValidationError`or `TypeError` from `vm.invoke`, before the job's durable operations start. The handler can catch it too. A call without the context, `vm.invoke("build", job_input=..., options=...)`, raises Python's `TypeError`. A first argument that is not a durable context, as in `vm.invoke("build", "build", ...)`, raises `AttributeError`.
244
+
An invalid job raises `ValidationError`, `TypeError`, or `ValueError` from `vm.invoke`, before the job's durable operations start. The handler can catch it too. A call without the context, `vm.invoke("build", job_input=..., options=...)`, raises Python's `TypeError`. A first argument that is not a durable context, as in `vm.invoke("build", "build", ...)`, raises `AttributeError`.
244
245
245
246
The handler's return value is checkpointed as the session's result. So it must be serializable by the SDK, and it counts toward the checkpoint size limit.
246
247
@@ -299,9 +300,9 @@ The MicroVM operations handle the network, the service, and the worker lifecycle
299
300
### Job inputs
300
301
301
302
1. Pass a JSON-serializable input with finite numbers. Any other input raises `TypeError` before any durable operation starts. See [Errors](#errors).
302
-
2. Keep the serialized job request at or under 6 MiB. The job request holds the input and a few job fields. Pass larger data by reference, for example as an S3 key.
303
-
- The Python worker reads a job request body of at most 6 MiB. It answers a larger body with HTTP 400.
304
-
-The operation does not check the size before it sends the job. So a larger body reaches the worker, and the job fails with `MicrovmDeliveryError`. It fails at once when the 400 reaches the operation. It fails only after the retries of both tiers when the connection is reset instead, and the MicroVM is billed meanwhile. [What the worker does](../aws-durable-execution-sdk-python-microvm-worker/README.md#what-the-worker-does) says when each case happens.
303
+
2. Keep the job input's JSON at or under `MAX_JOB_INPUT_BYTES`, 6 MiB minus 8 KiB, in UTF-8. Pass larger data by reference, for example as an S3 key.
304
+
- The Python worker reads a job request body of at most 6 MiB. The body holds the input's JSON and a few job fields.
305
+
-`microvm` and `vm.invoke` measure the input's JSON before any durable operation starts. A larger input raises `ValueError`. So `microvm` launches no MicroVM for it.
305
306
3. Expect a job to go over HTTP in three cases:
306
307
- The `run` hook payload of a `microvm` job is over 4,096 code points. The payload holds the input, the callback ID, and a few other fields.
307
308
- The `microvm` config sets `request.path`. The job then goes over HTTP for any input size.
Copy file name to clipboardExpand all lines: packages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/microvm/__init__.py
Copy file name to clipboardExpand all lines: packages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/microvm/job.py
+21Lines changed: 21 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -29,6 +29,27 @@
29
29
counts what the service counts.
30
30
"""
31
31
32
+
MAX_JOB_INPUT_BYTES=6*1024*1024-8*1024
33
+
"""The largest job input, in UTF-8 bytes of its JSON, that ``microvm`` and
34
+
``vm.invoke`` accept.
35
+
36
+
1. The worker accepts a job request body of at most 6 MiB.
37
+
2. The body holds the input as the same JSON that :func:`dumps` writes.
38
+
3. The body also holds fixed fields: the version, the Region, the MicroVM
39
+
ID, the callback ID, and the heartbeat timeout. Their names and the JSON
40
+
punctuation take 92 bytes.
41
+
4. The botocore ``lambda-microvms`` model allows a MicroVM ID of 1 to 256
42
+
characters, with no pattern. JSON writes a character in at most 6 bytes,
43
+
as a ``\\u00XX`` escape. So the MicroVM ID takes at most 1,536 bytes.
44
+
5. The Lambda model allows a callback ID of at most 1,024 characters of
45
+
``[A-Za-z0-9+/]+={0,2}``. JSON writes each of these characters in 1 byte.
46
+
So the callback ID takes at most 1,024 bytes.
47
+
6. The Region is short ASCII. The heartbeat timeout is a few digits.
48
+
7. So the fixed fields take less than 3 KiB. So 8 KiB covers them.
49
+
50
+
So an input whose JSON has at most 6 MiB minus 8 KiB fits in the body.
Copy file name to clipboardExpand all lines: packages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/microvm/operation.py
Copy file name to clipboardExpand all lines: packages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/microvm/session.py
+2Lines changed: 2 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -250,6 +250,8 @@ def invoke(
250
250
AttributeError: When ``context`` is not a durable context, for
251
251
example a job name passed as the first argument. Python
252
252
raises it at ``context.run_in_child_context``.
253
+
ValueError: When the JSON of ``job_input`` is over
254
+
``MAX_JOB_INPUT_BYTES`` UTF-8 bytes.
253
255
ValidationError: When ``name`` or ``options`` is not valid.
254
256
MicrovmNotRunningError: When the session's MicroVM has ended. The
255
257
session reports it when it checks the MicroVM state before
0 commit comments