Skip to content

Commit 334150b

Browse files
committed
Merge branch 'master' into claude/beautiful-noether-innrky
Two conflicts in requirements/api.md: - The run-start input validation #72 documents sits outside the section this branch deletes, so it is kept as #72 wrote it, above this branch's "Five endpoints are exceptions to the {data} envelope" line. - The Actor-runtime section: kept this branch's, as before. The OpenAPI document is normative for the upstream-fallback contract, and #72 extends the never-relayed error types, so setApiFallbackState's exhaustive list now names invalid-input and invalid-input-schema too. No conflict flagged that - #72's own edit to the same list lands inside the prose this branch deletes - and the list claims to be exhaustive, so leaving them out would have made the specification wrong. Nothing else in #72 reaches the namespace: input validation is faithful to the platform, message for message, so it warrants no platform note. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016VvUV6661cyYbxoHZv2Pby
2 parents 57f3938 + d0937e0 commit 334150b

43 files changed

Lines changed: 1890 additions & 169 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,23 @@ In a build or run log, everything the runtime itself has to say - dev-folder not
5555
attach line, the browser-view URL, migration markers, a run that could not be started - opens with a
5656
blue `[actor-runtime]` prefix. Your Actor's own output is passed through byte for byte.
5757

58+
## Input schema: defaults and validation
59+
60+
An Actor that declares an input schema (the `input` field of `.actor/actor.json`, `.actor/INPUT_SCHEMA.json`,
61+
or `INPUT_SCHEMA.json` at its root) gets the platform's behaviour locally: the schema's defaults are filled into
62+
every run's input, and an input the schema rejects fails the call with the API's own message instead of starting a
63+
container.
64+
65+
```bash
66+
apify call # runs on the schema's defaults
67+
apify call --input '{"maxPages":0}' # 400 Input is not valid: Field input.maxPages must be >= 1
68+
```
69+
70+
The schema is read at build time, so editing it locally needs an `apify push` even under a registered dev folder,
71+
and a schema the Apify meta-schema rejects fails the build with the defect in its log. Proxy group availability is
72+
not checked locally, and encrypted secret input fields stay unsupported - see
73+
`requirements/actor-driver.md`'s "Input schema, validation and defaults".
74+
5875
## Running with Podman instead of Docker
5976

6077
The runtime talks to the container engine only through its Docker-compatible API socket, and Podman
@@ -236,7 +253,9 @@ do. Like Python debug mode, this needs the runtime to run from its own built ima
236253

237254
Both bundled samples charge two events when their Actor is priced - `page-scraped` once per page and
238255
`crawl-finished` once at the end - and ship the pricing that defines them in `pricing.json`, so a run
239-
charges for real right after a push:
256+
charges for real right after a push. That pricing also declares both synthetic events, which the Actor
257+
never charges itself: `apify-actor-start` at run start (once per whole GB of the run's memory) and
258+
`apify-default-dataset-item` per item pushed to the default dataset.
240259

241260
```bash
242261
cd sample_actor_ts # or sample_actor_py
@@ -254,6 +273,19 @@ run's console page and the runs list show the same figures.
254273
platform: an update sends the Actor's existing entries unchanged plus at most one new one, starting after
255274
all of them. Making the Actor free again is therefore appending a `{"pricingModel": "FREE"}` entry.
256275

276+
The `PUT` above therefore prices an Actor that has no pricing yet. Once it has one, a second `PUT` of the
277+
same file is refused (`pricingInfos[0] differs from the Actor's existing pricing info`) - the stored
278+
entries carry the timestamps they were given, which the file does not. Append the file's entry to what
279+
the Actor already has instead:
280+
281+
```bash
282+
apify api PUT /v2/actors/<actorId> --body "$(apify api GET /v2/actors/<actorId> |
283+
jq --argjson new "$(jq '.pricingInfos[-1]' pricing.json)" '{pricingInfos: (.data.pricingInfos + [$new])}')"
284+
```
285+
286+
The Actor's console page has the same thing as a form: the box holds the stored array, and adding an
287+
entry below the existing ones does it without the shell.
288+
257289
Cap a run's spend the way a user does - the cap is a query parameter, with no `apify call` flag for it:
258290

259291
```bash

‎package.json‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,11 @@
2626
"test:watch": "vitest"
2727
},
2828
"dependencies": {
29+
"@apify/input_schema": "^3.29.2",
2930
"@crawlee/core": "4.0.0-beta.145",
3031
"@crawlee/fs-storage": "4.0.0-beta.145",
3132
"@novnc/novnc": "1.7.0",
33+
"ajv": "^8.20.0",
3234
"dockerode": "^4.0.5",
3335
"express": "^5.1.0",
3436
"json5": "^2.2.3",

‎pnpm-lock.yaml‎

Lines changed: 97 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎requirements/actor-driver.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,20 @@
3434
4. the platform's bundled default Dockerfile, for that build only - the pushed source itself is unchanged.
3535
- Matching is case-insensitive, exact-case wins ties, and every outcome is stated in the build log.
3636

37+
# Input schema, validation and defaults
38+
39+
- **Input schemas work as on the Apify platform**: the schema is read from the pushed source when the
40+
Actor is built - the `input` field of `.actor/actor.json`, else `.actor/INPUT_SCHEMA.json`, else
41+
`INPUT_SCHEMA.json` - a build whose schema cannot be read or is not a valid input schema fails with
42+
the reason in its log, and every run of a build is validated against that build's schema with its
43+
defaults applied, a rejected input starting nothing (`api.md`). A build with no input schema takes
44+
every input exactly as the caller sent it.
45+
- **Differences**: Apify Proxy group availability is not checked, so any `apifyProxyGroups` selection
46+
is accepted, while the rest of a `proxy` field is still validated; encrypted secret input fields
47+
stay unsupported (`unsupported.md`).
48+
- An Actor running from a registered dev folder uses its last build's schema: unlike a source edit,
49+
an edited input schema takes effect only after `apify push`.
50+
3751
# Bind mount volumes with Actor source code
3852

3953
- To let an Actor be re-run with source changes and no rebuild, the Actor's registered local dev
@@ -215,6 +229,8 @@ start`, ...) is refused by name, naming both the `CMD` fix and how to clear debu
215229
per subscription tier is charged at the lowest paid tier; nothing is ever billed or paid out; and the
216230
rules tying a price change to payout details, notice periods and subscription tiers do not apply.
217231
- An Actor's own charging code therefore runs here unchanged, with no local-testing switch.
232+
- A run's log states what it was pre-charged for starting, so the count is visible where the platform
233+
leaves it to be discovered on the bill.
218234
- The pricing can also be set from the console (`console.md`).
219235

220236
# Users

‎requirements/api.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,12 @@
1717
- `DELETE /v2/actor-builds/:buildId` and `DELETE /v2/actor-runs/:runId` on a **non-terminal** build/run
1818
are rejected, not aborted-then-deleted: `400` with error type `deleting-unfinished-build` (builds) or
1919
`cannot-remove-running-run` (runs), matching the Apify platform.
20+
- `POST /v2/actors/:actorId/runs` validates the input against the input schema of the build it
21+
resolved, when that build has one (`actor-driver.md`), and starts nothing when it does not pass:
22+
`400` `invalid-input` for a body that is not `application/json`, is not parseable JSON, is not a
23+
JSON object, or that the schema rejects, naming every offending field; `400` `invalid-input-schema`
24+
when the Actor's own schema is not valid. Both messages match the Apify platform's. A build with no
25+
input schema accepts any body, unvalidated.
2026
- Five endpoints are exceptions to the `{data}` envelope:
2127
- `GET /v2/logs/:buildOrRunId` (and its `actor-builds`/`actor-runs` aliases): the body is plain text,
2228
never `{data}`-wrapped, matching apify-client-js's `log().get()`.

‎requirements/test.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ Test case must verify full Actor development flow:
3636
- Push and build Actor in local actor runtime `apify push`
3737
- Run each sample Actor in the local actor runtime with `apify call --input '{"maxPages":N}'` for at least two different values of `N`, waiting for each run to finish
3838
- Assert via `apify datasets info <default dataset id>` that the default dataset's `itemCount` tracks `N` - the assertion is input-dependent, not just "some items exist"
39+
- Cover the input schema through the CLI too (`actor-driver.md`): `apify call` with no `--input` must run on the schema's defaults, asserted from the run's own `INPUT`, and `apify call` with an input the schema rejects must fail, naming the offending field
3940

4041
## Browser view
4142

‎requirements/unsupported.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,6 @@ real account, not in the runtime.
2424
- Ad-hoc webhooks on run start
2525
- Metered usage other than compute units: storage operations, data transfer and proxy
2626
- Billing: a run's charges and costs are reported, never invoiced or paid out
27-
- Input validation and defaults from the input schema
2827
- Encrypted secret input fields
2928
- Actor-level default run options
3029
- Dynamic and bounded memory from `.actor/actor.json`
@@ -101,7 +100,8 @@ real account, not in the runtime.
101100

102101
## Platform limits not enforced
103102

104-
- Memory steps and bounds (128 MB - 32 GB, powers of two)
103+
- Memory steps and bounds (128 MB - 32 GB, powers of two); a run asking for anything else is warned
104+
about in its log and started with it anyway
105105
- Record, item and input size limits
106106
- Concurrent run, rate and per-account quotas
107107
- Process, file-descriptor and shared-memory limits

‎sample_actor_crawler/src/main.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,13 @@ async def request_handler(context: ParselCrawlingContext) -> None:
2424

2525
async def main() -> None:
2626
async with Actor:
27+
# Both fields have a `default` in the input schema, so the runtime fills them in before the
28+
# run starts (the Apify platform does the same) - the Actor needs no fallback of its own.
2729
actor_input = await Actor.get_input() or {}
28-
start_url = actor_input.get("startUrl", "https://crawlee.dev")
30+
start_url = actor_input["startUrl"]
2931

3032
proxy_configuration = await Actor.create_proxy_configuration(
31-
actor_proxy_input=actor_input.get("proxyConfiguration")
33+
actor_proxy_input=actor_input["proxyConfiguration"]
3234
)
3335

3436
crawler = ParselCrawler(

‎sample_actor_nonstandard/app/main.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,9 @@ def main() -> None:
4646

4747
raw_input = api_request('GET', f'key-value-stores/{key_value_store_id}/records/INPUT')
4848
actor_input = json.loads(raw_input) if raw_input else {}
49-
item_count = int(actor_input.get('itemCount', 2))
49+
# `itemCount` has a `default` in the input schema, so the runtime fills it in before the run
50+
# starts - no fallback needed here.
51+
item_count = int(actor_input['itemCount'])
5052
print(f'main.py: pushing {item_count} item(s) to dataset {dataset_id}.')
5153

5254
# Both accepted body shapes of `POST /v2/datasets/:id/items`: one object, then an array.

0 commit comments

Comments
 (0)