Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
"test:watch": "vitest"
},
"dependencies": {
"@apify/actor-memory-expression": "^1.0.1",
"@apify/input_schema": "^3.29.2",
"@crawlee/core": "4.0.0-beta.145",
"@crawlee/fs-storage": "4.0.0-beta.145",
Expand Down
92 changes: 92 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 9 additions & 1 deletion requirements/actor-driver.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,15 @@ start`, ...) is refused by name, naming both the `CMD` fix and how to clear debu
- Run details and the run log are kept in internal records that persist across runtime restarts
(`storage.md`).

## Run memory

- Run memory follows the platform's `.actor/actor.json` rules (`defaultMemoryMbytes`, `minMemoryMbytes`,
`maxMemoryMbytes`). Differences:
- A `defaultMemoryMbytes` that fails to evaluate, and a memory changed by the min/max bounds, are
noted in the run log.
- `defaultMemoryMbytes` is not capped to a plan's maximum memory.
- A memory field its schema forbids fails the build.

## Resource limits

- Every run's container has a hard memory limit and a hard CPU limit. Memory is the run's `memoryMbytes`;
Expand All @@ -191,7 +200,6 @@ start`, ...) is refused by name, naming both the `CMD` fix and how to clear debu
- Limits are applied exactly as requested, even when they exceed the host's own capacity. Such a run is
warned about in its own log, naming the requested and the host figures; the limits still apply. When the
host's capacity cannot be determined, no warning is produced.
- Disk is not limited. `diskMbytes` is reported but never enforced.

## Run resource telemetry

Expand Down
2 changes: 1 addition & 1 deletion requirements/unsupported.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ real account, not in the runtime.
- Billing: a run's charges and costs are reported, never invoiced or paid out
- Encrypted secret input fields
- Actor-level default run options
- Dynamic and bounded memory from `.actor/actor.json`
- Extra named default storages (`storageIds`)
- Limited-permission Actors and per-run scoped tokens
- Automatic run and build retention
Expand Down Expand Up @@ -102,6 +101,7 @@ real account, not in the runtime.

- Memory steps and bounds (128 MB - 32 GB, powers of two); a run asking for anything else is warned
about in its log and started with it anyway
- Run disk limit (`diskMbytes`)
- Record, item and input size limits
- Concurrent run, rate and per-account quotas
- Process, file-descriptor and shared-memory limits
Expand Down
1 change: 1 addition & 0 deletions sample_actor_playwright/.actor/actor.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
"description": "Crawls up to maxRequestsPerCrawl same-hostname pages from startUrls with a headful Chrome driven by Playwright, pushing one dataset item per page.",
"version": "0.0",
"buildTag": "latest",
"defaultMemoryMbytes": 4096,
"meta": {
"templateId": "ts-crawlee-playwright-chrome"
},
Expand Down
1 change: 1 addition & 0 deletions sample_actor_playwright_py/.actor/actor.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
"description": "Crawls up to max_requests_per_crawl same-hostname pages from start_urls with a headful Chromium driven by Playwright (Crawlee for Python), pushing one dataset item per page.",
"version": "0.0",
"buildTag": "latest",
"defaultMemoryMbytes": 4096,
"meta": {
"templateId": "python-crawlee-playwright"
},
Expand Down
1 change: 1 addition & 0 deletions sample_actor_py/.actor/actor.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"description": "Crawls up to maxPages same-domain pages starting from startUrl, over a request queue, pushing one dataset item per page.",
"version": "0.0",
"buildTag": "latest",
"defaultMemoryMbytes": "get(input, 'maxPages', 1) * 128",
"meta": {
"templateId": "python-empty"
},
Expand Down
2 changes: 1 addition & 1 deletion sample_actor_py/.actor/input_schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"title": "Max pages",
"type": "integer",
"description": "The maximum number of pages to crawl, following same-domain links from the start URL.",
"default": 2,
"default": 10,
"minimum": 1
}
},
Expand Down
1 change: 1 addition & 0 deletions sample_actor_ts/.actor/actor.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
"description": "Crawls up to maxPages same-hostname pages starting from startUrl, over a request queue, pushing one dataset item per page.",
"version": "0.0",
"buildTag": "latest",
"defaultMemoryMbytes": "get(input, 'maxPages', 1) * 128",
"meta": {
"templateId": "ts-empty"
},
Expand Down
2 changes: 1 addition & 1 deletion sample_actor_ts/.actor/input_schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"title": "Max pages",
"type": "integer",
"description": "The maximum number of pages to crawl, following same-hostname links from the start URL.",
"default": 2,
"default": 10,
"minimum": 1
}
},
Expand Down
8 changes: 8 additions & 0 deletions skills/actor-runtime/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,14 @@ valid: Field input.maxPages must be >= 1`); no run is created and no container s
- Two local differences: Apify Proxy groups are not checked (any `apifyProxyGroups` selection is
accepted), and encrypted secret input fields are not supported.

## Run memory

With no `-m`/`--memory`, a run gets `defaultMemoryMbytes` from `.actor/actor.json` - a number or a
memory expression over the input, evaluated as the platform does - or 1024 MB. Any memory, explicit
or not, is then clamped to the Actor's `minMemoryMbytes` / `maxMemoryMbytes`, and the run log says so.
An expression that fails falls back to 1024 MB with a warning in the run log. The fields come from the
build, so changing them needs an `apify push`.

## Iterate without rebuilding (dev folder)

After that first `apify push`, the runtime registers the pushed directory as the Actor's **dev
Expand Down
Loading
Loading