-
Couldn't load subscription status.
- Fork 131
feat: Document new synthetic default dataset item event #2007
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -108,7 +108,7 @@ async def charge_for_api_product_detail(): | |||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| async def main(): | ||||||||||||||||||||||||
| await Actor.init() | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| # API call, or any other logic that you want to charge for | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| charge_result = await charge_for_api_product_detail() | ||||||||||||||||||||||||
|
|
@@ -136,7 +136,7 @@ Use our [SDKs](/sdk) (JS and, Python or use [`apify actor charge`](/cli/docs/nex | |||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| ### Use synthetic start event `apify-actor-start` | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| :::info Synthetic Actor recommended | ||||||||||||||||||||||||
| :::info Synthetic Actor start event recommended | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| We recommend using the synthetic Actor start event in PPE Actors. It benefits both you and your users. | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
|
|
@@ -179,13 +179,28 @@ Your Actor might already have a start event defined, such as `actor-start` or an | |||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| If you want to use the synthetic start event, remove the existing start event from your Actor and add the synthetic start event in Apify Console in the **Publication** tab. | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| ### Use synthetic default dataset item event `apify-actor-default-dataset-item` | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| ::::info Automatic per-item charging | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| The `apify-actor-default-dataset-item` synthetic event charges users for each item written to the run's default dataset. It lets you align PPE pricing with per-result use cases without adding charging code to your Actor. | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| :::: | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| #### How the synthetic default dataset item event works | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| - Apify automatically charges this event whenever your Actor writes an item to the default dataset (for example, when using `Actor.pushData` without opening a named dataset). | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
| - No code changes are required to charge this event. | ||||||||||||||||||||||||
| - You can remove the event in Apify Console if you don't want automatic charging for default dataset items. If you remove it, default dataset writes will no longer be charged automatically. | ||||||||||||||||||||||||
| - The event applies only to the default dataset of the run. Writes to other (non-default) datasets are not charged by this synthetic event. | ||||||||||||||||||||||||
|
Comment on lines
188
to
193
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not a fan of bullet lists like this for whole content of the heading, if possible I would recommend something like:
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We have the same formatting for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we do it once, it's fine, if it starts turning into a pattern, then it's not a good one and we should be avoiding it, thus my recommendation to change it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, its minor thing, lets not block the PR by it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. i would prefer to keep the format consistent with the other synthetic event (start) |
||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| ### Set memory limits | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| Set memory limits using `minMemoryMbytes` and `maxMemoryMbytes` in your [`actor.json`](https://docs.apify.com/platform/actors/development/actor-definition/actor-json) file to control platform usage costs. | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| ```json | ||||||||||||||||||||||||
| { | ||||||||||||||||||||||||
| "actorSpecification": 1, | ||||||||||||||||||||||||
| "actorSpecification": 1, | ||||||||||||||||||||||||
| "name": "name-of-my-scraper", | ||||||||||||||||||||||||
| "version": "0.0", | ||||||||||||||||||||||||
| "minMemoryMbytes": 512, | ||||||||||||||||||||||||
|
|
@@ -213,15 +228,15 @@ import { Actor } from 'apify'; | |||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| const processUrl = async (url) => { | ||||||||||||||||||||||||
| const response = await fetch(url); | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| if (response.status === 404) { | ||||||||||||||||||||||||
| // Charge for the work done and return error item in one call | ||||||||||||||||||||||||
| await Actor.pushData({ | ||||||||||||||||||||||||
| url: url, | ||||||||||||||||||||||||
| error: "404", | ||||||||||||||||||||||||
| errorMessage: "Page not found" | ||||||||||||||||||||||||
| }, 'scraped-result'); | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| return; | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
|
|
@@ -232,7 +247,7 @@ await Actor.init(); | |||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| const input = await Actor.getInput(); | ||||||||||||||||||||||||
| const { urls } = input; | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| for (const url of urls) { | ||||||||||||||||||||||||
| await processUrl(url); | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
|
|
@@ -251,30 +266,30 @@ import requests | |||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| async def process_url(url): | ||||||||||||||||||||||||
| response = requests.get(url) | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| if response.status_code == 404: | ||||||||||||||||||||||||
| # Charge for the work done and return error item in one call | ||||||||||||||||||||||||
| await Actor.push_data({ | ||||||||||||||||||||||||
| 'url': url, | ||||||||||||||||||||||||
| 'error': '404', | ||||||||||||||||||||||||
| 'errorMessage': 'Page not found' | ||||||||||||||||||||||||
| }, 'scraped-result') | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| return | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| # Rest of the process_url function | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| async def main(): | ||||||||||||||||||||||||
| await Actor.init() | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| input_data = await Actor.get_input() | ||||||||||||||||||||||||
| urls = input_data.get('urls', []) | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| for url in urls: | ||||||||||||||||||||||||
| await process_url(url) | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| # Rest of the Actor logic | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| await Actor.exit() | ||||||||||||||||||||||||
| ``` | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
|
|
@@ -294,7 +309,7 @@ However, we acknowledge that some events don't produce tangible results (such as | |||||||||||||||||||||||
| Examples: | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| - _`post` event_: Each charge adds one social media post to the dataset | ||||||||||||||||||||||||
| - _`profile` event_: Each charge adds one user profile to the dataset | ||||||||||||||||||||||||
| - _`profile` event_: Each charge adds one user profile to the dataset | ||||||||||||||||||||||||
| - _`processed-image` event_: Each charge adds one processed image to the dataset | ||||||||||||||||||||||||
| - _`ai-analysis` event_: Each charge processes one document through an AI workflow (no tangible output, but valuable processing) | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Q: how does the table look, isn't the last column super wide?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should break nicely, we can also make use of
<br/>in the worst case scenario to force text into new line how we want it