Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/build_and_deploy_docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ jobs:

- name: Build Docusaurus docs
run: make build-docs
env:
APIFY_SIGNING_TOKEN: ${{ secrets.APIFY_SIGNING_TOKEN }}

- name: Set up GitHub Pages
uses: actions/configure-pages@v5
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/run_code_checks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ jobs:
docs_check:
name: Docs check
uses: apify/workflows/.github/workflows/python_docs_check.yaml@main
secrets: inherit

integration_tests:
name: Integration tests
Expand Down
8 changes: 4 additions & 4 deletions docs/01_overview/01_introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@ id: introduction
title: Introduction
---

import CodeBlock from '@theme/CodeBlock';
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';

import IntroductionExample from '!!raw-loader!./code/01_introduction.py';
import IntroductionExample from '!!raw-loader!roa-loader!./code/01_introduction.py';

The Apify SDK for Python is the official library for creating [Apify Actors](https://docs.apify.com/platform/actors) using Python.

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{IntroductionExample}
</CodeBlock>
</RunnableCodeBlock>

## What are Actors?

Expand Down
14 changes: 7 additions & 7 deletions docs/01_overview/03_actor_structure.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ id: actor-structure
title: Actor structure
---

import CodeBlock from '@theme/CodeBlock';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CodeBlock from '@theme/CodeBlock';

import UnderscoreMainExample from '!!raw-loader!./code/actor_structure/main.py';
import MainExample from '!!raw-loader!./code/actor_structure/__main__.py';
import UnderscoreMainExample from '!!raw-loader!./code/actor_structure/__main__.py';
import MainExample from '!!raw-loader!./code/actor_structure/main.py';

All Python Actor templates follow the same structure.

Expand All @@ -20,14 +20,14 @@ which follows the [standard requirements file format](https://pip.pypa.io/en/sta
The Actor's source code is in the `src/` folder. This folder contains two important files: `main.py`, which contains the main function of the Actor, and `__main__.py`, which is the entrypoint of the Actor package, setting up the Actor [logger](../concepts/logging) and executing the Actor's main function via [`asyncio.run`](https://docs.python.org/3/library/asyncio-runner.html#asyncio.run).

<Tabs>
<TabItem value="main.py" label="main.py" default>
<TabItem value="__main__.py" label="__main.py__" default>
<CodeBlock className="language-python">
{MainExample}
{UnderscoreMainExample}
</CodeBlock>
</TabItem>
<TabItem value="__main__.py" label="__main.py__">
<TabItem value="main.py" label="main.py">
<CodeBlock className="language-python">
{UnderscoreMainExample}
{MainExample}
</CodeBlock>
</TabItem>
</Tabs>
Expand Down
13 changes: 10 additions & 3 deletions docs/01_overview/code/01_introduction.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import asyncio

import httpx
from bs4 import BeautifulSoup

Expand All @@ -6,12 +8,17 @@

async def main() -> None:
async with Actor:
actor_input = await Actor.get_input()
actor_input = await Actor.get_input() or {}
url = actor_input.get('url', 'https://apify.com')
async with httpx.AsyncClient() as client:
response = await client.get(actor_input['url'])
response = await client.get(url)
soup = BeautifulSoup(response.content, 'html.parser')
data = {
'url': actor_input['url'],
'url': url,
'title': soup.title.string if soup.title else None,
}
await Actor.push_data(data)


if __name__ == '__main__':
asyncio.run(main())
42 changes: 21 additions & 21 deletions docs/02_concepts/01_actor_lifecycle.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,21 @@ id: actor-lifecycle
title: Actor lifecycle
---

import CodeBlock from '@theme/CodeBlock';
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

import ClassContextExample from '!!raw-loader!./code/01_class_context.py';
import ClassManualExample from '!!raw-loader!./code/01_class_manual.py';
import InstanceContextExample from '!!raw-loader!./code/01_instance_context.py';
import InstanceManualExample from '!!raw-loader!./code/01_instance_manual.py';
import ClassContextExample from '!!raw-loader!roa-loader!./code/01_class_context.py';
import ClassManualExample from '!!raw-loader!roa-loader!./code/01_class_manual.py';
import InstanceContextExample from '!!raw-loader!roa-loader!./code/01_instance_context.py';
import InstanceManualExample from '!!raw-loader!roa-loader!./code/01_instance_manual.py';

import ErrorHandlingContextExample from '!!raw-loader!./code/01_error_handling_context.py';
import ErrorHandlingManualExample from '!!raw-loader!./code/01_error_handling_manual.py';
import ErrorHandlingContextExample from '!!raw-loader!roa-loader!./code/01_error_handling_context.py';
import ErrorHandlingManualExample from '!!raw-loader!roa-loader!./code/01_error_handling_manual.py';

import RebootExample from '!!raw-loader!./code/01_reboot.py';
import RebootExample from '!!raw-loader!roa-loader!./code/01_reboot.py';

import StatusMessageExample from '!!raw-loader!./code/01_status_message.py';
import StatusMessageExample from '!!raw-loader!roa-loader!./code/01_status_message.py';

This guide explains how an **Apify Actor** starts, runs, and shuts down, describing the complete Actor lifecycle. For information about the core concepts such as Actors, the Apify Console, storages, and events, check out the [Apify platform documentation](https://docs.apify.com/platform).

Expand All @@ -31,29 +31,29 @@ When the Actor exits, either normally or due to an exception, the SDK performs a

<Tabs groupId="request_queue">
<TabItem value="actor_class_with_context_manager" label="Actor class with context manager" default>
<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{ClassContextExample}
</CodeBlock>
</RunnableCodeBlock>
</TabItem>
<TabItem value="actor_class_with_manual_init_exit" label="Actor class with manual init/exit">
<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{ClassManualExample}
</CodeBlock>
</RunnableCodeBlock>
</TabItem>
</Tabs>

You can also create an [`Actor`](https://docs.apify.com/sdk/python/reference/class/Actor) instance directly. This does not change its capabilities but allows you to specify optional parameters during initialization, such as disabling automatic `sys.exit()` calls or customizing timeouts. The choice between using a context manager or manual initialization depends on how much control you require over the Actor's startup and shutdown sequence.

<Tabs groupId="request_queue">
<TabItem value="actor_instance_with_context_manager" label="Actor instance with context manager" default>
<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{InstanceContextExample}
</CodeBlock>
</RunnableCodeBlock>
</TabItem>
<TabItem value="actor_instance_with_manual_init_exit" label="Actor instance with manual init/exit">
<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{InstanceManualExample}
</CodeBlock>
</RunnableCodeBlock>
</TabItem>
</Tabs>

Expand All @@ -72,19 +72,19 @@ Catch exceptions only when necessary - for example, to retry network timeouts or

Below is a minimal context-manager example where an unhandled exception automatically fails the run, followed by a manual pattern giving you more control.

<CodeBlock className="language-python">{ErrorHandlingContextExample}</CodeBlock>
<RunnableCodeBlock className="language-python" language="python">{ErrorHandlingContextExample}</RunnableCodeBlock>

If you need explicit control over exit codes or status messages, you can manage the Actor manually using [`Actor.init`](https://docs.apify.com/sdk/python/reference/class/Actor#init), [`Actor.exit`](https://docs.apify.com/sdk/python/reference/class/Actor#exit), and [`Actor.fail`](https://docs.apify.com/sdk/python/reference/class/Actor#fail).

<CodeBlock className="language-python">{ErrorHandlingManualExample}</CodeBlock>
<RunnableCodeBlock className="language-python" language="python">{ErrorHandlingManualExample}</RunnableCodeBlock>

## Reboot

Rebooting (available on the Apify platform only) instructs the platform worker to restart your Actor from the beginning of its execution. Use this mechanism only for transient conditions that are likely to resolve after a fresh start — for example, rotating a blocked proxy pool or recovering from a stuck browser environment.

Before triggering a reboot, persist any essential state externally (e.g., to the key-value store or dataset), as all in-memory data is lost after reboot. The example below tracks a reboot counter in the default key-value store and allows at most three restarts before exiting normally.

<CodeBlock className="language-python">{RebootExample}</CodeBlock>
<RunnableCodeBlock className="language-python" language="python">{RebootExample}</RunnableCodeBlock>

## Status message

Expand All @@ -94,7 +94,7 @@ Update the status only when the user's understanding of progress changes - avoid

The SDK optimizes updates by sending an API request only when the message text changes, so repeating the same message incurs no additional cost.

<CodeBlock className="language-python">{StatusMessageExample}</CodeBlock>
<RunnableCodeBlock className="language-python" language="python">{StatusMessageExample}</RunnableCodeBlock>

## Conclusion

Expand Down
8 changes: 4 additions & 4 deletions docs/02_concepts/02_actor_input.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@ id: actor-input
title: Actor input
---

import CodeBlock from '@theme/CodeBlock';
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';

import InputExample from '!!raw-loader!./code/02_input.py';
import InputExample from '!!raw-loader!roa-loader!./code/02_input.py';

The Actor gets its [input](https://docs.apify.com/platform/actors/running/input) from the input record in its default [key-value store](https://docs.apify.com/platform/storage/key-value-store).

To access it, instead of reading the record manually, you can use the [`Actor.get_input`](../../reference/class/Actor#get_input) convenience method. It will get the input record key from the Actor configuration, read the record from the default key-value store,and decrypt any [secret input fields](https://docs.apify.com/platform/actors/development/secret-input).

For example, if an Actor received a JSON input with two fields, `{ "firstNumber": 1, "secondNumber": 2 }`, this is how you might process it:

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{InputExample}
</CodeBlock>
</RunnableCodeBlock>
50 changes: 25 additions & 25 deletions docs/02_concepts/03_storages.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@ id: storages
title: Working with storages
---

import CodeBlock from '@theme/CodeBlock';
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';

import OpeningStoragesExample from '!!raw-loader!./code/03_opening_storages.py';
import DeletingStoragesExample from '!!raw-loader!./code/03_deleting_storages.py';
import DatasetReadWriteExample from '!!raw-loader!./code/03_dataset_read_write.py';
import DatasetExportsExample from '!!raw-loader!./code/03_dataset_exports.py';
import KvsReadWriteExample from '!!raw-loader!./code/03_kvs_read_write.py';
import KvsIteratingExample from '!!raw-loader!./code/03_kvs_iterating.py';
import KvsPublicRecordExample from '!!raw-loader!./code/03_kvs_public_url.py';
import RqExample from '!!raw-loader!./code/03_rq.py';
import OpeningStoragesExample from '!!raw-loader!roa-loader!./code/03_opening_storages.py';
import DeletingStoragesExample from '!!raw-loader!roa-loader!./code/03_deleting_storages.py';
import DatasetReadWriteExample from '!!raw-loader!roa-loader!./code/03_dataset_read_write.py';
import DatasetExportsExample from '!!raw-loader!roa-loader!./code/03_dataset_exports.py';
import KvsReadWriteExample from '!!raw-loader!roa-loader!./code/03_kvs_read_write.py';
import KvsIteratingExample from '!!raw-loader!roa-loader!./code/03_kvs_iterating.py';
import KvsPublicRecordExample from '!!raw-loader!roa-loader!./code/03_kvs_public_url.py';
import RqExample from '!!raw-loader!roa-loader!./code/03_rq.py';

The `Actor` class provides methods to work either with the default storages of the Actor, or with any other storage, named or unnamed.

Expand Down Expand Up @@ -65,18 +65,18 @@ There are several methods for directly working with the default key-value store

The [`Actor.open_dataset`](../../reference/class/Actor#open_dataset), [`Actor.open_key_value_store`](../../reference/class/Actor#open_key_value_store) and [`Actor.open_request_queue`](../../reference/class/Actor#open_request_queue) methods can be used to open any storage for reading and writing. You can either use them without arguments to open the default storages, or you can pass a storage ID or name to open another storage.

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{OpeningStoragesExample}
</CodeBlock>
</RunnableCodeBlock>

## Deleting storages

To delete a storage, you can use the [`Dataset.drop`](../../reference/class/Dataset#drop),
[`KeyValueStore.drop`](../../reference/class/KeyValueStore#drop) or [`RequestQueue.drop`](../../reference/class/RequestQueue#drop) methods.

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{DeletingStoragesExample}
</CodeBlock>
</RunnableCodeBlock>

## Working with datasets

Expand All @@ -90,19 +90,19 @@ To read data from a dataset, you can use the [`Dataset.get_data`](../../referenc

To get an iterator of the data, you can use the [`Dataset.iterate_items`](../../reference/class/Dataset#iterate_items) method.

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{DatasetReadWriteExample}
</CodeBlock>
</RunnableCodeBlock>

### Exporting items

You can also export the dataset items into a key-value store, as either a CSV or a JSON record,
using the [`Dataset.export_to_csv`](../../reference/class/Dataset#export_to_csv)
or [`Dataset.export_to_json`](../../reference/class/Dataset#export_to_json) method.

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{DatasetExportsExample}
</CodeBlock>
</RunnableCodeBlock>

## Working with key-value stores

Expand All @@ -116,27 +116,27 @@ To write records into a key-value store, you can use the [`KeyValueStore.set_val
You can set the content type of a record with the `content_type` argument.
To delete a record, set its value to `None`.

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{KvsReadWriteExample}
</CodeBlock>
</RunnableCodeBlock>

### Iterating keys

To get an iterator of the key-value store record keys,
you can use the [`KeyValueStore.iterate_keys`](../../reference/class/KeyValueStore#iterate_keys) method.

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{KvsIteratingExample}
</CodeBlock>
</RunnableCodeBlock>

### Public URLs of records

To get a publicly accessible URL of a key-value store record,
you can use the [`KeyValueStore.get_public_url`](../../reference/class/KeyValueStore#get_public_url) method.

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{KvsPublicRecordExample}
</CodeBlock>
</RunnableCodeBlock>

## Working with request queues

Expand Down Expand Up @@ -168,6 +168,6 @@ To check if all the requests in the queue are handled, you can use the [`Request

### Full example

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{RqExample}
</CodeBlock>
</RunnableCodeBlock>
8 changes: 4 additions & 4 deletions docs/02_concepts/04_actor_events.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ id: actor-events
title: Actor events & state persistence
---

import CodeBlock from '@theme/CodeBlock';
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';

import ActorEventsExample from '!!raw-loader!./code/04_actor_events.py';
import ActorEventsExample from '!!raw-loader!roa-loader!./code/04_actor_events.py';

During its runtime, the Actor receives Actor events sent by the Apify platform or generated by the Apify SDK itself.

Expand Down Expand Up @@ -76,6 +76,6 @@ During its runtime, the Actor receives Actor events sent by the Apify platform o
To add handlers to these events, you use the [`Actor.on`](../../reference/class/Actor#on) method,
and to remove them, you use the [`Actor.off`](../../reference/class/Actor#off) method.

<CodeBlock className="language-python">
<RunnableCodeBlock className="language-python" language="python">
{ActorEventsExample}
</CodeBlock>
</RunnableCodeBlock>
Loading