chore: update general resource access docs #1988

danpoletaev · 2025-10-07T14:53:10Z

This PR updates general resource access documentation, more specifically:

adds more detailed information about how pre-signed URLs work
adds examples of how Actors can / should be adjusted to supported general resource access
updates the scoped token section on default storages so that it matches the latest implementation & contents of the blog

Closes https://github.com/apify/apify-core/issues/23249

apify-service-account · 2025-10-07T14:56:46Z

Preview for this PR was built for commit f72ff286 and is ready at https://pr-1988.preview.docs.apify.com!

apify-service-account · 2025-10-08T08:19:23Z

Preview for this PR was built for commit aaf52734 and is ready at https://pr-1988.preview.docs.apify.com!

apify-service-account · 2025-10-08T08:30:25Z

Preview for this PR was built for commit 992f083d and is ready at https://pr-1988.preview.docs.apify.com!

raethlo

Content-wise looks good, but I'd consider updating the structure to make it easier to read (comments below) and cut bits here and there for brevity.

sources/platform/collaboration/general-resource-access.md

apify-service-account · 2025-10-09T11:17:21Z

Preview for this PR was built for commit 57f3c965 and is ready at https://pr-1988.preview.docs.apify.com!

apify-service-account · 2025-10-09T11:29:23Z

Preview for this PR was built for commit fe6b89e0 and is ready at https://pr-1988.preview.docs.apify.com!

tobice · 2025-10-13T08:21:58Z

I'm aware of the PR, will take a look ~tomorrow 🎗️

tobice

Looks great, just a few nits I noticed 👍

sources/platform/collaboration/general-resource-access.md

tobice · 2025-10-14T07:47:26Z

sources/platform/collaboration/general-resource-access.md

+
+You can create pre-signed URLs either through the Apify Console or programmatically via the Apify API client.
+
+#### How to generate pre-signed URLs in Apify Console


(general comment)

@raethlo This is essentially... you know... sharing 😄

It's just weird that we have a share modal, and then an export modal, and they both do kind of the same thing. It would totally make sense to somehow integrate it: If you want to share a storage, you should be able to generate a pre-signed link with just content from there.

I have a vague feeling that this has been already brought up somewhere... 🤔

WDYT?

Yes I agree completely, we may have talked about it too when I was writing the previous iteration of docs, I think from UX standpoint we should def combine export/sharing.

apify-service-account · 2025-10-14T08:34:34Z

Preview for this PR was built for commit 30801e58 and is ready at https://pr-1988.preview.docs.apify.com!

apify-service-account · 2025-10-14T08:41:15Z

Preview for this PR was built for commit b89e6736 and is ready at https://pr-1988.preview.docs.apify.com!

apify-service-account · 2025-10-14T08:55:18Z

Preview for this PR was built for commit af5d0ffc and is ready at https://pr-1988.preview.docs.apify.com!

TC-MO

Few changes requested, generally speaking our style guide calls for bold to be reserved for UI elements, and italics to be used for emphasis

TC-MO · 2025-10-15T11:00:25Z

sources/platform/collaboration/general-resource-access.md

+
+#### What links can be pre-signed
+
+Only selected **dataset** and **key-value store** endpoints support pre-signed URLs.  


Suggested change

Only selected **dataset** and **key-value store** endpoints support pre-signed URLs.

Only selected _dataset_ and _key-value store_ endpoints support pre-signed URLs.

TC-MO · 2025-10-15T11:00:58Z

sources/platform/collaboration/general-resource-access.md

+| **Datasets** | [Dataset items](/api/v2/dataset-items-get) (`/v2/datasets/:datasetId/items`) | Temporary or Permanent | The link provides access to all dataset items. |
+| **Key-value stores** | [List of keys](/api/v2/key-value-store-keys-get) (`/v2/key-value-stores/:storeId/keys`) | Temporary or Permanent | Returns the list of keys in a store. |
+| **Key-value stores** | [Single record](/api/v2/key-value-store-record-get) (`/v2/key-value-stores/:storeId/records/:recordKey`) | **Permanent only** | The public URL for a specific record is always permanent - it stays valid as long as the record exists. |
+


Suggested change

| **Datasets** | [Dataset items](/api/v2/dataset-items-get) (`/v2/datasets/:datasetId/items`) | Temporary or Permanent | The link provides access to all dataset items. |

| **Key-value stores** | [List of keys](/api/v2/key-value-store-keys-get) (`/v2/key-value-stores/:storeId/keys`) | Temporary or Permanent | Returns the list of keys in a store. |

| **Key-value stores** | [Single record](/api/v2/key-value-store-record-get) (`/v2/key-value-stores/:storeId/records/:recordKey`) | **Permanent only** | The public URL for a specific record is always permanent - it stays valid as long as the record exists. |

| _Datasets_ | [Dataset items](/api/v2/dataset-items-get) (`/v2/datasets/:datasetId/items`) | Temporary or Permanent | The link provides access to all dataset items. |

| _Key-value stores_ | [List of keys](/api/v2/key-value-store-keys-get) (`/v2/key-value-stores/:storeId/keys`) | Temporary or Permanent | Returns the list of keys in a store. |

| _Key-value stores_ | [Single record](/api/v2/key-value-store-record-get) (`/v2/key-value-stores/:storeId/records/:recordKey`) | _Permanent only_ | The public URL for a specific record is always permanent - it stays valid as long as the record exists. |

TC-MO · 2025-10-15T11:01:35Z

sources/platform/collaboration/general-resource-access.md

+- `itemsPublicUrl` – a pre-signed URL providing access to dataset items  
+- `keysPublicUrl` – a pre-signed URL providing access to key-value store keys  
+
+These automatically generated URLs are **valid for 14 days**.


Suggested change

These automatically generated URLs are **valid for 14 days**.

These automatically generated URLs are _valid for 14 days_.

TC-MO · 2025-10-15T11:02:29Z

sources/platform/collaboration/general-resource-access.md

+:::note

-To generate a pre-signed link, you can use the **Export** button in Console, or call the appropriate API client method.
+The link will include a signature **only if the general resource access is set to Restricted**. For unrestricted datasets, the link will work without a signature.


Suggested change

The link will include a signature **only if the general resource access is set to Restricted**. For unrestricted datasets, the link will work without a signature.

The link will include a signature _only if the general resource access is set to Restricted_. For unrestricted datasets, the link will work without a signature.

TC-MO · 2025-10-15T11:02:59Z

sources/platform/collaboration/general-resource-access.md

+
+:::
+
+**Dataset items:**


Shouldn't this be H5?

TC-MO · 2025-10-15T11:03:10Z

sources/platform/collaboration/general-resource-access.md

+
+You can generate pre-signed URLs programmatically for datasets and key-value stores:
+
+**Dataset items**


Shouldn't this be H5?

TC-MO · 2025-10-15T11:03:30Z

sources/platform/collaboration/general-resource-access.md

+const permanentItemsUrl = await datasetClient.createItemsPublicUrl();
+```
+
+**Key-value store list of keys**


Shouldn't this be H5?

TC-MO · 2025-10-15T11:03:33Z

sources/platform/collaboration/general-resource-access.md

+const permanentKeysPublicUrl = await storeClient.createKeysPublicUrl();
+```
+
+**Key-value store record**


Shouldn't this be H5?

TC-MO · 2025-10-15T11:04:37Z

sources/platform/collaboration/general-resource-access.md

+
+If you need finer control — for example, generating links without using Apify client — you can sign URLs manually using our reference implementation.
+
+👉 [See reference implementation in Apify clients](https://github.com/apify/apify-client-js/blob/5efd68a3bc78c0173a62775f79425fad78f0e6d1/src/resource_clients/dataset.ts#L179)


Let's try to minimize emoji usage

Suggested change

👉 [See reference implementation in Apify clients](https://github.com/apify/apify-client-js/blob/5efd68a3bc78c0173a62775f79425fad78f0e6d1/src/resource_clients/dataset.ts#L179)

[Check the reference implementation in Apify clients](https://github.com/apify/apify-client-js/blob/5efd68a3bc78c0173a62775f79425fad78f0e6d1/src/resource_clients/dataset.ts#L179)

TC-MO · 2025-10-15T11:05:05Z

sources/platform/collaboration/general-resource-access.md

+
+👉 [See reference implementation in Apify clients](https://github.com/apify/apify-client-js/blob/5efd68a3bc78c0173a62775f79425fad78f0e6d1/src/resource_clients/dataset.ts#L179)
+
+Manual signing uses standard **HMAC (SHA-256)** with `urlSigningSecretKey` of the resource and can be easily integrated.


Suggested change

Manual signing uses standard **HMAC (SHA-256)** with `urlSigningSecretKey` of the resource and can be easily integrated.

Manual signing uses standard _HMAC (SHA-256)_ with `urlSigningSecretKey` of the resource and can be easily integrated.

danpoletaev · 2025-10-15T13:27:59Z

@TC-MO Lint is failing on the files I've never touched. I tried to merge master hoping that it was resolved, but it wasn't. Can you, please, fix it or let me know if I should fix it?

apify-service-account · 2025-10-15T13:28:50Z

Preview for this PR was built for commit 49d39ee1 and is ready at https://pr-1988.preview.docs.apify.com!

apify-service-account · 2025-10-15T13:32:00Z

Preview for this PR was built for commit 6057eed0 and is ready at https://pr-1988.preview.docs.apify.com!

TC-MO · 2025-10-15T14:09:05Z

Just FYI the files were touched, but that may have been done by the linter? Otherwise neither markdownlint nor vale would touch them as they are set up to only check changed files during PR checks, I'll take a look at what is the issue and fix it in about an hour :)

danpoletaev added 6 commits October 7, 2025 13:48

chore: update pre-signed URLs docs

57e2bbb

chore: improve docs

e90ba12

chore: updated docs

540ca2d

fix: lint

b57d05a

chore: improve docs

8e7188d

chore: fix lint

f72ff28

danpoletaev self-assigned this Oct 7, 2025

danpoletaev requested a review from TC-MO as a code owner October 7, 2025 14:53

github-actions bot added this to the 125th sprint - Core Services team milestone Oct 7, 2025

github-actions bot added the t-core-services Issues with this label are in the ownership of the core services team. label Oct 7, 2025

refactor: clean up

aaf5273

Merge branch 'master' into chore/update-general-resource-access-docs

992f083

danpoletaev requested a review from raethlo October 8, 2025 08:27

raethlo reviewed Oct 8, 2025

View reviewed changes

refactor: clean up

57f3c96

refactor: clean up

fe6b89e

danpoletaev requested a review from tobice October 13, 2025 07:19

tobice approved these changes Oct 14, 2025

View reviewed changes

refactor: clean up

30801e5

danpoletaev requested a review from raethlo October 14, 2025 08:32

Merge branch 'master' into chore/update-general-resource-access-docs

b89e673

refactor: clean up

af5d0ff

TC-MO requested changes Oct 15, 2025

View reviewed changes

refactor: clean up

49d39ee

danpoletaev requested a review from TC-MO October 15, 2025 13:26

refactor: clean up

6057eed


		You can create pre-signed URLs either through the Apify Console or programmatically via the Apify API client.

		#### How to generate pre-signed URLs in Apify Console


		#### What links can be pre-signed

		Only selected dataset and key-value store endpoints support pre-signed URLs.

	Only selected dataset and key-value store endpoints support pre-signed URLs.
	Only selected _dataset_ and _key-value store_ endpoints support pre-signed URLs.

	These automatically generated URLs are valid for 14 days.
	These automatically generated URLs are _valid for 14 days_.

	The link will include a signature only if the general resource access is set to Restricted. For unrestricted datasets, the link will work without a signature.
	The link will include a signature _only if the general resource access is set to Restricted_. For unrestricted datasets, the link will work without a signature.


		You can generate pre-signed URLs programmatically for datasets and key-value stores:

		Dataset items


		If you need finer control — for example, generating links without using Apify client — you can sign URLs manually using our reference implementation.

		👉 [See reference implementation in Apify clients](https://github.com/apify/apify-client-js/blob/5efd68a3bc78c0173a62775f79425fad78f0e6d1/src/resource_clients/dataset.ts#L179)

	👉 [See reference implementation in Apify clients](https://github.com/apify/apify-client-js/blob/5efd68a3bc78c0173a62775f79425fad78f0e6d1/src/resource_clients/dataset.ts#L179)
	[Check the reference implementation in Apify clients](https://github.com/apify/apify-client-js/blob/5efd68a3bc78c0173a62775f79425fad78f0e6d1/src/resource_clients/dataset.ts#L179)


		👉 [See reference implementation in Apify clients](https://github.com/apify/apify-client-js/blob/5efd68a3bc78c0173a62775f79425fad78f0e6d1/src/resource_clients/dataset.ts#L179)

		Manual signing uses standard HMAC (SHA-256) with `urlSigningSecretKey` of the resource and can be easily integrated.

	Manual signing uses standard HMAC (SHA-256) with `urlSigningSecretKey` of the resource and can be easily integrated.
	Manual signing uses standard _HMAC (SHA-256)_ with `urlSigningSecretKey` of the resource and can be easily integrated.

chore: update general resource access docs #1988

Are you sure you want to change the base?

chore: update general resource access docs #1988

Conversation

danpoletaev commented Oct 7, 2025

Uh oh!

apify-service-account commented Oct 7, 2025

Uh oh!

apify-service-account commented Oct 8, 2025

Uh oh!

apify-service-account commented Oct 8, 2025

Uh oh!

raethlo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

apify-service-account commented Oct 9, 2025

Uh oh!

apify-service-account commented Oct 9, 2025

Uh oh!

tobice commented Oct 13, 2025

Uh oh!

tobice left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tobice Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apify-service-account commented Oct 14, 2025

Uh oh!

apify-service-account commented Oct 14, 2025

Uh oh!

apify-service-account commented Oct 14, 2025

Uh oh!

TC-MO left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danpoletaev commented Oct 15, 2025

Uh oh!

apify-service-account commented Oct 15, 2025

Uh oh!

apify-service-account commented Oct 15, 2025

Uh oh!

TC-MO commented Oct 15, 2025

Uh oh!

tobice Oct 14, 2025 •

edited

Loading