Skip to content

Commit 0c8dade

Browse files
committed
Add fs.s3a.connection.ssl.enabled and fs.s3a.path.style.access to s3 storage connector creation docs
1 parent c2de0fe commit 0c8dade

File tree

1 file changed

+2
-0
lines changed
  • docs/user_guides/fs/data_source/creation

1 file changed

+2
-0
lines changed

docs/user_guides/fs/data_source/creation/s3.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,8 @@ Here you can specify any additional spark options that you wish to add to the sp
7676

7777
To connect to a S3 compatiable storage other than AWS S3, you can add the option with key as `fs.s3a.endpoint` and the endpoint you want to use as value. The data source will then be able to read from your specified S3 compatible storage.
7878

79+
You can also add options to configure the S3A client. For example, to disable SSL certificate verification, you can add the option with key as `fs.s3a.connection.ssl.enabled` and value as `false`. You can also configure other options such as `fs.s3a.path.style.access` if you use s3 compliant storage which does not support virtual hosting.
80+
7981
!!! warning "Spark Configuration"
8082
When using the data source within a Spark application, the credentials are set at application level. This allows users to access multiple buckets with the same data source within the same application (assuming the credentials allow it).
8183
You can disable this behaviour by setting the option `fs.s3a.global-conf` to `False`. If the `global-conf` option is disabled, the credentials are set on a per-bucket basis and users will be able to use the credentials to access data only from the bucket specified in the data source configuration.

0 commit comments

Comments
 (0)