Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Access the data through S3 Compat #308

Open
buremba opened this issue Feb 11, 2025 · 0 comments
Open

Access the data through S3 Compat #308

buremba opened this issue Feb 11, 2025 · 0 comments

Comments

@buremba
Copy link

buremba commented Feb 11, 2025

Apparently the data is stored in R2 and data.gharchive.org returns S3-like response. Unfortunately the engines such as Snowflake doesn't work with remote URLs and require S3 API implementation. The following query returns Endpoint data.gharchive.org not allowed..

CREATE STAGE my_r2_stage
  URL = 's3compat://my_bucket/files/'
  ENDPOINT = 'data.gharchive.org'

Would it be possible to share the R2 bucket URL so we can pull the data through the following script?

CREATE or REPLACE STAGE my_s3compat_stage
URL = 's3compat://your_bucket/'
ENDPOINT = 'your_account_id.r2.cloudflarestorage.com'
region = 'region_name'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant