New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow users to configure sources (Datasets Only) #10

Open

7 tasks done

ylyangtw opened this issue Apr 22, 2024 · 0 comments

Open

7 tasks done

Allow users to configure sources (Datasets Only) #10

ylyangtw opened this issue Apr 22, 2024 · 0 comments

Assignees

Labels

Contributor

ylyangtw commented Apr 22, 2024 •

edited

Loading

Refactor earthcube utility to read tenant config from minio
Add a sensor feature to fetch the config from the minio
Once a config change is detected, the sensor triggers a run of data pipeline automatically
A google form for the inputs
Have a script to validate the URLs and jsonlds
A script to transform google sheet to sitemap and upload it to s3 storage
Run the data pipeline in scheduler

ylyangtw changed the title ~~Allow users to add repos~~ Allow users to add sources

ylyangtw mentioned this issue

Add other repos to the geochemistry instance #7

Closed

ylyangtw added this to Decoder

ylyangtw moved this to In Progress in Decoder

ylyangtw changed the title ~~Allow users to add sources~~ Allow users to configure sources

ylyangtw self-assigned this

lmarini added the data ingestion label

ylyangtw changed the title ~~Allow users to configure sources~~ Allow users to configure sources (Datasets Only)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment