Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data extracts from raw-data-api should always be downloaded, update project creation workflow #2175

Open
4 tasks
spwoodcock opened this issue Feb 10, 2025 · 0 comments
Assignees
Labels
backend Related to backend code effort:low Likely a few hours enhancement New feature or request priority:high Should be addressed as a priority

Comments

@spwoodcock
Copy link
Member

spwoodcock commented Feb 10, 2025

  • When we create a data extract via raw-data-api, we should not treat this as special as we do currently:
    • We authenticate with raw-data-api (one extra layer of complexity for administrators of FMTM aside from HOT), plus link directly to the raw-data-api flatgeobuf URL in the S3 bucket.
    • Instead we should simply transfer this file to our bucket, with the bundled Minio instance as part of FMTM (or to the remote attached S3 bucket).
    • This way all data extracts are in the same place, and all treated the same.
    • Originally raw-data-api was the primary data source of FMTM, but now we should treat it as a data source option (one of many).
    • During project creation create extracts as normal and display on the frontend, but save the last extract URL that was created.
    • During project submit, send EITHER raw-data-api parsed GeoJSON, or the custom data extract GeoJSON to the same endpoint for upload.
    • The endpoint should upload the geojson as flatgeobuf to our S3.
    • Delete env var param RAW_DATA_API_AUTH_TOKEN references?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Related to backend code effort:low Likely a few hours enhancement New feature or request priority:high Should be addressed as a priority
Projects
Development

No branches or pull requests

1 participant