Populate DB tables with data from SFData

[agennadi]

Context

We need to create Postgres tables based on SQLAlchemy models and populate them with the subset (1000 rows) of the GeoJSON data from the SFData API endpoints.

Definition of Done

• Table Address is populated and all required fields are mapped accurately to the model.
• Table Soft Story is populated and all required fields are mapped accurately to the model.
• Table Seismic Hazard is populated and all required fields are mapped accurately to the model.
• Table Tsunami is populated and all required fields are mapped accurately to the model.
• Table Landslide is populated and all required fields are mapped accurately to the model.
• Table Liquefaction Hazard is populated and all required fields are mapped accurately to the model.
• The code is modular, well-documented, and tested.

Engineering Details

• Addresses dataset: https://data.sfgov.org/Geographic-Locations-and-Boundaries/Addresses-with-Units-Enterprise-Addressing-System/ramy-di5m/about_data
• Soft story dataset: https://data.sfgov.org/Housing-and-Buildings/Map-of-Soft-Story-Properties/jwdp-cqyc
• Seismic Hazard dataset: https://data.sfgov.org/-/San-Francisco-Seismic-Hazard-Zones/re79-p8j5/about_data
• Liquefaction hazard dataset: https://data.sfgov.org/Geographic-Locations-and-Boundaries/Soil-Liquefaction-Hazard-Zone/i4t7-35u3/about_data
• Tsunami dataset: https://services2.arcgis.com/zr3KAIbsRSUyARHG/ArcGIS/rest/services/CA_Tsunami_Hazard_Area/FeatureServer/0
• Landslide dataset: https://data.sfgov.org/Public-Safety/Landslide-Susceptibility-Hazard-Zones/bna4-itif/about_data

[mackcooper1408]

I'm assuming this is really the meat of the ETL service, correct? If so, let's maybe just make sure we put some kind of limit on how much we're loading locally (people probably don't want to load all the datasets onto their computer haha).

[agennadi]

Thanks for the feedback, I updated the issue. We will put a limit of 100 rows for the time being.