This repository was archived by the owner on Sep 11, 2023. It is now read-only.
Turn off temporal interpolation of numerical weather predictions (NWPs) #135
Labels
data
New data source or feature; or modification of existing data source
enhancement
New feature or request
Detailed Description
The raw numerical weather predictions (NWPs) are hourly. But
nowcasting_dataset
interpolates the hourly data to produce 5-minutely NWPs in the pre-prepared batches.Interpolating the NWPs massively blows up the filesize of the batches and - in theory (!) - this interpolation shouldn't be necessary for 'fully attentional' ML models like Perceiver, which don't need the inputs to all be aligned in time.
Context
"fully attentional" models view their inputs as a set. To tell the model about the position of each row of input data, we manually encode the position in time and space. As such, we should be able to feed in hourly NWPs and 5-minutely satellite images and PV data, and the model should learn how these things relate to each other in time and space.
Implementation
The implementation should be quite simple, because the code already handles inputs at different sample periods after the GSP pull request. We might need to make sure the
NetCDFDataset
correctly interprets the new batches.The text was updated successfully, but these errors were encountered: