Encode "real world" positions of the input data

[JackKelly]

As well as (or instead of) Fourier-encoding the pixel position relative to the edge of the image (e.g. 'top left in the image'), it might be interesting to encode:

• The absolute geospatial position (i.e. the lat, lon) (maybe give the geospatial pos for every pixel?!?)
• Datetime features:
  • time-of-day
  • day-of-year
• Angle & azimuth of the sun (maybe this is only relevant for PV data?)

Should these encodings be computed on-the-fly during ML training? Or should the encodings be 'burnt into' the batches on disk produced by nowcasting_dataset? I'd perhaps lean towards the former but keen to hear opinions!

[jacobbieker]

If the NetCDF data written by nowcasting_dataset includes the geospatial coordinates I'd say compute them on the fly. For satflow I computed them once and saved them alongside the other data, but that's only because I was just using Numpy arrays, so didn't have a simple way to recalculate them on the fly.

[JackKelly]

Yeah, that sounds like a plan: nowcasting_dataset includes the geospatial (and temporal) coordinates on-disk, and then the ML model can compute the embeddings on the fly.

[JackKelly]

Maybe converting "real world" positions to position encodings suitable for Perceiver could be done in the proposed thin data loading layer in nowcasting_dataloader: openclimatefix/nowcasting_dataset#97

[JackKelly]

This is implemented by @jacobbieker's absolute position encoding in nowcasting_dataloader: openclimatefix/nowcasting_dataloader#7