Use rio-cogeo internally

[dionhaefner]

Currently, we have our own functions for optimizing and validating COG, which are more or less copied from rio-cogeo.

I did this because rio-cogeo did not feel quite mature enough at the time, but I think it might be time to re-evaluate.

[vincentsarago]

👋 @dionhaefner I'll be more than happy to see this happening, feel free to give any feedback on the current implementation.
I was planning to do the 1.0.0 release today but I don't think I'll have time and we might ship cogeotiff/rio-cogeo#62 in 1.0.0

The only big difference is that rio-cogeo force everything to be done in-memory while you set up something to handle it differently cogeotiff/rio-cogeo#21

[dionhaefner]

Sure, we'd be happy to work with you to make this happen!

I think out-of-core operation will be a hard requirement for us as we often deal with enormous rasters. When we have some capacities we could look into merging that into rio-cogeo?

[mrpgraae]

Out-of-core operation is indeed a hard requirement for us. But yes, let's see if we can get that into rio-cogeo and use that from then on, that would be great for everyone.

Don't know how your time is looking @dionhaefner, but we might have some capacity for this here in DHI-GRAS within the next 3-4 weeks or so.

[dionhaefner]

Don't know how your time is looking @dionhaefner, but we might have some capacity for this here in DHI-GRAS within the next 3-4 weeks or so.

That's great, since my bandwidth is going to be limited for a while :)

[dionhaefner]

@vincentsarago Could you run me through what you are hoping to achieve with the web optimization (cogeotiff/rio-cogeo#62)? I think it looks very interesting, but I'm not sure where the actual advantages are / what your motivation is. Are you hoping for better performance (e.g. for a tool like Terracotta), or do you primarily want to avoid reprojection artifacts?

[vincentsarago]

sure @dionhaefner The idea is to have a COG aligned on the web mercator grid and zoom levels corresponding to overview levels. The main purpose is to reduce the number of internal tiles fetched to the strict minimum.

avoid reprojection artifacts

Those will in fact happen when creating the COG.

TBH I'm not really planning to use this options but I can see how it could be a nice feature when dealing with mosaics or global dataset.

[dionhaefner]

This could be amazing for Terracotta. We have had something similar to this in the past, but I couldn't get it to work right when retrieving the tiles because boundless windowed reads didn't work properly with rasterio (so we had to revert to using a VRT). Would be interesting to see whether it makes a big difference performance-wise.