zarr integration

[d-v-b]

it would be very cool to see the various image-based encoding strategies used here defined as zarr codecs, so that they could be used by zarr libraries alongside "regular" compression. There's an ongoing effort to standardize some of the image-focused compression routines defined in the imagecodecs package, i'm not sure how much overlap there is between the ffmpeg-centric routines defined in this project and the stuff we are discussing in that issue, but it would be awesome to coordinate as best we can!

[OscarPellicer]

Hi Davis!

Thanks for opening this issue and for the excellent suggestion. Using modern video compression technology within the Zarr ecosystem is a great idea, and I'm glad my work has sparked some interest in that direction.

Though my research on video compression shows that it makes sense to use for Earth System Data, xarrayvideo however operates at a much higher level, using precompiled ffmpeg binaries to encode entire data arrays into video files. I was initially unsure how this workflow could map directly to a Zarr codec, which is much more low-level and, as I understand it, operates on individual chunks.

However, after reading your comment and the linked zarr-extensions issue, I understand that the idea is more to use the philosophy rather than the library, by creating a Zarr codec that leverages the highly efficient codecs like H.265/AV1, which as I can see are already implemented in imagecodecs.

It might make a lot of sense to have chunks compressed as short videos, hence exploiting the time dimension present in many of these data sources. This would capture temporal redundancy and potentially increase compression. My main concern is that accessing a single time-slice from within a video-compressed chunk would require decoding many preceding frames (~100 depending on the encoding configuration), which could be very slow and might negate one of Zarr's key advantages. Or maybe whole chunks are always fully loaded into memory? In that case this would not be a problem, I'm just not sure how it is done.

While I don't have the expertise to write the numcodecs wrapper itself, I'm very supportive of this effort, and I'd be happy to contribute in any way I can, such as by providing compression benchmarks to help justify the work, or suggesting encoding configurations that I have tried successfully.

Thanks again for starting this interesting conversation, and let me know your thoughts!

[d-v-b]

Or maybe whole chunks are always fully loaded into memory? In that case this would not be a problem, I'm just not sure how it is done.

Yes, this is generally how chunks are handled. With a "normal" compression routine like gzip, the entire chunk has to be decoded before it can be treated as an array, so zarr implementations are used to this.

Zarr integration might make sense from your POV if you want to support a zarr -> zarr workflow, so users can take advantage of video compression without needing to save their data into a new container format. I'm working on some code changes to zarr python that should make this process a lot easier, and I can update this issue when I have more results on that.

[OscarPellicer]

Sounds good to me! Let me know when there are updates on this!