Support Puffin file format #744
Assignees
Comments
This was referenced Nov 30, 2024
|
Can someone assign this ticket to me to avoid confusion? |
This was referenced Dec 7, 2024
|
We have also implemented a puffin inside greptimedb (https://github.com/GreptimeTeam/greptimedb/tree/main/src/puffin), would like to push this forward together 🍻 |
Great! Looking forward to see your contribution! |
Xuanwo
pushed a commit
that referenced
this issue
Feb 8, 2025
Part of #744 # Summary - Add PuffinReader # Context - This is the fourth of a number of PRs to add support for Iceberg Puffin file format. - It might be helpful to refer to the overarching [PR](#714) from which these changes were split to understand better how these changes will fit in to the larger picture. - It may also be helpful to refer to the Java reference implementation for PuffinReader [here](https://github.com/apache/iceberg/blob/8cd5b1985d3f9c55ab2ced174559a8416b6ca1b4/core/src/main/java/org/apache/iceberg/puffin/PuffinReader.java#L123).
liurenjie1024
pushed a commit
that referenced
this issue
Apr 1, 2025
Part of #744 # Summary - Add PuffinWriter # Context - This is the fifth of a number of PRs to add support for Iceberg Puffin file format. - It might be helpful to refer to the overarching #714 from which these changes were split to understand better how these changes will fit in to the larger picture. - It may also be helpful to refer to the Java reference implementation for PuffinWriter [here](https://github.com/apache/iceberg/blob/1d9fefeb9680d782dc128f242604903e71c32f97/core/src/main/java/org/apache/iceberg/puffin/PuffinWriter.java#L43).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Why
There has been increased interest in Iceberg's Puffin file format recently. This is partially driven by the fact that the Iceberg V3 Spec added support for deletion vectors which are expected to be stored in Puffin files.
However, as was recently noted on the dev-list, currently only the iceberg-java SDK supports reading or writing Puffin files. Iceberg-rust itself has zero support for the Puffin file format today. The purpose of this ticket is to change that by adding support for Puffin to iceberg-rust and iceberg-python (through exposed bindings).
How
I have already raised a PR for adding Puffin support to iceberg-rust here: #714. However as that PR is quite large, I am splitting up the code and submitting it as multiple PRs in the following order:
pubandpub useannotationsThe text was updated successfully, but these errors were encountered: