(fix): structured dtype fill value consolidated metadata #3015
Conversation
github-actions
bot
added
the
needs release notes
ilan-gold
force-pushed
the
ig/fix_structured_dtype_consolidated
branch
from
3f97416 to
8e23bf9
Compare
|
@d-v-b is this pending on anything? we would like to see this in the next patch release as well if possible :) |
tests/test_metadata/test_v2.py
Outdated
| def test_structured_dtype_fill_value_serialization(tmp_path): | ||
| group_path = tmp_path / "test.zarr" | ||
| root_group = zarr.open_group(group_path, mode="w", zarr_format=2) | ||
| root_group.create_array( | ||
| name="structured_headers", | ||
| shape=(100, 100), | ||
| chunks=(100, 100), | ||
| dtype=np.dtype([("foo", "i4"), ("bar", "i4")]), | ||
| ) | ||
|
|
||
| zarr.consolidate_metadata(root_group.store, zarr_format=2) |
There was a problem hiding this comment.
this test is weak (it only tests that consolidate metadata doesn't error). Do you think it makes sense to make this test a big stronger, e.g. by checking that the fill value was actually encoded the way we think it should have been?
Also I think we need a check to ensure that if the dtype is void and the fill value is None, then there's no base64 encoding (I know from the implementation in this PR that the test will pass, but it's good to have the test in any case)
There was a problem hiding this comment.
this test is weak (it only tests that consolidate metadata doesn't error). Do you think it makes sense to make this test a big stronger, e.g. by checking that the fill value was actually encoded the way we think it should have been?
Oh wow, I totally intended to do that (hence the name of the test).
Also I think we need a check to ensure that if the dtype is void and the fill value is None, then there's no base64 encoding
Great suggestion
There was a problem hiding this comment.
Thanks! tried both!
|
thanks for the ping @tasansal, i left some comments about the test, but personally It's OK with me if those comments are ignored. Long term it's not sustainable for us to have duplicated fill value encoding logic across the codebase, and I think some upcoming PRs will help by heavily consolidating this, but I think it's OK in the short-term if this is pushed out quickly. |
Closes #2998
TODO:
docs/user-guide/*.rstchanges/