-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Feature request
Since we switched to fsspec.parquet.open_parquet_file for our parquet reading code, we added extra round-trips for remote filesystems to check if the destination is a directory. We should allow users to specify if the destination is a directory or a file, so we can skip this check.
I propose to add read_parquet(..., is_dir: bool | None = None), which will perform that check, the same way it is implemented in HATS:
astronomy-commons/hats#601
This would help to resolve this HATS issue:
astronomy-commons/hats#578
Before submitting
Please check the following:
- I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
- I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
- If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request