Is your feature request related to a problem? Please describe.
cudf.DataFrame.to_csv gained ZSTD support in #21518, but the codec has to be named explicitly. df.to_csv("out.csv.zst") silently writes uncompressed CSV bytes under a .zst name, whereas pandas defaults to_csv(compression="infer") and compresses based on the extension. The read direction already infers: read_csv defaults to compression="infer", which maps to CompressionType.AUTO and is resolved from the file extension by infer_compression_type in cpp/src/io/functions.cpp. So the two directions are asymmetric today.
Describe the solution you'd like
Accept compression="infer" in to_csv and resolve it from the output path's extension, matching what read_csv accepts and what pandas does. Only codecs the writer supports would be inferable (currently .zst/.zstd); an extension whose codec the writer does not support should raise rather than silently write plain bytes. Changing the default to "infer" for pandas parity is a separate, breaking decision worth discussing alongside this.
Describe alternatives you've considered
Doing the inference in libcudf instead, by allowing compression_type::AUTO in csv_writer_options and resolving it from sink_info. This is less appealing: sink_info is not always a filepath, and the existing infer_compression_type is keyed on source_info. Keeping inference in the Python layer, where the path is already known, is simpler.
The status quo is also an option: to_csv documents that the codec is not inferred and that 'infer' is unsupported, so the behavior is at least not surprising for anyone who reads the docstring.
Additional context
Raised in review of #21518: #21518 (comment)
Is your feature request related to a problem? Please describe.
cudf.DataFrame.to_csvgained ZSTD support in #21518, but the codec has to be named explicitly.df.to_csv("out.csv.zst")silently writes uncompressed CSV bytes under a.zstname, whereas pandas defaultsto_csv(compression="infer")and compresses based on the extension. The read direction already infers:read_csvdefaults tocompression="infer", which maps toCompressionType.AUTOand is resolved from the file extension byinfer_compression_typeincpp/src/io/functions.cpp. So the two directions are asymmetric today.Describe the solution you'd like
Accept
compression="infer"into_csvand resolve it from the output path's extension, matching whatread_csvaccepts and what pandas does. Only codecs the writer supports would be inferable (currently.zst/.zstd); an extension whose codec the writer does not support should raise rather than silently write plain bytes. Changing the default to"infer"for pandas parity is a separate, breaking decision worth discussing alongside this.Describe alternatives you've considered
Doing the inference in libcudf instead, by allowing
compression_type::AUTOincsv_writer_optionsand resolving it fromsink_info. This is less appealing:sink_infois not always a filepath, and the existinginfer_compression_typeis keyed onsource_info. Keeping inference in the Python layer, where the path is already known, is simpler.The status quo is also an option:
to_csvdocuments that the codec is not inferred and that'infer'is unsupported, so the behavior is at least not surprising for anyone who reads the docstring.Additional context
Raised in review of #21518: #21518 (comment)