take

One way to do this would be to write DataFusion specific metadata into the files (e..g add something to https://docs.rs/parquet/latest/parquet/file/properties/struct.WriterProperties.html#method.key_value_metadata)

It would be great if we can avoid making something DataFusion specific. Maybe someone could do some research and find how other systems handle this

There is field for sort column, but it seems rowgroup level metadata, so when we set the sort column to parquet, it will applied to rowgroup level metadata.

apache/arrow-rs#3103

One way to do this would be to write DataFusion specific metadata into the files (e..g add something to https://docs.rs/parquet/latest/parquet/file/properties/struct.WriterProperties.html#method.key_value_metadata)

@alamb This is a good idea for file level metadata storage. And i am wandering do we need to add sort column to parquet file metadata also besides the row group level metadata, so we can use it in datafusion?

Copying from #13933 (comment):

So this would look something like:

1. COPY (SELECT ... ORDER BY x, y) to 'foo.parquet'
2. foo.parquet has some new metadata like DATAFUSION_ORDER_BY = x ASC, y ASC`
3. On the next SELECT ... FROM 'foo.parquet' the ParquetExec would look for the DATAFUSION_ORDER_BY metadata in the file and if present update the EquivalenceProperties to reflect an ordering `x,

Already updated the PR, thanks!