Empty projection reads every column of every data file #3472

Describe the bug

A scan with an empty projection (TableScanBuilder::select_empty(), so FileScanTask::project_field_ids() == []) reads every column of every data file. An engine that only needs a row count, for example Spark's df.isEmpty which becomes select().limit(1), pays the full I/O cost of the table. Reading only the Parquet footers would be enough.

Two places treat the empty list as "read everything".

crates/iceberg/src/arrow/reader/pipeline.rs: the metadata-only path excludes the empty list.

let metadata_only_projection = !field_ids.is_empty()
    && field_ids
        .iter()
        .all(|id| PRUNABLE_METADATA_FIELDS.contains(id));

crates/iceberg/src/arrow/reader/projection.rs: the empty list falls through to all columns.

if field_ids.is_empty() {
    return Ok(ProjectionMask::all());
}

A comment in pipeline.rs says this is deliberate so that COUNT(*) keeps its row count. The metadata-only path already gets the row count from the RowNumber virtual column with ProjectionMask::none, so reading all columns is not needed for that.

Related: #676 and #677 added select_empty() but did not change the reader.

To Reproduce

Add this test to the tests module in crates/iceberg/src/arrow/reader/pipeline.rs. It uses the existing write_parquet_with_wide_column, metadata_projection_task and scan_task helpers.

#[tokio::test]
async fn empty_projection_reads_no_data_columns() {
    let tmp_dir = TempDir::new().unwrap();
    let dir = tmp_dir.path().to_str().unwrap();

    let empty = metadata_projection_task(
        write_parquet_with_wide_column(dir, "empty.parquet", vec![], vec![]),
        id_and_wide_schema(),
        vec![],
    );
    let (batches, empty_bytes) = scan_task(empty).await;
    let rows: usize = batches.iter().map(|b| b.num_rows()).sum();
    assert_eq!(rows, 3);

    let wide = metadata_projection_task(
        write_parquet_with_wide_column(dir, "wide.parquet", vec![], vec![]),
        id_and_wide_schema(),
        vec![2],
    );
    let (_, wide_bytes) = scan_task(wide).await;

    assert!(
        empty_bytes < wide_bytes,
        "empty projection should read fewer bytes than the wide column: {empty_bytes} vs {wide_bytes}"
    );
}

Measured bytes_read on main for the same 3-row file:

The empty projection reads at least as much as the wide column alone, so the test above fails.

Expected behavior

An empty projection reads no data columns and still returns the correct row count. With the !field_ids.is_empty() && guard removed from metadata_only_projection, the same file reads 1928 bytes, equal to the _pos-only case, and still returns 3 rows with 0 columns.

Willingness to contribute

I can contribute a fix for this bug independently.