Scan with an empty projection (selected_fields=()) returns 0 rows #4137

Apache Iceberg version

0.12.0 (latest release)

Please describe the bug 🐞

Also reproduces on main (d37149c).

table.scan(selected_fields=()).to_arrow() returns an empty table instead of a table with the scan's row count and no columns. to_arrow_batch_reader() does the same, with or without a row_filter. A zero-column result is how callers ask for a row count without reading data: Ray's Dataset.count() pushes an empty projection into read_iceberg and got 0 (ray-project/ray#63913), which Ray worked around in ray-project/ray#65175 by reading one column and dropping it.

import tempfile

import pyarrow as pa
import pyiceberg
from pyiceberg.catalog.sql import SqlCatalog
from pyiceberg.schema import Schema
from pyiceberg.types import LongType, NestedField

wh = tempfile.mkdtemp()
cat = SqlCatalog("c", uri=f"sqlite:///{wh}/cat.db", warehouse=f"file://{wh}")
cat.create_namespace("db")
t = cat.create_table("db.t", schema=Schema(NestedField(1, "id", LongType(), required=False)))
t.append(pa.table({"id": pa.array(range(100), pa.int64())}))

print(pyiceberg.__version__)
print(t.scan(selected_fields=()).to_arrow().num_rows)  # expected 100
print(t.scan(selected_fields=()).to_arrow_batch_reader().read_all().num_rows)  # expected 100
print(t.scan(selected_fields=(), row_filter="id >= 50").to_arrow().num_rows)  # expected 50

0.12.0
0
0
0

Two places drop the row count, both in pyiceberg/io/pyarrow.py:

• _to_requested_schema builds each batch with pa.RecordBatch.from_struct_array(...). A struct array with no children has length 0, so every batch comes back with 0 rows.
• ArrowScan.to_table combines batches with pa.concat_tables, which also returns 0 rows for tables without columns (pyarrow 25.0.1), so fixing the first place alone is not enough for to_arrow().

I have a fix: return batch.select([]) from _to_requested_schema when the requested schema is empty, and build the result with pa.Table.from_batches in to_table for an empty projection. Tests cover to_arrow(), the batch reader, a row filter, and a table with deleted rows. I can open a PR.

Willingness to contribute

• I can contribute a fix for this bug independently