REST create_table omits partition spec-id; Java-based servers (Polaris) reject it #3469

Apache Iceberg Rust version

0.10.1 (also present on main at 3bb8167)

Describe the bug

RestCatalog::create_table serializes the partition spec without spec-id when the UnboundPartitionSpec has none set. Since #2610, unset ids are omitted rather than sent as null.

Servers that parse the request with Iceberg Java's PartitionSpecParser require spec-id. For example, Apache Polaris 1.3.0 rejects the request with a bare HTTP 400 Bad Request (WebApplicationException) and logs nothing that points to the cause. Building the spec with .with_spec_id(0) makes the same request succeed.

#2702 notes the same parser behaviour for TableUpdate::AddSpec on the commit path. The create-table path hits it too.

To Reproduce

1. Start Polaris 1.3.0 (in-memory persistence, bootstrap credentials) with an internal catalog.
2. Create a table through RestCatalog with a partition spec built by UnboundPartitionSpec::builder().add_partition_field(...), without with_spec_id.
3. Polaris answers 400.

The same request body replayed with curl:

• "partition-spec": {"fields": [{"source-id": 27, "name": "receive_day", "transform": "day"}]} returns 400.
• "partition-spec": {"spec-id": 0, "fields": [...]} returns 200.
• Adding a field-id without spec-id still returns 400.

Expected behavior

A table created through the REST catalog with an unbound partition spec is accepted by Java-based REST servers. One option is to send spec-id: 0 in CreateTableRequest when it is unset, since a new table's first spec is always 0.

Willingness to contribute

Happy to test a fix against Polaris.

AI disclosure: drafted with an AI coding assistant; reproduced and verified by me.