Support Storage Credentials for Vended Credentials #2594
Description
Feature Request / Improvement
Summary
Currently, PyIceberg only loads storage credentials from the config section of the table response.
Meanwhile, Java Iceberg has added support for vended credentials via the storage-credentials field.
📋 Current Behavior
In pyiceberg/catalog/rest.py, the Table is initialized using only config and metadata properties:
def _response_to_table(self, identifier_tuple: Tuple[str, ...], table_response: TableResponse) -> Table:
return Table(
identifier=identifier_tuple,
metadata_location=table_response.metadata_location,
metadata=table_response.metadata,
io=self._load_file_io(
{**table_response.metadata.properties, **table_response.config},
table_response.metadata_location,
),
catalog=self,
config=table_response.config,
)The load_file_io call only considers the merged properties (metadata + config):
def load_file_io(properties: Properties = EMPTY_DICT, location: Optional[str] = None) -> FileIO:
...As a result, any credentials returned via the storage-credentials field in the REST catalog response are ignored.
✅ Expected Behavior
According the Iceberg specification clients should first check for credentials in the storage-credentials field, and only fall back to config if not present.
Proposed behavior (pseudocode):
def _response_to_table(self, identifier_tuple: Tuple[str, ...], table_response: TableResponse) -> Table:
credentials = table_response.storage_credentials or {}
return Table(
identifier=identifier_tuple,
metadata_location=table_response.metadata_location,
metadata=table_response.metadata,
io=self._load_file_io(
{**table_response.metadata.properties, **credentials, **table_response.config},
table_response.metadata_location,
),
catalog=self,
config=table_response.config,
)🧩 References
The same behavior has been implemented in Java Iceberg: