[zhuanshenbsj1]

…BLOB fields on PK tables

Purpose

In Java Paimon (SchemaValidation.validateBlob), BLOB columns are supported on both append-only tables (primaryKeys.isEmpty(), requiring row-tracking. enabled = true and data-evolution.enabled = true) and primary-key tables (!primaryKeys.isEmpty(), with dedicated primary-key BLOB validation rules in validatePrimaryKeyBlob).

However, in pypaimon:
1. SchemaManager._validate_blob_fields unconditionally required row-tracking.enabled = true and data-evolution.enabled = true whenever a table had any
BLOB column, which prevented creating any primary-key BLOB table from Python (since row-tracking.enabled cannot be enabled on primary-key tables).
2. FileStoreWrite._create_data_writer blocked all primary-key tables containing any BLOB column, even when all BLOB columns were inline-stored (blob-descriptor-field or blob-view-field) and stored directly as BYTES inside normal Parquet/ORC/Avro data files without .blob sidecar files or rowIdtracking.

This PR aligns pypaimon schema validation with Java and enables creating primary-key BLOB tables, as well as writing and upserting inline BLOB fields
(blob-descriptor-field and blob-view-field) on primary-key tables in Python.

Tests

- `pytest paimon-python/pypaimon/tests/schema_manager_test.py` 
- `pytest paimon-python/pypaimon/tests/blob_table_test.py`

[JingsongLi]

The Python/Java parity and inline PK BLOB write/upsert use cases are useful. I found one schema-compatibility issue below. Verification: schema_manager_test.py and blob_table_test.py passed with PyArrow 19.0.1 (115 passed, 1 skipped, 36 subtests passed); additional format/bucket upsert and delete checks passed. Three Python-created partial-update BLOB configurations were independently rejected when loading the same persisted tables through Java FileStoreTableFactory.create.

[JingsongLi]

Requirement fit: SUPPORTED. Python-created primary-key BLOB tables and inline descriptor/view upserts are useful, and the previously reported partial-update schema restrictions are now addressed.
Implementation: FINDINGS.

On Java 8 / Python 3.13 / PyArrow 19: the requested schema/blob suites pass (116 tests), and 11 Java BLOB tests pass with normal Maven checks. Six real Python-written tables (Parquet/ORC/Avro x descriptor/view) passed two-commit insert/upsert/DELETE/null readback in both Python and this exact branch's Java reader.

However, the added consumer guard regresses the existing writer-configuration contract. All six Python variants of test_filtered_blob_batches_allow_configuration_and_real_writes_capture_it fail locally at this new guard, matching the current Python CI logs. Replacing only this setter with its exact merge-base implementation makes all six pass. Native variants were skipped locally because the Rust module is unavailable; this review does not claim the other native CI failures are resolved.

[JingsongLi]

[P2] Preserve consumer reconfiguration for future bucket writers

This new guard rejects previously valid with_blob_consumer(...) and with_blob_consumer(None) calls once any bucket exists. FileStoreWrite captures the consumer when constructing each DedicatedFormatWriter, so an existing bucket keeps its original callback while newly opened buckets can use a replacement; both baseline Python and Java's BaseAppendFileStoreWrite.withBlobConsumer allow that behavior. It also breaks all six Python variants of the existing native_filtered_writer_configuration_test.py::test_filtered_blob_batches_allow_configuration_and_real_writes_capture_it after the first real write. They pass when only this setter is restored to the baseline version. Remove this guard and preserve the per-bucket capture behavior; the separate URI-reader-factory prewrite restriction should not be applied to the consumer setter.

[zhuanshenbsj1]

@JingsongLi Thank you for the detailed review and catching the consumer guard regression!

I've addressed this feedback in commit bd34991 ([python] Preserve blob consumer reconfiguration and adapt inline blob write_row):

Changes made:
1. ✅ Removed the guard in table_write.py: Removed the data_writers check that was rejecting valid with_blob_consumer() reconfigurations.
2. ✅ Restored per-bucket capture behavior: Preserved the contract where existing bucket writers keep their original consumer while newly opened buckets use the updated one, aligning with Java's BaseAppendFileStoreWrite.
3. ✅ Refactored inline blob encoding in file_store_write.py: Cleanly isolated Append table streaming blobs from Primary-Key table inline BLOB Arrow conversion for write_row.

Verification:
All six Python variants of test_filtered_blob_batches_allow_configuration_and_real_writes_capture_it have been verified locally and pass cleanly.

[JingsongLi]

Requirement fit: SUPPORTED. Implementation: FINDINGS.

The prior consumer-reconfiguration regression is fixed. On this head, Python 3.13 / PyArrow 22.0.0: schema_manager_test.py, blob_table_test.py, and native_filtered_writer_configuration_test.py passed (137 tests plus 46 subtests; 8 native variants skipped because the Rust extension is unavailable).

One additional P2 remains in the explicit postpone-fixed-bucket write_row path, reproduced with real committed tables and byte-input/fixed-bucket controls below.

[JingsongLi]

[P2] Normalize inline BLOB rows before postpone bucket planning

This normalization is too late for table.new_postpone_fixed_bucket_write_builder().new_write(). On a fresh PK table with bucket=-2, postpone.batch-write-fixed-bucket=true, blob-descriptor-field=picture, and write.native.enabled=false, writing a GenericRow containing Blob.from_descriptor_bytes(...) to a new partition raises ValueError: Blob values cannot be converted to Arrow without materializing the stream. PythonPostponeFixedBucketWrite.write_row first calls row_values_to_arrow_table at postpone_batch_table_write.py:132 to estimate the unknown partition's size, so it never reaches this inline encoding.

I reproduced this with Python 3.13 / PyArrow 22.0.0. Serialized descriptor bytes through Arrow commit and read correctly on the same table; after that partition exists, the same Blob-object row succeeds. A fresh ordinary fixed-bucket PK table also succeeds. Please share the inline-reference normalization with the postpone sizing/buffering path before its Arrow conversion, and cover the first write into a new partition for descriptor/view fields.

[zhuanshenbsj1]

@JingsongLi Thanks for catching this! Fixed in commit c713303:

1. Extracted normalize_inline_blob_row in pypaimon/write/row_utils.py and shared it between PythonPostponeFixedBucketWrite.write_row (before row_values_to_arrow_table when sizing/buffering a new partition) and FileStoreWrite.write_row.
2. Added test_postpone_fixed_bucket_pk_inline_blob_write_row_new_partition in blob_table_test.py covering the first write_row into new unplanned partitions for both blob-descriptor-field (Blob.from_descriptor_bytes) and blob-view-field (Blob.from_view) with automatic bucket planning.