Skip to content

[python] Include partition and bucket context in file deletion conflicts - #10810

Merged
JingsongLi merged 1 commit into
apache:masterfrom
taoran92:fix/python-commit-conflict-partitions
Oct 27, 2026
Merged

JingsongLi merged 1 commit into
apache:masterfrom
taoran92:fix/python-commit-conflict-partitions

Conversation

@taoran92

@taoran92 taoran92 commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Purpose

Closes #10809.

When the Python committer rejects a deletion of a file that is no longer present, the error identifies only the file. Include readable partition and bucket context, for example (partition=dt=2026-10-26, bucket=0), so users can locate the affected data.

Reuse the existing partition conversion and escaping rules, including the configured default partition name. Report <non-partitioned> for unpartitioned tables and fall back to the original partition values if formatting fails. Conflict detection, exception types, and retry behavior remain unchanged.

Related Java improvement: #10566. This PR covers the Python backend's missing-file deletion conflict path.

Tests

Run from paimon-python with Python 3.11 and PyArrow 19.0.1:

python -m pytest -q pypaimon/tests/write/conflict_detection_test.py \
  pypaimon/tests/overwrite_commit_conflict_test.py
  • all tests passed, including partition formatting, fallback diagnostics, and repeated file deletion through the Python committer.

@taoran92
taoran92 marked this pull request as ready for review October 26, 2026 07:33

@Akash3121 Akash3121 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JingsongLi

Copy link
Copy Markdown
Contributor

+1

@JingsongLi
JingsongLi merged commit 1dbe05f into apache:master Oct 27, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Include partition and bucket context in Python file deletion conflicts

3 participants