GH-49942: [Python] Protect PyBuffer and NumPyBuffer destructors against interpreter finalization

[pitrou]

Rationale for this change

In #38637 we protected the OwnedRef and OwnedRefNoGIL destructors against being called when the Python interpreter is being finalized, but we didn't address other classes with custom destructors, such as PyBuffer and NumPyBuffer.

What changes are included in this PR?

Avoid executing Python C API code if the interpreter is finalized while a C++ destructor runs.

Are these changes tested?

Only manually, using the initial reproducer in #45214.

Are there any user-facing changes?

No.

• GitHub Issue: [Python] Protect PyBuffer and NumPyBuffer destructors against interpreter finalization #49942

[pitrou]

@github-actions crossbow submit -g python

[github-actions]

Revision: bbe7634

Submitted crossbow builds: ursacomputing/crossbow @ actions-62ce37605c

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.10
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-1.3.4-numpy-1.21.2
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.12-pandas-latest-numpy-1.26
test-conda-python-3.12-pandas-latest-numpy-latest
test-conda-python-3.13
test-conda-python-3.13-pandas-nightly-numpy-nightly
test-conda-python-3.13-pandas-upstream_devel-numpy-nightly
test-conda-python-3.14
test-conda-python-emscripten
test-debian-13-python-3-amd64
test-debian-13-python-3-i386
test-fedora-42-python-3
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-python-3

[pitrou]

@github-actions crossbow submit wheelmac* wheelwin*

[github-actions]

Revision: bbe7634

Submitted crossbow builds: ursacomputing/crossbow @ actions-62ea8dd650

Task	Status
wheel-macos-monterey-cp310-cp310-amd64
wheel-macos-monterey-cp310-cp310-arm64
wheel-macos-monterey-cp311-cp311-amd64
wheel-macos-monterey-cp311-cp311-arm64
wheel-macos-monterey-cp312-cp312-amd64
wheel-macos-monterey-cp312-cp312-arm64
wheel-macos-monterey-cp313-cp313-amd64
wheel-macos-monterey-cp313-cp313-arm64
wheel-macos-monterey-cp313-cp313t-amd64
wheel-macos-monterey-cp313-cp313t-arm64
wheel-macos-monterey-cp314-cp314-amd64
wheel-macos-monterey-cp314-cp314-arm64
wheel-macos-monterey-cp314-cp314t-amd64
wheel-macos-monterey-cp314-cp314t-arm64
wheel-windows-cp310-cp310-amd64
wheel-windows-cp311-cp311-amd64
wheel-windows-cp312-cp312-amd64
wheel-windows-cp313-cp313-amd64
wheel-windows-cp313-cp313t-amd64
wheel-windows-cp314-cp314-amd64
wheel-windows-cp314-cp314t-amd64

[raulcd]

Thanks for this @pitrou ! The failures on CI are unrelated and already tracked on opened issues

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 0 benchmarking runs that have been run so far on merge-commit d4e0e26.

None of the specified runs were found on the Conbench server.

The full Conbench report has more details.