Bump supported cpython version to 3.14 for testing #4811

clin1234 · 2024-12-21T18:06:38Z

This is to mainly silence this abominable error:

        error: the configured Python interpreter version (3.14) is newer than PyO3's maximum supported version (3.13)
        = help: please check if an updated version of PyO3 is available. Current version: 0.23.1

davidhewitt · 2024-12-21T22:26:15Z

Thanks for the PR. I am not comfortable bumping this check at this point. If we did, unsuspecting users will likely install software built with the next PyO3 version (call it 0.24) when 3.14 stable lands... and they will likely have a terrible experience with random crashes and other frustrating hard-to-debug and dangerous issues.

At the same time, I appreciate the desire to get testing with 3.14 and that PyO3 should not be a blocker for this. See #4662 (comment)
for my latest suggestion on what to do here.

We also have an undocumented environment variable UNSAFE_PYO3_SKIP_VERSION_CHECK=1 which can be used for development / testing. We could document that option, though with the heavy caveat that it's highly likely any unsupported version of Python will crash pretty quickly.

If you are testing PyO3 with 3.14 you may need to make patches to pyo3-ffi to match any changes in the current alphas. I think I'm ok with having such patches in PyO3 to allow testing to make progress, though of course we will not guarantee any stability until at least the beta versions land.

clin1234 · 2025-01-17T18:56:42Z

To investigate:

Following tests fail in 3.14 with Rust stable in the form akin to

internal error: entered unreachable code
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869\library/std\src\panicking.rs:665
   1: core::panicking::panic_fmt
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869\library/core\src\panicking.rs:76
   2: core::panicking::panic
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869\library/core\src\panicking.rs:148
   3: pyo3::instance::Borrowed<pyo3::types::string::PyString>::data
             at .\src\types\string.rs:408
   4: pyo3::types::string::impl$2::data
             at .\src\types\string.rs:313
   5: pyo3::types::string::tests::test_string_data_ucs1::closure$0
             at .\src\types\string.rs:676
   6: pyo3::marker::Python::with_gil<pyo3::types::string::tests::test_string_data_ucs1::closure_env$0,tuple$<> >
             at .\src\marker.rs:412
   7: pyo3::types::string::tests::test_string_data_ucs1
             at .\src\types\string.rs:674
   8: pyo3::types::string::tests::test_string_data_ucs1::closure$0
             at .\src\types\string.rs:673
   9: core::ops::function::FnOnce::call_once<pyo3::types::string::tests::test_string_data_ucs1::closure_env$0,tuple$<> >
             at C:\Users\runneradmin\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:250
  10: core::ops::function::FnOnce::call_once
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869\library/core\src\ops\function.rs:250
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

types::string::tests::test_string_data_ucs1
types::string::tests::test_string_data_ucs1_invalid
types::string::tests::test_string_data_ucs2
types::string::tests::test_string_data_ucs2_invalid
types::string::tests::test_string_data_ucs4
types::string::tests::test_string_data_ucs4_invalid
clippy failing on beta Rust toolchains with

   error: this looks like a formatting argument but it is not part of a formatting macro
   --> src/lib.rs:3:22
    |
  3 |       feature = "nightly",
    |  ______________________^
  4 | |     feature(auto_traits, negative_impls, try_trait_v2)
    | |_^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#literal_string_with_formatting_args
    = note: `-D clippy::literal-string-with-formatting-args` implied by `-D warnings`
    = help: to override `-D warnings` add `#[allow(clippy::literal_string_with_formatting_args)]`

clin1234 · 2025-01-17T19:34:22Z

@davidhewitt Might need your set of eyeballs, something has changed significantly in regards to 3.14 with strings representation(?)

davidhewitt · 2025-01-18T10:06:45Z

This is a great example of the kind of churn which premature 3.14 support will burden us with.

The layout of PyUnicodeObject will have changed internally, so I suspect that PyUnicode::data is now UB.

Personally I'm disinterested in fixing it for an alpha which might yet churn again, I suggest just disabling this API on 3.14 with a note that we would prefer not to reintroduce it.

ZeroIntensity · 2025-01-18T18:41:07Z

It's great to test in alphas, but PyO3 probably isn't one of the projects that should do that, at least for the non-stable APIs; we mess with the internals all the time in alphas. I'd wait until the beta freeze (or to at least get close to it) before trying to test 3.14.

I also personally don't think PyO3 support for it would help catch too many bugs:

Bugs tend to be in the new features, not the old ones that have already been tested. Unless you're planning on trying to also add PyO3 support for all the new C API functions (which I really don't recommend), then it's unlikely that the early birds will catch that much. (And for the minority that do catch regressions, it's probably not worth the burden on PyO3.)
The other issue is that there's a bit of a programming language barrier between Rust developers and C(Python) developers. There have been several Rust-related reports in the past where it's difficult to tell whether something is a user-error, a PyO3 issue, or an actual core problem. I suspect introducing PyO3 support for the 3.14 alphas will cause quite a bit of noise, and we just don't have enough Rust experts on our end to efficiently triage all the issues.
Tangentially related to that point, I don't know how much PyO3 exposes implementation details. If you guys expose/use some private APIs on your end, we're bound to get reports about it if we break them.

All that said, it might be worth trying to support 3.14 early for the limited API only, because we have to ensure stability for that anyway. I don't know how feasible that is for you guys, though.

clin1234 · 2025-01-24T23:01:35Z

CI passes, might need to consolidate commits

codspeed-hq · 2025-02-13T02:35:29Z

CodSpeed Performance Report

Merging #4811 will not alter performance

_{Comparing clin1234:main (87a6522) with main (28dc143)}

Summary

✅ 88 untouched benchmarks

ngoldbaum · 2025-02-20T18:27:00Z

Good news! I was able to fix ffi-check on the GIL-enabled build. There are still a ton of FFI updates that need to be done that aren't checked by the ffi-check macros, but at least there aren't any ABI mismatches any more.

I got part way through updating the FFI for the GIL-disabled build but there have been some pretty substantial changes to refcounting (at least, there might be more) and it's requiring a bit more of a fine-toothed perusal of the headers to make sure everything is right.

I'll go ahead and push just the updates for the GIL-enabled build and will come back another day with the GIL-disabled updates as well as some substantial refactoring to reflect upstream refactoring and changes.

ngoldbaum · 2025-02-20T18:28:02Z

btw, you may want to redo this PR so it isn't from the main branch on your fork, unless you don't mind me pushing to main to update this PR.

ngoldbaum · 2025-02-20T19:19:26Z

but there have been some pretty substantial changes to refcounting

Looking again, I think it's mostly just a refactor? So maybe not so bad.

ngoldbaum · 2025-02-20T23:29:38Z

(accidentally posted this on the issue instead of here, sorry for the double notification)

The last pushes fix ffi-check on the free-threaded build but unfortunately it breaks it on the GIL-enabled build.

@davidhewitt it looks like CPython made a change that breaks ffi-check, similar to how it was broken in Python 3.12 - ob_refcnt changed representation once again. Now it's a union that contains an inline anonymous struct and I'm not sure how to modify the ffi-check macro to account for this.

See here inside CPython for what I'm talking about.

In ffi-check there's this code you wrote in 2023:

pyo3/pyo3-ffi-check/macro/src/lib.rs

Lines 148 to 155 in 241080a

    
           let bindgen_field_ident = if (pyo3_build_config::get().version >= PY_3_12) 
        
               && struct_name == "PyObject" 
        
               && field_name == "ob_refcnt" 
        
           { 
        
               // PyObject since 3.12 implements ob_refcnt as a union; bindgen creates 
        
               // an anonymous name for the field 
        
               Ident::new("__bindgen_anon_1", Span::call_site()) 
        
           } else if struct_name == "PyMemberDef" && field_name == "type_code" {

Do you happen to know how I should I modify it now that there's an inline struct in the union as well?

clin1234 · 2025-02-20T23:56:39Z

btw, you may want to redo this PR so it isn't from the main branch on your fork, unless you don't mind me pushing to main to update this PR.

Go ahead

clin1234 · 2025-03-13T17:31:22Z

(accidentally posted this on the issue instead of here, sorry for the double notification)

The last pushes fix ffi-check on the free-threaded build but unfortunately it breaks it on the GIL-enabled build.

@davidhewitt it looks like CPython made a change that breaks ffi-check, similar to how it was broken in Python 3.12 - ob_refcnt changed representation once again. Now it's a union that contains an inline anonymous struct and I'm not sure how to modify the ffi-check macro to account for this.

See here inside CPython for what I'm talking about.

In ffi-check there's this code you wrote in 2023:

pyo3/pyo3-ffi-check/macro/src/lib.rs

Lines 148 to 155 in 241080a

let bindgen_field_ident = if (pyo3_build_config::get().version >= PY_3_12)

&& struct_name == "PyObject"

&& field_name == "ob_refcnt"

{

// PyObject since 3.12 implements ob_refcnt as a union; bindgen creates

// an anonymous name for the field

Ident::new("__bindgen_anon_1", Span::call_site())

} else if struct_name == "PyMemberDef" && field_name == "type_code" {

Do you happen to know how I should I modify it now that there's an inline struct in the union as well?

Any tips on handling no-GIL specifics within the FFI check?

ngoldbaum · 2025-03-13T17:34:42Z

You'll need to install a free-threaded Python build and run ffi-check locally. See https://py-free-threading.github.io/installing_cpython/.

Any discrepancies flagged by the tests relative to the headers should be corrected.

That said, it looks like everything passed on the last run? The free-threaded tests run ffi-check.

clin1234 · 2025-03-13T17:41:34Z

You'll need to install a free-threaded Python build and run ffi-check locally. See https://py-free-threading.github.io/installing_cpython/.

Any discrepancies flagged by the tests relative to the headers should be corrected.

That said, it looks like everything passed on the last run? The free-threaded tests run ffi-check.

Sorry, I mean to clarify how to handle them within the macro itself...

davidhewitt

Thanks for all the work here.

I think the HangThread change is correct; to me it looks like other changes in CPython have made it possible to crash the interpreter when allocating a new thread state during finalization. I reported this in python/cpython#132948

I think it would be good to merge this branch and then iterate further on main there's just a small number of corrections that need to be made, also the check-feature-powerset job has correctly identified the abi3-py314 feature needs to be added to the root Cargo.toml.

davidhewitt · 2025-04-25T12:41:02Z

pyo3-build-config/Cargo.toml

 abi3-py313 = ["abi3"]
+abi3-py314 = ["abi3"]


Suggested change

abi3-py313 = ["abi3"]

abi3-py314 = ["abi3"]

abi3-py313 = ["abi3-py314"]

abi3-py314 = ["abi3"]

davidhewitt · 2025-04-25T12:41:50Z

pyo3-ffi/Cargo.toml

@@ -35,6 +35,7 @@ abi3-py310 = ["abi3-py311", "pyo3-build-config/abi3-py310"]
 abi3-py311 = ["abi3-py312", "pyo3-build-config/abi3-py311"]
 abi3-py312 = ["abi3-py313", "pyo3-build-config/abi3-py312"]
 abi3-py313 = ["abi3", "pyo3-build-config/abi3-py313"]


Suggested change

abi3-py313 = ["abi3", "pyo3-build-config/abi3-py313"]

abi3-py313 = ["abi3-py314", "pyo3-build-config/abi3-py313"]

davidhewitt · 2025-04-25T12:43:16Z

pyo3-ffi/src/cpython/code.rs

We can hopefully just revert the changes in this file now that #5064 is merged

davidhewitt · 2025-04-25T12:43:50Z

pyo3-ffi/src/cpython/dictobject.rs

    pub ma_version_tag: u64,
+    #[cfg(Py_3_14)]
+    pub _ma_watcher_tag: u64,


Let's make this private field inaccessible unless we have good reason to allow it.

Suggested change

pub _ma_watcher_tag: u64,

_ma_watcher_tag: u64,

davidhewitt · 2025-04-25T12:45:39Z

pyo3-ffi/src/cpython/initconfig.rs

+    pub thread_inherit_context: c_int,
+    #[cfg(Py_3_14)]
+    pub context_aware_warnings: c_int,
+    // FIXME: this was backported to 3.13.2


And reverted in 3.13.3 :)

Suggested change

// FIXME: this was backported to 3.13.2

pytests/pytest.ini

davidhewitt · 2025-04-25T12:50:46Z

pyo3-ffi/src/refcount.rs

+use std::sync::atomic::Ordering::Relaxed;
+
+#[cfg(Py_3_14)]
+pub const _Py_STATICALLY_ALLOCATED_FLAG: c_int = 1 << 7;


Suggested change

pub const _Py_STATICALLY_ALLOCATED_FLAG: c_int = 1 << 7;

const _Py_STATICALLY_ALLOCATED_FLAG: c_int = 1 << 7;

(or at most pub(crate))

davidhewitt · 2025-04-25T12:51:23Z

pyo3-ffi/src/refcount.rs

+#[cfg(Py_GIL_DISABLED)]
+pub const _Py_IMMORTAL_REFCNT_LOCAL: u32 = u32::MAX;
+
+#[cfg(Py_GIL_DISABLED)]
+pub const _Py_REF_SHARED_SHIFT: isize = 2;


Suggested change

#[cfg(Py_GIL_DISABLED)]

pub const _Py_IMMORTAL_REFCNT_LOCAL: u32 = u32::MAX;

#[cfg(Py_GIL_DISABLED)]

pub const _Py_REF_SHARED_SHIFT: isize = 2;

#[cfg(Py_GIL_DISABLED)]

const _Py_IMMORTAL_REFCNT_LOCAL: u32 = u32::MAX;

#[cfg(Py_GIL_DISABLED)]

const _Py_REF_SHARED_SHIFT: isize = 2;

davidhewitt · 2025-04-25T12:54:09Z

pyo3-ffi/src/refcount.rs

+    ((6 as c_long) << (28 as c_long)) as Py_ssize_t;
+
+#[cfg(all(Py_3_14, Py_GIL_DISABLED))]
+pub const _Py_IMMORTAL_INITIAL_REFCNT: Py_ssize_t = c_uint::MAX as Py_ssize_t;


Suggested change

pub const _Py_IMMORTAL_INITIAL_REFCNT: Py_ssize_t = c_uint::MAX as Py_ssize_t;

const _Py_IMMORTAL_INITIAL_REFCNT: Py_ssize_t = c_uint::MAX as Py_ssize_t;

pyo3-ffi/src/refcount.rs

ngoldbaum · 2025-04-25T16:34:35Z

@clin1234 hope you don't mind - I'm going to apply David's comments and try to get this into a mergeable state today.

* Introspection: add function signatures No annotations or explicit default values yet Fixes an issue related to object identifiers path * Better default value * Refine arguments struct * Introduce VariableLengthArgument * Adds pyfunctions tests * Adds some serialization tests

ngoldbaum · 2025-04-25T17:27:19Z

I think the HangThread change is correct; to me it looks like other changes in CPython have made it possible to crash the interpreter when allocating a new thread state during finalization. I reported this in python/cpython#132948

In the meantime, what should we do here to avoid a segfault when the tests finish?

davidhewitt

I think for now let's just mark the pytests step as continue-on-error for dev versions, like we do for the whole job? That way we'll still upload coverage.

Also I think we should probably move ffi-check to run before all testing, because I would think if the ffi definitions are broken then ffi-check would help stop segfaults caused our end.

davidhewitt · 2025-04-26T06:18:56Z

pytests/pytest.ini

@@ -0,0 +1,3 @@
+# see gh-5094 for details


Maybe use a PyO3 issue URL, I think I might get confused with a cpython issue otherwise.

ngoldbaum · 2025-04-26T17:33:57Z

Alright, I think this is ready now.

davidhewitt

Thank you both for doing so much hard work here!

* Add news item * Move news file * Fix version limit check in noxfile.py * Bump Python version for testing debug builds * 3.14 is available from GH's setup-python action * Bump maximum supported CPython version in pyo3-ffi * Rework PyASCIIObject and PyUnicodeObject to be compatible with 3.14 Due to python/cpython#128196, data types within `PyASCIIObject.state` have changed, resulting in test failures when building against 3.14. * Run `cargo fmt --all` * Actually add Py_3_14 as a legitimate macro When `rustc` is invoked, the macro is included with the `--check-cfg` flag, but not with the `--cfg` flag. This caused errors about duplicate definitions to spew out when building with stable Rust toolchains. * Revert "Actually add Py_3_14 as a legitimate macro" This reverts commit 5da57af. * Fix version macro placement for 3.14-specific getters and setters * Import 'c_ushort' only if compiling against CPython 3.14 or later * Add wrapper functions for the statically_allocated field * Remove unused libc::c_ushort * Add (hopefully) final version-specific macros * Port 3.14-specific 64-bit code of Py_INCREF * Don't expose PyDictObject.ma_version_tag when building against 3.14 or later * fix ffi-check on the GIL-enabled ABI * fix older pythons * fix ffi-check on older pythons * WIP: update for 3.14t * fix ffi-check on the free-threaded build * fix clippy * fix clippy on older python versions * fix cargo check on the MSRV * fix ffi-check on 3.13t * fix CI which is using 3.13.1 * fix copy/paste error in noxfile * update ffi bindings for the latest changes in 3.14 * update layout of refcnt field on gil-enabled build * delete unused HangThread struct * fix ffi-check on GIL-enabled build * Revert "delete unused HangThread struct" This reverts commit 3dd439d. * config-out HangThread * fix 3.13 ffi-check * fix debug python build error * fix graalpy build * Ignore DeprecationWarnings from the pytest_asyncio module in tests * Add abi3-py314 * fix free-threading issue in `test_coroutine` (PyO3#5069) * Introspection: add function signatures (PyO3#5025) * Introspection: add function signatures No annotations or explicit default values yet Fixes an issue related to object identifiers path * Better default value * Refine arguments struct * Introduce VariableLengthArgument * Adds pyfunctions tests * Adds some serialization tests * respond to david's code review * add comment and fix test failure * fix check-feature-powerset * fix clippy * fix wasip1 clippy * fix 32 bit python 3.14 bug * mark test-py step continue-on-error for dev python builds * use github issue URL * run ffi-check before running tests * fix ffi-check for 3.14.0a7 --------- Co-authored-by: Nathan Goldbaum <[email protected]> Co-authored-by: David Hewitt <[email protected]> Co-authored-by: Thomas Tanon <[email protected]>

clin1234 force-pushed the main branch from 6c352a6 to 61f3fcc Compare December 23, 2024 13:02

clin1234 force-pushed the main branch from 225adb5 to 4ccc8ff Compare January 17, 2025 15:47

clin1234 mentioned this pull request Jan 18, 2025

Incompatible change in internal string representation (encountered with Rust bindings) (?) python/cpython#128972

Closed

clin1234 force-pushed the main branch 5 times, most recently from 4c94698 to ab16e04 Compare January 23, 2025 20:11

clin1234 force-pushed the main branch 3 times, most recently from e3447aa to df9a455 Compare February 13, 2025 02:20

clin1234 mentioned this pull request Feb 20, 2025

Tips on handling ffi-check? #4908

Closed

ngoldbaum mentioned this pull request Feb 27, 2025

Avoid data races in BorrowFlag #4948

Merged

clin1234 force-pushed the main branch 2 times, most recently from 8cef308 to 471bd35 Compare March 10, 2025 22:24

davidhewitt mentioned this pull request Apr 25, 2025

Crash with PyGILState_Ensure called from background thread during finalization on Python 3.14.0a7 python/cpython#132948

Closed

davidhewitt reviewed Apr 25, 2025

View reviewed changes

davidhewitt and others added 4 commits April 25, 2025 10:35

fix free-threading issue in test_coroutine (PyO3#5069)

6880abd

Merge branch 'main' into 3.14-testing

6e67f1e

respond to david's code review

46a4dfc

ngoldbaum mentioned this pull request Apr 25, 2025

asyncio tests trigger deprecation warnings in 3.14 #5094

Closed

ngoldbaum added 3 commits April 25, 2025 11:27

add comment and fix test failure

6ced135

fix check-feature-powerset

dcdd9e6

fix clippy

e2f28e6

ngoldbaum added free-threading FFI labels Apr 25, 2025

ngoldbaum added 3 commits April 25, 2025 12:12

fix wasip1 clippy

9889a86

fix 32 bit python 3.14 bug

0c97f89

Merge branch 'main' into 3.14-testing

840cf88

davidhewitt reviewed Apr 26, 2025

View reviewed changes

ngoldbaum added 4 commits April 26, 2025 09:14

mark test-py step continue-on-error for dev python builds

6d06135

use github issue URL

c059f39

run ffi-check before running tests

191ae30

fix ffi-check for 3.14.0a7

87a6522

davidhewitt approved these changes Apr 26, 2025

View reviewed changes

davidhewitt added this pull request to the merge queue Apr 26, 2025

Merged via the queue into PyO3:main with commit 5635b0f Apr 26, 2025
93 of 96 checks passed

clin1234 mentioned this pull request Apr 28, 2025

Add support for CPython 3.14 PyO3/maturin#2574

Closed

cdce8p mentioned this pull request May 23, 2025

Update pyo3 to 0.25.0 and add support for Python 3.14.0b2 ijl/orjson#570

Open

	abi3-py313 = ["abi3", "pyo3-build-config/abi3-py313"]
	abi3-py313 = ["abi3-py314", "pyo3-build-config/abi3-py313"]

	pub const _Py_STATICALLY_ALLOCATED_FLAG: c_int = 1 << 7;
	const _Py_STATICALLY_ALLOCATED_FLAG: c_int = 1 << 7;

	pub const _Py_IMMORTAL_INITIAL_REFCNT: Py_ssize_t = c_uint::MAX as Py_ssize_t;
	const _Py_IMMORTAL_INITIAL_REFCNT: Py_ssize_t = c_uint::MAX as Py_ssize_t;

Bump supported cpython version to 3.14 for testing #4811

Bump supported cpython version to 3.14 for testing #4811

Uh oh!

Conversation

clin1234 commented Dec 21, 2024

Uh oh!

davidhewitt commented Dec 21, 2024

Uh oh!

clin1234 commented Jan 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clin1234 commented Jan 17, 2025

Uh oh!

davidhewitt commented Jan 18, 2025

Uh oh!

ZeroIntensity commented Jan 18, 2025

Uh oh!

clin1234 commented Jan 24, 2025

Uh oh!

codspeed-hq bot commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging #4811 will not alter performance

Summary

Uh oh!

ngoldbaum commented Feb 20, 2025

Uh oh!

ngoldbaum commented Feb 20, 2025

Uh oh!

ngoldbaum commented Feb 20, 2025

Uh oh!

ngoldbaum commented Feb 20, 2025

Uh oh!

clin1234 commented Feb 20, 2025

Uh oh!

clin1234 commented Mar 13, 2025

Uh oh!

ngoldbaum commented Mar 13, 2025

Uh oh!

clin1234 commented Mar 13, 2025

Uh oh!

davidhewitt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ngoldbaum commented Apr 25, 2025

Uh oh!

ngoldbaum commented Apr 25, 2025

Uh oh!

davidhewitt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ngoldbaum commented Apr 26, 2025

Uh oh!

davidhewitt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

clin1234 commented Jan 17, 2025 •

edited

Loading

codspeed-hq bot commented Feb 13, 2025 •

edited

Loading