Add CPython 3.15 support to mixed-mode debugging by rchiodo · Pull Request #8611 · microsoft/PTVS

Rich Chiodo (rchiodo) · 2026-08-13T20:55:43Z

Summary

Adds CPython 3.15 support to the mixed-mode (Python/Native) debugger, building on the 3.14 work in #8586. It makes the _Py_DebugOffsets reader version-aware so it can parse both the 3.14 and 3.15 offset-table layouts, and adds the V315 language version so the 3.14+ code paths (_PyStackRef masking, in-process line-number decoding) fire for 3.15.

Background

CPython 3.15 keeps the self-describing _Py_DebugOffsets table introduced/formalized in 3.14, but the flat layout of the table grew, which breaks a parser that assumes the fixed 3.14 layout:

thread_state gained 7 fields (3 inserted after current_frame: base_frame, last_profiled_frame, last_profiled_frame_seq; 4 appended after status: holds_gil, gil_requested, current_exception, exc_state) — 9 → 16 fields.
A new err_stackitem group (exc_value) is inserted between thread_state and interpreter_frame.
type_object gained tp_basicsize / tp_dictoffset; a new heap_type_object group (size, ht_cached_keys) follows it.
unicode_object gained compactunicodeobject_size; gc gained frame, generation_stats_size, generation_stats.

Because groups after thread_state shift, the flat offsets of interpreter_frame / code_object in the table differ between 3.14 (table size 760) and 3.15 (table size 888). The parser must select the layout by version.

The interpreter-frame field offsets, _PyStackRef decode (low-bit tag mask), and the co_linetable format are unchanged in production 3.15 builds, so the existing 3.14+ masking and line-number code applies to 3.15 as-is.

What this does

Version-aware _Py_DebugOffsets reader:

PyDebugOffsets now parses the header/version first, then selects the matching layout (Layout314, Layout315) built from shared field arrays plus version-specific arrays, and reads that layout's table size. Unknown versions are rejected.
Is314 is joined by Is315, and callers gate on the new IsSupported (3.14 or 3.15) instead of Is314. TableSizeFor(major, minor) replaces the single static table size; TryRead reads the larger MaxTableSize.

Language-version wiring (required):

PythonLanguageVersion.V315 = 0x030f and GetPythonLanguageVersion handles "315". Without this, LanguageVersion would be None, the >= V314 localsplus / f_executable stackref masks would never fire, and Debug.Assert(version != None) would trip — producing empty locals / broken frames exactly like pre-3.14.

No hot-path map change:

DebugOffsetsFieldProvider needs no mapping change: every hot-path struct/field it reads (_PyInterpreterFrame, PyCodeObject, _ts) exists in both layouts; only its doc comment was refreshed.

Files

Proxies/Structs/PyDebugOffsets.cs — version-aware layout (Layout314 760B / Layout315 888B), GetLayout, Is315, IsSupported, TableSizeFor, version-selecting TryParse, TryRead reads MaxTableSize.
PythonRuntimeInfo.cs — case "315" → V315; StructFieldOffsetProvider gates on IsSupported.
Common/Parsing/PythonLanguageVersion.cs — V315 = 0x030f.
Proxies/Structs/DebugOffsetsFieldProvider.cs — doc comment only (map unchanged).
Tests/DebuggerTests/PyDebugOffsetsTests.cs, PyDebugOffsetsProviderTests.cs — TableSizeFor; real recorded 3.15.0rc1 vectors (standard + free-threaded) with a full expected-offset table; a synthetic 3.15 ordinal test retained as an independent structural check; unknown-version rejection.

Testing

Unit tests pass (26/26 parser + provider tests).
The 3.15 coverage runs against real _Py_DebugOffsets tables recorded in-process from CPython 3.15.0rc1 — both the standard python315.dll and the free-threaded python315t.dll build (888-byte tables, version 0x30f00c1). The vectors were captured with ctypes id()-based semantic cross-checks (e.g. ob_type, co_linetable, co_firstlineno, tuple length, str length) so the recorded offsets are verified against live objects, not just self-consistent. The reader then parses those bytes back to the exact offsets the interpreter reports.
The free-threaded vector demonstrates the expected layout shifts the table is designed to absorb: co_linetable 136→152, co_firstlineno 68→84, ob_type 8→24, and the TLBC fields (tlbc_index, co_tlbc, tlbc_generation) non-zero only in the free-threaded build.
Manually verified end-to-end against the Examples/PythonNative C++ app embedding CPython 3.15 in VS 18: call stack, line numbers, stepping between Python and native code, and the Locals/Autos/Watch/globals windows (including nested frames) all work.

Notes

As with 3.14, only the mixed-mode hot path is driven by the table; struct sizes and all non-hot-path fields still come from the PDB. Free-threaded (3.15t) layout shifts are covered by the table automatically.
A synthetic 3.15 ordinal test is retained alongside the real vectors: it pins each group/field to the exact ordinal the CPython 3.15 header dictates, independent of the recorded values.

Extends the mixed-mode debugger's CPython 3.14 support to 3.15. The 3.15 _Py_DebugOffsets table grew (thread_state gained profiling/GIL/exception fields, a new err_stackitem and heap_type_object group were inserted, and type_object/unicode_object/gc gained fields), which shifts the byte positions of the frame and code-object groups the mixed-mode stack walk relies on. A 3.14-only flat parser would misread a 3.15 table, so the reader is now version-aware. - PythonLanguageVersion: add V315 (0x030f). - PythonRuntimeInfo: map python315(_d).dll -> V315; gate the debug-offsets field provider on IsSupported (3.14 or 3.15) instead of Is314. - PyDebugOffsets: parse the version prefix first, then select the matching ordered layout (3.14 vs 3.15) and consume that layout's table size. Unknown versions now return false so callers fall back to the PDB. Replace the static TableSize with TableSizeFor(major, minor). - DebugOffsetsFieldProvider: unchanged map (every hot-path group/field it reads exists in both layouts); refreshed doc. The existing '>= V314' stackref masks and in-process line-number path already fire for V315, and every version-gated struct proxy uses open-ended MinVersion gates, so 3.15 is covered without further changes. Tests: version-aware TableSizeFor; synthetic 3.15 structural tests asserting each group/field lands at the ordinal the CPython 3.15 header dictates; a 3.15 provider test; unknown-version rejection. A real recorded 3.15 vector will be added once a 3.15 build is available. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

azure-pipelines · 2026-08-13T20:55:53Z

Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run.

Heejae Chang (heejaechang) · 2026-08-13T20:56:24Z

🔒 Automated review in progress — Heejae Chang (@heejaechang) is auto-reviewing this PR.

Replace the synthetic-only 3.15 coverage with real _Py_DebugOffsets tables recorded in-process from CPython 3.15.0rc1 (standard python315.dll and free-threaded python315t.dll), captured with ctypes id()-based semantic cross-checks so the recorded offsets are verified against live objects. - RawV315 / RawV315T: real 888-byte tables (version 0x30f00c1). - ExpectedV315: full per-field offset table for the standard build. - Real parse/offset/free-threaded/TLBC tests mirroring the 3.14 cases; the free-threaded build shows the expected shifts (co_linetable 136->152, ob_type 8->24, TLBC fields non-zero). - Real provider hot-path + free-threaded shift tests. - Keep the synthetic ordinal test as an independent structural check. Confirms the version-aware reader parses real 3.15 memory to the same offsets the interpreter reports. 26/26 DebuggerTests offset tests pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Heejae Chang (heejaechang) · 2026-08-13T21:09:04Z

GitHub cannot anchor PR review comments to unchanged lines in the diff. Falling back to a general PR comment for Python/Product/Debugger.Concord/Proxies/Structs/PyDebugOffsets.cs:L240.

Warning · Non-blocking recommendation

Derive MaxTableSize from KnownLayouts so a future larger layout cannot leave TryRead undersized. [verified]

Heejae Chang (heejaechang)

Approved via Review Center.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rich Chiodo (rchiodo) · 2026-08-13T22:14:12Z

Addressed the unanchored MaxTableSize feedback in a76e643: the value is now derived by iterating KnownLayouts, so future larger layouts automatically increase the live-read buffer.

Stella Huang (StellaHuang95)

Approved via Review Center.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

sonarqubecloud · 2026-08-13T23:00:56Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Heejae Chang (heejaechang) · 2026-08-13T23:07:07Z

+                PointerProxy.RemoveTagBits(taggedExecutable,
+                    PyInterpreterFrame.GetStackReferenceTagMask(PythonLanguageVersion.V315)));
+        }
+


Warning · Non-blocking recommendation

This verifies the extracted helpers directly, so it would still pass if f_code, tagged pointer reads, or localsplus stopped applying them. Add the closest feasible consumer-level or Glass integration test that feeds a tagged 3.15 value through the production dereference path.

[verified]

Rich Chiodo (rchiodo) · 2026-08-13T23:18:21Z

/azp run

azure-pipelines · 2026-08-13T23:18:34Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Rich Chiodo (rchiodo) requested a review from a team as a code owner August 13, 2026 20:55

Rich Chiodo (rchiodo) mentioned this pull request Aug 13, 2026

Add CPython 3.15 support to mixed-mode debugging rchiodo/PTVS#1

Closed