gh-131591: Implement PEP 768 #131592

pablogsal · 2025-03-22T20:05:40Z

Issue: Implement PEP 768 – Safe external debugger interface for CPython #131591

pablogsal · 2025-03-22T20:06:09Z

CC: @godlygeek @ivonastojanovic

Fix syntax error

ghost · 2025-03-25T16:00:03Z

All commit authors signed the Contributor License Agreement.

Python/initconfig.c

godlygeek

Initial observations (I've looked at everything but the interesting stuff, haven't gotten to Python/remote_debugging.c or Lib/test/test_sys.py yet)

Include/cpython/pystate.h

Include/internal/pycore_debug_offsets.h

Include/internal/pycore_interp_structs.h

Python/ceval_gil.c

Python/sysmodule.c

The PyRuntime section can be found in either the main executable (python.exe) or the Python DLL (pythonXY.dll, where X and Y represent the major and minor version numbers). Scan the modules of the process to locate the PyRuntime address in one of these modules.

Ensure that the caller and target processes have matching architectures before proceeding. Attaching to a 64-bit process from a 32-bit process will fail when using CreateToolhelp32Snapshot, and attempting to attach to a 32-bit process from a 64-bit process may cause issues during PE file parsing. To avoid potential errors abort the operation if the architectures differ.

This reverts commit 746ecfc. That commit isn't correct. It conflates the return from `IsWow64Process` (whether the process is running under 32-bit emulation on a 64-bit process) with whether the process is 64-bit. A 32-bit process on 32-bit Windows would have `IsWow64Process` set our `isWow64` flag to `FALSE`, and we'd then incorrectly return `TRUE` from `is_process64Bit`.

…_support External debugger Windows support

This is useful because debuggers will write to this variable remotely, and it's helpful for it to have a well known size rather than one that could vary per platform.

Require the flags for turning this on to be set to exactly 1 to avoid accidentally triggering remote debugging in the case of heap corruption. Make a heap copy of the script path before using it to avoid the buffer being overwritten while we're still using it by another debugger.

This avoids the need to reopen it by (wide character) path.

Previously it was leaked if `PyObject_AsFileDescriptor` failed.

I think this was a copy paste error.

The remote process must be a compatible version. Explain our compatibility rules.

We can clean this up by freeing strings as soon as we're done with them, and remove some duplicate code.

This was documented as working, but in fact we assumed that the path would always be a unicode string. Factor the version that only accepts a unicode string into a helper function, and have the caller call `os.fsdecode` on the input path, which accepts either a byte string or a unicode string and always returns a unicode string.

Previously we were returning the number of bytes, which we didn't need and would never be different than the argument that was passed into the function.

This field in the structure is defined as an `unsigned long`, not as a `pid_t`.

We already did a `strlen` on the `debugger_script_path`, so it's too late to check if it's NULL here!

This helps to harden a bit against heap corruption.

Lib/test/test_sys.py

bedevere-bot · 2025-03-31T12:45:49Z

🤖 New build scheduled with the buildbot fleet by @pablogsal for commit 80856d3 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F131592%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

pablogsal · 2025-03-31T12:53:54Z

I'm going to close this and reopen because the CI got stuck somehow

pablogsal · 2025-03-31T12:54:17Z

#131937

pythongh-131591: Implement PEP 768

6f6b4cd

bedevere-app bot mentioned this pull request Mar 22, 2025

Implement PEP 768 – Safe external debugger interface for CPython #131591

Open

pablogsal and others added 6 commits March 23, 2025 00:39

Add tests and use io.open_code

9b86022

Some fixes

af84100

Prepare for windows support

19ef7ae

Update windows files

444453c

Fix syntax error

1d3ad3c

Merge pull request #97 from ivonastojanovic/fix_windows_files

eeec1f6

Fix syntax error

godlygeek reviewed Mar 26, 2025

View reviewed changes

Python/initconfig.c Show resolved Hide resolved

godlygeek reviewed Mar 26, 2025

View reviewed changes

Python/initconfig.c Show resolved Hide resolved

godlygeek reviewed Mar 26, 2025

View reviewed changes

ivonastojanovic and others added 17 commits March 28, 2025 18:14

Read process memory on Windows

96798c3

Write to process memory on Windows

075ca65

Merge pull request #98 from ivonastojanovic/external_debugger_windows…

6076548

…_support External debugger Windows support

Remove unused struct member

a9d3ea9

Set an explicit size for a struct member

e235e62

This is useful because debuggers will write to this variable remotely, and it's helpful for it to have a well known size rather than one that could vary per platform.

Revert an accidental whitespace change

d51dda0

Factor running a debugger script into a helper function

38a4d51

Use _fdopen to open the debugger script on Windows

997b557

This avoids the need to reopen it by (wide character) path.

Ensure the debugger script is always closed

f6dec59

Previously it was leaked if `PyObject_AsFileDescriptor` failed.

Fix incorrect docstring

d273c5b

I think this was a copy paste error.

Document restrictions for remote_exec

c9a2146

The remote process must be a compatible version. Explain our compatibility rules.

Simplify handling of Windows paths

4af1744

We can clean this up by freeing strings as soon as we're done with them, and remove some duplicate code.

godlygeek added 10 commits March 28, 2025 21:04

Refactor to avoid duplicate error handling

c8779cd

Check for debug offset compatibility before using the offsets

6889042

Give a some variables shorter names

7f7aa8b

Have read_memory/write_memory return 0 on success

fbecfdb

Previously we were returning the number of bytes, which we didn't need and would never be different than the argument that was passed into the function.

Improve an error message

fa98f64

Remove a debugging printf

5b4cb00

Fix a remote read to use the correct type

0dd7797

This field in the structure is defined as an `unsigned long`, not as a `pid_t`.

Improve the error message when we can't find the requested thread

b8a0503

Remove a useless check

9344d1d

We already did a `strlen` on the `debugger_script_path`, so it's too late to check if it's NULL here!

Only accept a flag of 1 to mean remote debugging is enabled

9368d38

This helps to harden a bit against heap corruption.

pablogsal marked this pull request as ready for review March 29, 2025 11:38

pablogsal requested review from a team, ericsnowcurrently, erlend-aasland and corona10 as code owners March 29, 2025 11:38

bedevere-app bot added the awaiting core review label Mar 29, 2025

graingert reviewed Mar 29, 2025

View reviewed changes

Lib/test/test_sys.py Outdated Show resolved Hide resolved

graingert reviewed Mar 29, 2025

View reviewed changes

Lib/test/test_sys.py Outdated Show resolved Hide resolved

pablogsal added 5 commits March 31, 2025 11:35

Merge remote-tracking branch 'upstream/main'

166f4d6

Lint

d253966

simplify socket handling

8e04fdd

Add NEWS entry

0c2b275

Add more docs

80856d3

pablogsal added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 31, 2025

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 31, 2025

pablogsal closed this Mar 31, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-131591: Implement PEP 768 #131592

gh-131591: Implement PEP 768 #131592

pablogsal commented Mar 22, 2025 •

edited by bedevere-app bot

Loading

pablogsal commented Mar 22, 2025

ghost commented Mar 25, 2025 •

edited by ghost

Loading

godlygeek left a comment

bedevere-bot commented Mar 31, 2025

pablogsal commented Mar 31, 2025

pablogsal commented Mar 31, 2025

gh-131591: Implement PEP 768 #131592

gh-131591: Implement PEP 768 #131592

Conversation

pablogsal commented Mar 22, 2025 • edited by bedevere-app bot Loading

pablogsal commented Mar 22, 2025

ghost commented Mar 25, 2025 • edited by ghost Loading

godlygeek left a comment

Choose a reason for hiding this comment

bedevere-bot commented Mar 31, 2025

pablogsal commented Mar 31, 2025

pablogsal commented Mar 31, 2025

pablogsal commented Mar 22, 2025 •

edited by bedevere-app bot

Loading

ghost commented Mar 25, 2025 •

edited by ghost

Loading