gh-155725: Remove PyGILState_Ensure usage from tracemalloc - #156409
gh-155725: Remove PyGILState_Ensure usage from tracemalloc#156409kumaraditya303 wants to merge 3 commits into
Conversation
tracemalloc no longer acquires the GIL nor creates a temporary thread state when tracing memory allocations. A thread with no attached thread state now records the trace with the "<unknown>" traceback instead of attaching a thread state to capture the Python traceback. Threads without a thread state used to pay for a GIL acquisition plus a full thread state creation and destruction on every traced raw allocation, only to record an empty traceback anyway.
Documentation build overview
|
…e attached Store traceback frame filenames as interned NUL terminated UTF-8 strings instead of Python str objects, so that capturing a traceback no longer uses or modifies Python objects. Threads without an attached thread state now capture their Python traceback by walking the frames of the thread state most recently bound to the thread; only threads which never had a thread state record the traceback as "<unknown>".
ZeroIntensity
left a comment
ZeroIntensity
left a comment
There was a problem hiding this comment.
Thanks, this is a much better approach.
| int kind = PyUnicode_KIND(obj); | ||
| const void *data = PyUnicode_DATA(obj); | ||
| Py_ssize_t length = PyUnicode_GET_LENGTH(obj); |
There was a problem hiding this comment.
Is it safe to access these without holding a thread state? I'm worried there might be some complications with stop-the-world and/or the GC.
There was a problem hiding this comment.
Yes, this is safe. obj here is the file name which has the same lifetime as code object referenced by a frame on the current thread. The frame can only be popped or deallocated by the current thread so the frame can't go away while we are tracking it in tracemalloc. The GC or the stw cannot collect it either because the current thread even though it might be suspended holds strong refs to these objects.
Also, in this function we take care to not call any API which might acquire critical section or locks, we just read the raw string data which is safe as long as we have a strong ref to it kept which in this case is kept by the frame stack.
2 participants
tracemalloc no longer acquires the GIL nor creates a temporary thread state when tracing memory allocations.
tracemallocraw-domain allocator hook violatesPYMEM_DOMAIN_RAWcontract #155725