gh-127989: C API: Refer to "attached thread states" instead of the GIL (GH-127990)

Co-authored-by: Victor Stinner <[email protected]>

Author: ZeroIntensity

Doc/c-api/dict.rst

@@ -127,7 +127,7 @@ Dictionary Objects
       Prefer the :c:func:`PyDict_GetItemWithError` function instead.
 
    .. versionchanged:: 3.10
-      Calling this API without :term:`GIL` held had been allowed for historical
+      Calling this API without an :term:`attached thread state` had been allowed for historical
       reason. It is no longer allowed.
 
 

Doc/c-api/exceptions.rst

@@ -413,7 +413,7 @@ Querying the error indicator
    own a reference to the return value, so you do not need to :c:func:`Py_DECREF`
    it.
 
-   The caller must hold the GIL.
+   The caller must have an :term:`attached thread state`.
 
    .. note::
 
@@ -675,7 +675,7 @@ Signal Handling
 
    .. note::
       This function is async-signal-safe.  It can be called without
-      the :term:`GIL` and from a C signal handler.
+      an :term:`attached thread state` and from a C signal handler.
 
 
 .. c:function:: int PyErr_SetInterruptEx(int signum)
@@ -702,7 +702,7 @@ Signal Handling
 
    .. note::
       This function is async-signal-safe.  It can be called without
-      the :term:`GIL` and from a C signal handler.
+      an :term:`attached thread state` and from a C signal handler.
 
    .. versionadded:: 3.10
 

Doc/c-api/init.rst

@@ -578,8 +578,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option
    * ``list[str]``
    * ``dict[str, str]``
 
-   The caller must hold the GIL. The function cannot be called before
-   Python initialization nor after Python finalization.
+   The caller must have an :term:`attached thread state`. The function cannot
+   be called before Python initialization nor after Python finalization.
 
    .. versionadded:: 3.14
 
@@ -601,8 +601,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option
    * Return a new reference on success.
    * Set an exception and return ``NULL`` on error.
 
-   The caller must hold the GIL. The function cannot be called before
-   Python initialization nor after Python finalization.
+   The caller must have an :term:`attached thread state`. The function cannot
+   be called before Python initialization nor after Python finalization.
 
    .. versionadded:: 3.14
 
@@ -616,8 +616,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option
    * Raise a :exc:`ValueError` if the option is read-only (cannot be set).
    * Raise a :exc:`TypeError` if *value* has not the proper type.
 
-   The caller must hold the GIL. The function cannot be called before
-   Python initialization nor after Python finalization.
+   The caller must have an :term:`attached thread state`. The function cannot
+   be called before Python initialization nor after Python finalization.
 
    .. versionadded:: 3.14
 

Doc/c-api/init_config.rst

@@ -110,12 +110,12 @@ The three allocation domains are:
 
 * Raw domain: intended for allocating memory for general-purpose memory
   buffers where the allocation *must* go to the system allocator or where the
-  allocator can operate without the :term:`GIL`. The memory is requested directly
-  from the system. See :ref:`Raw Memory Interface <raw-memoryinterface>`.
+  allocator can operate without an :term:`attached thread state`. The memory
+  is requested directly from the system. See :ref:`Raw Memory Interface <raw-memoryinterface>`.
 
 * "Mem" domain: intended for allocating memory for Python buffers and
   general-purpose memory buffers where the allocation must be performed with
-  the :term:`GIL` held. The memory is taken from the Python private heap.
+  an :term:`attached thread state`. The memory is taken from the Python private heap.
   See :ref:`Memory Interface <memoryinterface>`.
 
 * Object domain: intended for allocating memory for Python objects. The
@@ -139,8 +139,8 @@ Raw Memory Interface
 ====================
 
 The following function sets are wrappers to the system allocator. These
-functions are thread-safe, the :term:`GIL <global interpreter lock>` does not
-need to be held.
+functions are thread-safe, so a :term:`thread state` does not
+need to be :term:`attached <attached thread state>`.
 
 The :ref:`default raw memory allocator <default-memory-allocators>` uses
 the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc`
@@ -213,8 +213,7 @@ The :ref:`default memory allocator <default-memory-allocators>` uses the
 
 .. warning::
 
-   The :term:`GIL <global interpreter lock>` must be held when using these
-   functions.
+   There must be an :term:`attached thread state` when using these functions.
 
 .. versionchanged:: 3.6
 
@@ -327,8 +326,7 @@ The :ref:`default object allocator <default-memory-allocators>` uses the
 
 .. warning::
 
-   The :term:`GIL <global interpreter lock>` must be held when using these
-   functions.
+   There must be an :term:`attached thread state` when using these functions.
 
 .. c:function:: void* PyObject_Malloc(size_t n)
 
@@ -485,12 +483,12 @@ Customize Memory Allocators
    zero bytes.
 
    For the :c:macro:`PYMEM_DOMAIN_RAW` domain, the allocator must be
-   thread-safe: the :term:`GIL <global interpreter lock>` is not held when the
-   allocator is called.
+   thread-safe: a :term:`thread state` is not :term:`attached <attached thread state>`
+   when the allocator is called.
 
    For the remaining domains, the allocator must also be thread-safe:
    the allocator may be called in different interpreters that do not
-   share a ``GIL``.
+   share a :term:`GIL`.
 
    If the new allocator is not a hook (does not call the previous allocator),
    the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the
@@ -507,8 +505,8 @@ Customize Memory Allocators
          :c:func:`Py_InitializeFromConfig` to install a custom memory
          allocator. There are no restrictions over the installed allocator
          other than the ones imposed by the domain (for instance, the Raw
-         Domain allows the allocator to be called without the GIL held). See
-         :ref:`the section on allocator domains <allocator-domains>` for more
+         Domain allows the allocator to be called without an :term:`attached thread state`).
+         See :ref:`the section on allocator domains <allocator-domains>` for more
          information.
 
        * If called after Python has finish initializing (after
@@ -555,7 +553,7 @@ Runtime checks:
   called on a memory block allocated by :c:func:`PyMem_Malloc`.
 - Detect write before the start of the buffer (buffer underflow).
 - Detect write after the end of the buffer (buffer overflow).
-- Check that the :term:`GIL <global interpreter lock>` is held when
+- Check that there is an :term:`attached thread state` when
   allocator functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex:
   :c:func:`PyObject_Malloc`) and :c:macro:`PYMEM_DOMAIN_MEM` (ex:
   :c:func:`PyMem_Malloc`) domains are called.
@@ -620,8 +618,8 @@ PYMEM_CLEANBYTE (meaning uninitialized memory is getting used).
    The :c:func:`PyMem_SetupDebugHooks` function now also works on Python
    compiled in release mode.  On error, the debug hooks now use
    :mod:`tracemalloc` to get the traceback where a memory block was allocated.
-   The debug hooks now also check if the GIL is held when functions of
-   :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are
+   The debug hooks now also check if there is an :term:`attached thread state` when
+   functions of :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are
    called.
 
 .. versionchanged:: 3.8

Doc/c-api/memory.rst

@@ -709,7 +709,7 @@ since multiple such modules can be created from a single definition.
    mechanisms (either by calling it directly, or by referring to its
    implementation for details of the required state updates).
 
-   The caller must hold the GIL.
+   The caller must have an :term:`attached thread state`.
 
    Return ``-1`` with an exception set on error, ``0`` on success.
 
@@ -720,6 +720,6 @@ since multiple such modules can be created from a single definition.
    Removes the module object created from *def* from the interpreter state.
    Return ``-1`` with an exception set on error, ``0`` on success.
 
-   The caller must hold the GIL.
+   The caller must have an :term:`attached thread state`.
 
    .. versionadded:: 3.3

Doc/c-api/module.rst

@@ -16,7 +16,7 @@ kernel/git/torvalds/linux.git/tree/tools/perf/Documentation/jit-interface.txt>`_
 In Python, these helper APIs can be used by libraries and features that rely
 on generating machine code on the fly.
 
-Note that holding the Global Interpreter Lock (GIL) is not required for these APIs.
+Note that holding an :term:`attached thread state` is not required for these APIs.
 
 .. c:function:: int PyUnstable_PerfMapState_Init(void)
 

Doc/c-api/perfmaps.rst

@@ -55,7 +55,7 @@ Reflection
 
 .. c:function:: PyFrameObject* PyEval_GetFrame(void)
 
-   Return the current thread state's frame, which is ``NULL`` if no frame is
+   Return the :term:`attached thread state`'s frame, which is ``NULL`` if no frame is
    currently executing.
 
    See also :c:func:`PyThreadState_GetFrame`.

Doc/c-api/reflection.rst

@@ -232,7 +232,7 @@ Operating System Utilities
 
    The file descriptor is created non-inheritable (:pep:`446`).
 
-   The caller must hold the GIL.
+   The caller must have an :term:`attached thread state`.
 
    .. versionadded:: 3.14
 
@@ -378,8 +378,8 @@ accessible to C code.  They all work with the current interpreter thread's
    silently abort the operation by raising an error subclassed from
    :class:`Exception` (other errors will not be silenced).
 
-   The hook function is always called with the GIL held by the Python
-   interpreter that raised the event.
+   The hook function is always called with an :term:`attached thread state` by
+   the Python interpreter that raised the event.
 
    See :pep:`578` for a detailed description of auditing.  Functions in the
    runtime and standard library that raise events are listed in the

Doc/c-api/sys.rst

@@ -56,7 +56,7 @@ range.
 system time.)
 
 As any other C API (unless otherwise specified), the functions must be called
-with the :term:`GIL` held.
+with an :term:`attached thread state`.
 
 .. c:function:: int PyTime_Monotonic(PyTime_t *result)
 
@@ -78,29 +78,29 @@ Raw Clock Functions
 -------------------
 
 Similar to clock functions, but don't set an exception on error and don't
-require the caller to hold the GIL.
+require the caller to have an :term:`attached thread state`.
 
 On success, the functions return ``0``.
 
 On failure, they set ``*result`` to ``0`` and return ``-1``, *without* setting
-an exception. To get the cause of the error, acquire the GIL and call the
-regular (non-``Raw``) function. Note that the regular function may succeed after
+an exception. To get the cause of the error, :term:`attach <attached thread state>` a :term:`thread state`,
+and call the regular (non-``Raw``) function. Note that the regular function may succeed after
 the ``Raw`` one failed.
 
 .. c:function:: int PyTime_MonotonicRaw(PyTime_t *result)
 
    Similar to :c:func:`PyTime_Monotonic`,
-   but don't set an exception on error and don't require holding the GIL.
+   but don't set an exception on error and don't require an :term:`attached thread state`.
 
 .. c:function:: int PyTime_PerfCounterRaw(PyTime_t *result)
 
    Similar to :c:func:`PyTime_PerfCounter`,
-   but don't set an exception on error and don't require holding the GIL.
+   but don't set an exception on error and don't require an :term:`attached thread state`.
 
 .. c:function:: int PyTime_TimeRaw(PyTime_t *result)
 
    Similar to :c:func:`PyTime_Time`,
-   but don't set an exception on error and don't require holding the GIL.
+   but don't set an exception on error and don't require an :term:`attached thread state`.
 
 
 Conversion functions

Doc/c-api/time.rst

@@ -731,7 +731,7 @@ and :c:data:`PyType_Type` effectively act as defaults.)
       object becomes part of a refcount cycle, that cycle might be collected by
       a garbage collection on any thread).  This is not a problem for Python
       API calls, since the thread on which :c:member:`!tp_dealloc` is called
-      will own the Global Interpreter Lock (GIL).  However, if the object being
+      with an :term:`attached thread state`.  However, if the object being
       destroyed in turn destroys objects from some other C or C++ library, care
       should be taken to ensure that destroying those objects on the thread
       which called :c:member:`!tp_dealloc` will not violate any assumptions of

Doc/c-api/typeobj.rst

@@ -403,8 +403,8 @@ the new attribute values.  We might be tempted, for example to assign the
 But this would be risky.  Our type doesn't restrict the type of the
 ``first`` member, so it could be any kind of object.  It could have a
 destructor that causes code to be executed that tries to access the
-``first`` member; or that destructor could release the
-:term:`Global interpreter Lock <GIL>` and let arbitrary code run in other
+``first`` member; or that destructor could detach the
+:term:`thread state <attached thread state>` and let arbitrary code run in other
 threads that accesses and modifies our object.
 
 To be paranoid and protect ourselves against this possibility, we almost
@@ -413,8 +413,8 @@ don't we have to do this?
 
 * when we absolutely know that the reference count is greater than 1;
 
-* when we know that deallocation of the object [#]_ will neither release
-  the :term:`GIL` nor cause any calls back into our type's code;
+* when we know that deallocation of the object [#]_ will neither detach
+  the :term:`thread state <attached thread state>` nor cause any calls back into our type's code;
 
 * when decrementing a reference count in a :c:member:`~PyTypeObject.tp_dealloc`
   handler on a type which doesn't support cyclic garbage collection [#]_.

Doc/extending/newtypes_tutorial.rst

@@ -132,6 +132,28 @@ Glossary
       iterator's :meth:`~object.__anext__` method until it raises a
       :exc:`StopAsyncIteration` exception.  Introduced by :pep:`492`.
 
+   attached thread state
+
+      A :term:`thread state` that is active for the current OS thread.
+
+      When a :term:`thread state` is attached, the OS thread has
+      access to the full Python C API and can safely invoke the
+      bytecode interpreter.
+
+      Unless a function explicitly notes otherwise, attempting to call
+      the C API without an attached thread state will result in a fatal
+      error or undefined behavior.  A thread state can be attached and detached
+      explicitly by the user through the C API, or implicitly by the runtime,
+      including during blocking C calls and by the bytecode interpreter in between
+      calls.
+
+      On most builds of Python, having an attached thread state implies that the
+      caller holds the :term:`GIL` for the current interpreter, so only
+      one OS thread can have an attached thread state at a given moment. In
+      :term:`free-threaded <free threading>` builds of Python, threads can concurrently
+      hold an attached thread state, allowing for true parallelism of the bytecode
+      interpreter.
+
    attribute
       A value associated with an object which is usually referenced by name
       using dotted expressions.
@@ -622,6 +644,10 @@ Glossary
       multi-threaded applications and makes it easier to use multi-core CPUs
       efficiently. For more details, see :pep:`703`.
 
+      In prior versions of Python's C API, a function might declare that it
+      requires the GIL to be held in order to use it. This refers to having an
+      :term:`attached thread state`.
+
    hash-based pyc
       A bytecode cache file that uses the hash rather than the last-modified
       time of the corresponding source file to determine its validity. See
@@ -1295,6 +1321,29 @@ Glossary
       See also :term:`binary file` for a file object able to read and write
       :term:`bytes-like objects <bytes-like object>`.
 
+   thread state
+
+      The information used by the :term:`CPython` runtime to run in an OS thread.
+      For example, this includes the current exception, if any, and the
+      state of the bytecode interpreter.
+
+      Each thread state is bound to a single OS thread, but threads may have
+      many thread states available.  At most, one of them may be
+      :term:`attached <attached thread state>` at once.
+
+      An :term:`attached thread state` is required to call most
+      of Python's C API, unless a function explicitly documents otherwise.
+      The bytecode interpreter only runs under an attached thread state.
+
+      Each thread state belongs to a single interpreter, but each interpreter
+      may have many thread states, including multiple for the same OS thread.
+      Thread states from multiple interpreters may be bound to the same
+      thread, but only one can be :term:`attached <attached thread state>` in
+      that thread at any given moment.
+
+      See :ref:`Thread State and the Global Interpreter Lock <threads>` for more
+      information.
+
    token
 
       A small unit of source code, generated by the