Skip to content

Crash with 3.13 which persisted into 3.14 #158312

Description

@ZattsPlace

Crash report

What happened?

I have an AI agent that attempted to fix a crash in Python 3.13 and this is the report it gave me.

CPython 3.14.7 segfault in _DK_ENTRIES / dict insertion (split-keys path)

Components: Interpreter Core (Objects/dictobject.c, Include/internal/pycore_dict.h)
Version: Python 3.14.7 (main, Aug 10 2026, 07:46:56) [GCC 16.1.1 20260728]
OS: Linux (Arch Linux, x86-64)
Interpreter: /usr/bin/python3.14 (system package, python 3.14.7-1 from pacman, NOT a uv-built interpreter — debug symbols available via debuginfod)
Triplet: x86_64-linux-gnu

Summary

CPython 3.14.7 segfaults inside _DK_ENTRIES() in pycore_dict.h:252 during a dict
insertion via the split-keys lookup path (unicodekeys_lookup_unicode → insert_split_key
→ insertdict). The dk (keys table) pointer passed to _DK_ENTRIES is 0x64 — a tiny
non-zero offset from NULL — causing a SEGV_MAPERR (accessing unmapped memory at address
0x64). This is a corrupted dict internal pointer, likely from a use-after-free or memory
corruption in the split-keys dict implementation.

Steps to reproduce

Reproducible by running the Hermes Agent CLI (hermes) under Python 3.14.7. The crash
occurs during startup, inside a class __init__ that performs a dict.__setitem__ call.
The exact Python-level trigger is unknown because the segfault prevents a Python traceback,
but the C-level backtrace is fully symbolized.

Command line that triggered the crash:

/usr/bin/python3 /home/zatt/.local/share/mise/installs/pipx-hermes-agent/latest/bin/hermes

Hermes version: 0.19.0 (pipx-installed, Python 3.14 site-packages)

Machine details:

  • Arch Linux, kernel (stock)
  • CPU: x86-64, 24 cores
  • RAM: 62 GiB (plenty free — no OOM)
  • Boot ID: 35e3d5a8b45040bfb3e1f64466561d78

Expected behavior

dict.__setitem__ completes normally during class initialization. No crash.

Actual behavior

Process receives SIGSEGV (signal 11, SEGV_MAPERR) at libpython3.14.so.1.0 offset
0x18a542. The crashing instruction dereferences dk->dk_log2_index_bytes where dk = 0x64.

Backtrace (fully symbolized via debuginfod)

#0  _DK_ENTRIES (dk=<optimized out>) at ./Include/internal/pycore_dict.h:252
    252→ size_t index = (size_t)1 << dk->dk_log2_index_bytes;

#1  do_lookup (mp=0x0, dk=0x64, key=<optimized out>, hash=<optimized out>, check_lookup=<optimized out>) at Objects/dictobject.c:1005

#2  unicodekeys_lookup_unicode (dk=dk@entry=0x64, key=key@entry=0x7d8754ea62b0,
       hash=-915039993034951925) at Objects/dictobject.c:1098

#3  insert_split_key (keys=0x64, key=0x7d8754ea62b0,
       hash=-915039993034951925) at Objects/dictobject.c:1846

#4  insertdict.isra.0 (mp=mp@entry=0x7d8754ed50b0, key=0x7d8754ea62b0,
       hash=-915039993034951925, value=0x7d8754e46100, interp=<optimized out>) at Objects/dictobject.c:1900

#5  setitem_take2_lock_held (mp=0x7d8754ed50b0, key=<optimized out>, value=<optimized out>) at Objects/dictobject.c:2699

#6  _PyDict_SetItem_Take2 (mp=0x7d8754ed50b0, key=<optimized out>, value=<optimized out>) at Objects/dictobject.c:2707

#7  _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=0x7d8768014ac8, throwflag=<optimized out>) at Python/generated_cases.c.h:9988
    → calling _PyDict_SetItem_Take2

#8  _PyEval_EvalFrame (throwflag=0, tstate=0x576ec8f711b0, frame=0x7d8768014448) at ./Include/internal/pycore_ceval.h:120

#9–21  Python call chain: _PyEval_Vector → _PyFunction_Vectorcall → _PyObject_VectorcallDictTstate
        → _PyObject_Call_Prepend → call_method → slot_tp_init → type_call → _PyObject_MakeTpCall
        → ... more eval frames ...

The crashing frame (#0) has these notable registers:

  • rdi = 0x64 — the dk pointer (corrupted: should be a valid dict keys table address)
  • rbx = rdx = r12 = r14 = 0xf34d1ff973e0330b — the hash value being looked up
  • rsi = 0x7d8754ea62b0 — the Unicode key object

Register state at crash (frame #0)

rax  = 0x64          (same as dk — the corrupted pointer)
rbx  = 0xf34d1ff973e0330b  (hash value, -915039993034951925 as signed)
rcx  = 0x7d8754e46100  (value being inserted)
rdx  = 0xf34d1ff973e0330b  (hash, duplicate in rdx)
rdi  = 0x64          (dk pointer — THIS IS THE BUG)
rsi  = 0x7d8754ea62b0  (key object)
r10  = 0x64          (duplicate of dk)
r15  = 0x64          (duplicate of dk)

Note that rdi, r10, and r15 all hold 0x64 — this is the corrupted dk pointer
being shuffled between registers before the dereference at _DK_ENTRIES.

Thread context

The crashing thread (TID 5567, "Thread-5 (proce") is one of 7 threads in the process.
Other threads at crash time:

  • Thread 5063 (main): blocked in epoll_wait via select module — the event loop
  • Thread 5065: blocked in _PySemaphore_Wait / _PyParkingLot_Park (queue module)
  • Thread 5566: blocked in clock_nanosleep
  • Thread 5568: blocked in _PyMutex_LockTimed (lock contention)
  • Threads 5697, 5569, 5698: similar semaphore/wait states

No other thread was in a dict operation. The crash is isolated to thread 5567 doing a
dict insertion during class __init__.

What I've ruled out

  1. OOM / resource exhaustion: 57 GiB available RAM, 125 GiB swap free. Not OOM.
  2. Arch/Omarchy system issue: This is a CPython interpreter bug, not an OS issue.
    The system Python from pacman (3.14.7-1) is the interpreter; no local patches.
  3. uv-built Python: This is the system /usr/bin/python3.14, not a uv-built
    interpreter. Debug symbols resolve cleanly via debuginfod.
  4. Third-party C extension: The loaded modules include standard library extensions
    (select, _socket, _ssl, _sqlite3, _decimal, etc.) and PyYAML's C module. The crash
    is in core CPython dict code, not in an extension. The Python code triggering it is
    a class __init__ doing a dict setitem — pure Python + CPython internals.

Additional context

  • Hermes Agent: Open-source AI agent CLI (https://github.com/nousresearch/hermes —
    note: this is the Hermes Agent you're using to chat with me right now). Version 0.19.0,
    installed via pipx. The crash happens at startup during internal initialization.
  • Core dump: Available at /var/lib/systemd/coredump/core.hermes.1000.35e3d5a8b45040bfb3e1f64466561d78.5063.1790528814000000.zst
    (29.2 MB, Zstandard-compressed, present on disk)
  • Core dumpctl info: coredumpctl info 5063 shows full details
  • Reproducibility: Single crash observed at this timestamp. Earlier crashes of
    python3.14 today (04:07–04:25 CDT) were from different processes and the uv-built
    interpreter — not directly related.

Possible fix directions

The dk pointer being 0x64 suggests either:

  1. A dict object's dk field was freed and the memory reused, leaving a stale small
    non-NULL pointer (use-after-free in split-keys dict lifecycle)
  2. A dict was partially initialized and a lookup/insertion was attempted on it
  3. Memory corruption elsewhere overwrote the dk field with a small value

The split-keys path (unicodekeys_lookup_unicode → insert_split_key) is the
"split dict" optimization for dicts with all-Unicode keys. This is commonly used for
object __dict__ and for kwargs dicts. Frame #14 is slot_tp_init — a class __init__
— so the dict in question is likely either the object's __dict__ or a kwargs dict
passed to __init__.

I did not attach the core dump to this issue because it contains the process's full
memory (potentially including sensitive data). I can provide it privately to a core
developer if needed — let me know the secure channel.

Files to attach (if useful)

  • The core dump is available on request (see note above about sensitivity)
  • coredumpctl info 5063 output can be pasted here
  • Full symbolized backtrace (already above)
  • Register state at crash (already above)
  • coredumpctl list showing the crash entry

Labels

type-bug, interpreter-core, 3.14, segfault, dict

# Add a code block here, if required

CPython versions tested on:

3.14, 3.13

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pendingThe issue will be closed if no feedback is providedtype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions