Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: SERVER socket is not compatible with asyncio #2058

Open
1 task done
steamraven opened this issue Jan 29, 2025 · 2 comments
Open
1 task done

BUG: SERVER socket is not compatible with asyncio #2058

steamraven opened this issue Jan 29, 2025 · 2 comments

Comments

@steamraven
Copy link

This is a pyzmq bug

  • This is a pyzmq-specific bug, not an issue of zmq socket behavior. Don't worry if you're not sure! We'll figure it out together.

What pyzmq version?

26.2.0 (pip --no-binary)

What libzmq version?

4.3.5 (bundled, built with DRAFT_API)

Python version (and how it was installed)

python 3.12.7 from Apt

OS

Ubuntu 24.10

What happened?

Immediate exception when creating a SERVER socket using an asyncio context

pyzmq installed via Draft Install Doc

Code to reproduce bug

import zmq
from zmq.asyncio import Context
context = Context()
socket = context.socket(zmq.SERVER)

Traceback, if applicable

Traceback (most recent call last):
  File "/home/vscode/.local/lib/python3.11/site-packages/zmq/sugar/attrsettr.py", line 56, in __getattr__
    return self._get_attr_opt(upper_key, opt)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/zmq/sugar/attrsettr.py", line 68, in _get_attr_opt
    return self.get(opt)
           ^^^^^^^^^^^^^
  File "_zmq.py", line 853, in zmq.backend.cython._zmq.Socket.get
  File "_zmq.py", line 1362, in zmq.backend.cython._zmq._getsockopt
  File "_zmq.py", line 179, in zmq.backend.cython._zmq._check_rc
zmq.error.ZMQError: Invalid argument

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspaces/mythic/mythic_server/test_zmq_cliserv.py", line 4, in <module>
    socket = context.socket(zmq.SERVER)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/zmq/sugar/context.py", line 354, in socket
    socket_class(  # set PYTHONTRACEMALLOC=2 to get the calling frame
  File "/home/vscode/.local/lib/python3.11/site-packages/zmq/_future.py", line 231, in __init__
    self._fd = self._shadow_sock.FD
               ^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/zmq/sugar/attrsettr.py", line 62, in __getattr__
    raise AttributeError(f"{key} attribute is write-only")
AttributeError: FD attribute is write-only

More info

This appears to be a problem with the underlying SERVER socket being thread-safe, and the _future._AsyncSocket requesting a FD on the shadow socket.

PYzmq requests the FD from the shadow socket here in pyzmq/zmq/_future.py
self._fd = self._shadow_sock.FD

self._fd = self._shadow_sock.FD

The SERVER socket is created thread-safe here in libzmq/src/server.cpp
socket_base_t (parent_, tid_, sid_, true),
https://github.com/zeromq/libzmq/blob/34f7fa22022bed9e0e390ed3580a1c83ac4a2834/src/server.cpp#L13

The zmq_getsockattr for FD returns error if threadsafe here in libzmq/src/socket_base.cpp

    if (option_ == ZMQ_FD) {
        if (_thread_safe) {
            // thread safe socket doesn't provide file descriptor
            errno = EINVAL;
            return -1;
        }

https://github.com/zeromq/libzmq/blob/34f7fa22022bed9e0e390ed3580a1c83ac4a2834/src/socket_base.cpp#L447C1-L452C10

Unfortunately, my skill in socket and async programming are not good enough to come up with any solutions.

@steamraven
Copy link
Author

My use case for SERVER vs ROUTER is drastically simplifies the behavior of dropped messages when high-water marks are hit

@steamraven
Copy link
Author

Sorry, duplicate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant