CRITICAL Listen failure: [Errno 88] Socket operation on non-socket #420

hardikbansal · 2019-05-21T18:48:18Z

hardikbansal
May 21, 2019

I have made a server using Django REST framework and Django channels but I am unable to configure supervisor to run with it. My supervisor config is as follow:

[program:frnd_django]
socket=tcp://localhost:8000
directory=/home/coldbrewtech/frnd/backend/dating-app-backend/django-server/cbproj/
environment=DJANGO_SETTINGS_MODULE=cbproj.dev_settings
command=/home/coldbrewtech/frnd/backend/env/bin/daphne -u /run/daphne/daphne%(process_num)d.sock --fd 0 --access-log - --proxy-headers cbproj.asgi:application
stdout_logfile=/home/coldbrewtech/frnd/backend/dating-app-backend/django-server/cbproj/logs/django_out.log
stderr_logfile=/home/coldbrewtech/frnd/backend/dating-app-backend/django-server/cbproj/logs/django_err.log
numprocs=4
process_name=asgi%(process_num)d
autostart=true
autorestart=true
user=root
startsecs=10
stopwaitsecs=600
stopasgroup=true
stopsignal=KILL

And the error that I am getting is as follow:

2019-05-21 18:03:50,111 INFO     Starting server at fd:fileno=0, unix:/run/daphne/daphne3.sock
2019-05-21 18:03:50,121 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
2019-05-21 18:03:50,125 INFO     Configuring endpoint fd:fileno=0
2019-05-21 18:03:50,130 INFO     Starting server at fd:fileno=0, unix:/run/daphne/daphne1.sock
2019-05-21 18:03:50,131 INFO     Starting server at fd:fileno=0, unix:/run/daphne/daphne0.sock
2019-05-21 18:03:50,132 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
2019-05-21 18:03:50,133 INFO     Configuring endpoint fd:fileno=0
2019-05-21 18:03:50,134 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
2019-05-21 18:03:50,145 INFO     Configuring endpoint fd:fileno=0
2019-05-21 18:03:50,154 INFO     Starting server at fd:fileno=0, unix:/run/daphne/daphne2.sock
2019-05-21 18:03:50,158 CRITICAL Listen failure: [Errno 88] Socket operation on non-socket
2019-05-21 18:03:50,159 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
2019-05-21 18:03:50,160 INFO     Configuring endpoint fd:fileno=0
2019-05-21 18:03:50,163 CRITICAL Listen failure: [Errno 88] Socket operation on non-socket
2019-05-21 18:03:50,160 INFO     Configuring endpoint unix:/run/daphne/daphne3.sock
2019-05-21 18:03:50,168 CRITICAL Listen failure: [Errno 88] Socket operation on non-socket
2019-05-21 18:03:50,177 INFO     Configuring endpoint unix:/run/daphne/daphne0.sock
2019-05-21 18:03:50,181 INFO     Configuring endpoint unix:/run/daphne/daphne1.sock
2019-05-21 18:03:50,192 CRITICAL Listen failure: [Errno 88] Socket operation on non-socket
2019-05-21 18:03:50,204 INFO     Configuring endpoint unix:/run/daphne/daphne2.sock

I am not sure whether this is daphne issue or some supervisor issue

carltongibson · 2019-06-05T07:38:25Z

carltongibson
Jun 5, 2019
Maintainer

Sorry, this isn't an actionable issue as-is. All I can really suggest is putting a breakpoint into Server.run() to see what's going wrong. (The supervisor process has the correct permissions etc?)

See the Channels docs for support options.

0 replies

SwordElucidator · 2020-02-18T05:50:09Z

SwordElucidator
Feb 18, 2020

@hardikbansal Hey, did you solve it?

0 replies

Surgo · 2020-02-24T10:33:53Z

Surgo
Feb 24, 2020

Same error on ubuntu buster (Docker).

root@5439c0483e8b:~# daphne --verbosity 3 --fd 0 example:application
2020-02-24 10:28:57,245 INFO     Starting server at fd:fileno=0
2020-02-24 10:28:57,246 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
2020-02-24 10:28:57,246 INFO     Configuring endpoint fd:fileno=0
2020-02-24 10:28:57,258 CRITICAL Listen failure: [Errno 88] Socket operation on non-socket
root@5439c0483e8b:~# daphne --verbosity 3 --endpoint fd:fileno=0 example:application
2020-02-24 10:29:23,778 INFO     Starting server at fd:fileno=0
2020-02-24 10:29:23,779 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
2020-02-24 10:29:23,780 INFO     Configuring endpoint fd:fileno=0
2020-02-24 10:29:23,796 CRITICAL Listen failure: [Errno 88] Socket operation on non-socket

I think it is not a supervisor's problem. It occurred daphne cli too. CC: @carltongibson

0 replies

Surgo · 2020-02-26T09:57:34Z

Surgo
Feb 26, 2020

It's cpython's bug in my case.
https://bugs.python.org/issue39685
Works fine with cpython 3.7.x.

0 replies

carltongibson · 2020-02-26T12:59:59Z

carltongibson
Feb 26, 2020
Maintainer

@Surgo can you try #298?

0 replies

Surgo · 2020-02-27T01:45:37Z

Surgo
Feb 27, 2020

@carltongibson Same error occured :(

root@8b08497ba8f9:/var/www/webapp# python --version
Python 3.8.1
root@8b08497ba8f9:/var/www/webapp# grep selectors /usr/local/lib/python3.8/site-packages/daphne/server.py
import selectors  # isort:skip
selector = selectors.SelectSelector()
root@8b08497ba8f9:/var/www/webapp# daphne --fd 0 example:application
2020-02-27 01:39:58,930 INFO     Starting server at fd:fileno=0
2020-02-27 01:39:58,932 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
2020-02-27 01:39:58,932 INFO     Configuring endpoint fd:fileno=0
2020-02-27 01:39:59,003 CRITICAL Listen failure: [Errno 88] Socket operation on non-socket
root@8b08497ba8f9:/var/www/webapp# daphne --endpoint fd:fileno=0 example:application
2020-02-27 01:40:53,491 INFO     Starting server at fd:fileno=0
2020-02-27 01:40:53,493 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
2020-02-27 01:40:53,493 INFO     Configuring endpoint fd:fileno=0
2020-02-27 01:40:53,511 CRITICAL Listen failure: [Errno 88] Socket operation on non-socket

0 replies

Surgo · 2020-02-27T02:09:09Z

Surgo
Feb 27, 2020

Maybe same issue that using socket.fromfd

Fails to connect to endpoints with Python 3.8 sshuttle/sshuttle#381 -> https://bugs.python.org/issue39685

0 replies

Surgo · 2020-02-27T02:48:43Z

Surgo
Feb 27, 2020

Its failure.printTraceback() on my env

root@3efa1ccb0e6e:/var/www/webapp# daphne --verbosity 3 --fd 0 example:application
2020-02-27 02:40:10,543 INFO     Starting server at fd:fileno=0
2020-02-27 02:40:10,548 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
2020-02-27 02:40:10,551 INFO     Configuring endpoint fd:fileno=0
2020-02-27 02:40:10,681 CRITICAL Listen failure: [Errno 88] Socket operation on non-socket
2020-02-27 02:40:10,682 ERROR    Traceback (most recent call last):
2020-02-27 02:40:10,803 ERROR      File "/usr/local/bin/daphne", line 8, in <module>
2020-02-27 02:40:10,805 ERROR        sys.exit(CommandLineInterface.entrypoint())
2020-02-27 02:40:10,807 ERROR      File "/usr/local/lib/python3.8/site-packages/daphne/cli.py", line 191, in entrypoint
2020-02-27 02:40:10,810 ERROR        cls().run(sys.argv[1:])
2020-02-27 02:40:10,817 ERROR      File "/usr/local/lib/python3.8/site-packages/daphne/cli.py", line 311, in run
2020-02-27 02:40:10,821 ERROR        self.server.run()
2020-02-27 02:40:10,824 ERROR      File "/usr/local/lib/python3.8/site-packages/daphne/server.py", line 125, in run
2020-02-27 02:40:10,826 ERROR        listener = ep.listen(self.http_factory)
2020-02-27 02:40:10,829 ERROR    --- <exception caught here> ---
2020-02-27 02:40:10,856 ERROR      File "/usr/local/lib/python3.8/site-packages/twisted/internet/endpoints.py", line 1264, in listen
2020-02-27 02:40:10,871 ERROR        port = self.reactor.adoptStreamPort(
2020-02-27 02:40:10,893 ERROR      File "/usr/local/lib/python3.8/site-packages/twisted/internet/posixbase.py", line 457, in adoptStreamPort
2020-02-27 02:40:10,909 ERROR        p = tcp.Port._fromListeningDescriptor(
2020-02-27 02:40:10,917 ERROR      File "/usr/local/lib/python3.8/site-packages/twisted/internet/tcp.py", line 1325, in _fromListeningDescriptor
2020-02-27 02:40:10,994 ERROR        port = socket.fromfd(fd, addressFamily, cls.socketType)
2020-02-27 02:40:10,997 ERROR      File "/usr/local/lib/python3.8/socket.py", line 544, in fromfd
2020-02-27 02:40:11,012 ERROR        return socket(family, type, proto, nfd)
2020-02-27 02:40:11,017 ERROR      File "/usr/local/lib/python3.8/socket.py", line 231, in __init__
2020-02-27 02:40:11,021 ERROR        _socket.socket.__init__(self, family, type, proto, fileno)
2020-02-27 02:40:11,026 ERROR    builtins.OSError: [Errno 88] Socket operation on non-socket

socket.fromfd args == (0, <AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 0)

0 replies

carltongibson · 2020-02-27T08:05:15Z

carltongibson
Feb 27, 2020
Maintainer

See #254. There's a comment there about needing to manually create the /run/daphne dir.
(There are some other comments too)

Let me know if that helps. Maybe there's a docs change we can make. (Not really a Daphne issue but if folks keep hitting it...)

0 replies

Surgo · 2020-02-28T07:35:45Z

Surgo
Feb 28, 2020

#254 is unix domain socket case and missing its directory.
This case are filedescriptor --fd 0 or --endpoint fd:fileno=0.
And I think that is not only daphne's trouble.... Python or twisted ???

>>> import socket
>>> socket.fromfd(0, socket.AF_INET, socket.SOCK_STREAM)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/socket.py", line 544, in fromfd
    return socket(family, type, proto, nfd)
  File "/usr/local/lib/python3.8/socket.py", line 231, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 88] Socket operation on non-socket

0 replies

carltongibson · 2020-02-28T12:45:02Z

carltongibson
Feb 28, 2020
Maintainer

Yeah, I don't think any of there issue are really Daphne issues per se. (More like OS and permissions issues.)

But folks keep turning up with them. We don't really get a resolution. And then something similar pops up a few months later. So if we could document the common issues it might smooth the wheel.

0 replies

ronnievdc · 2020-06-18T08:17:55Z

ronnievdc
Jun 18, 2020

I'm running into this issue myself. After some hours of investigating i found the following...

From what i understand is that supervisor fcgi is a process manager. It listens to a tcp socket and spawns child processes that handle the incoming traffic. Before spawning the child (daphne) it creates a socket on fd:0 for that specific process, so that process can listen on fd:0. So, only daphne instances spawned by supervisor can listen to fd:0.

Running daphne --fd 0 from commandline will fail. The socket fd:0 it is not created and the error message does reflect that.

I think most issues come up because the directory /run/daphne/ is not created (or not with the right permissions).

0 replies

carltongibson · 2020-06-18T08:53:21Z

carltongibson
Jun 18, 2020
Maintainer

Hi @ronnievdc Thanks for your comments. Do you think a note or admonition would be worthwhile in the guide https://github.com/django/channels/blob/master/docs/deploying.rst#nginxsupervisor-ubuntu ?

0 replies

ronnievdc · 2020-06-18T09:30:28Z

ronnievdc
Jun 18, 2020

Both, a mention about creating the /run/daphne/ dir with the correct permissions and not running daphne as standalone with --fd 0 would have saved me hours of researching. so a note would be great.

0 replies

carltongibson · 2020-06-18T09:52:02Z

carltongibson
Jun 18, 2020
Maintainer

Fancy making that PR? (Since it's fresh for you, you're probably Best Qualified™ ATM...)

0 replies

stevecj · 2022-06-30T10:24:14Z

stevecj
Jun 30, 2022

None of the responses here have mentioned that [program:frnd_django] should be [fcgi-program:frnd_django].

That said, I am having the same problem when I do use [fcgi-program:f<name>] as I am supposed to.

[fcgi-program:daphne]
priority = 810
autostart = false
startsecs = 10
startretries = 2
stopwaitsecs = 30

socket=tcp://127.0.0.1:8088

directory = /home/administrator/hub/2.2.1/django

# Each process needs to have a separate socket file, so use process_num
command = /home/administrator/hub/2.2.1/bin/daphne -u /home/administrator/hub/data/tmp/daphne%(process_num)d.sock --fd 0 --access-log - --proxy-headers designhub.asgi:application

# Number of instances
numprocs=2

# Give each process a unique name so they can be told apart
process_name=daphne_asgi%(process_num)d

stdout_logfile = /home/administrator/hub/2.2.1/logs/daphneso.log
stderr_logfile = /home/administrator/hub/2.2.1/logs/daphnese.log

main:daphne_asgi0: ERROR (spawn error)
main:daphne_asgi1: ERROR (spawn error)

INFO: daphne.cli Starting server at fd:fileno=0, unix:/home/administrator/hub/data/tmp/daphne1.sock; (cli.py:261) pr: MainProcess (16806)
INFO: daphne.server HTTP/2 support not enabled (install the http2 and tls Twisted extras); (server.py:113) pr: MainProcess (16806)
INFO: daphne.server Configuring endpoint fd:fileno=0; (server.py:122) pr: MainProcess (16806)
INFO: daphne.cli Starting server at fd:fileno=0, unix:/home/administrator/hub/data/tmp/daphne0.sock; (cli.py:261) pr: MainProcess (16805)
INFO: daphne.server HTTP/2 support not enabled (install the http2 and tls Twisted extras); (server.py:113) pr: MainProcess (16805)
INFO: daphne.server Configuring endpoint fd:fileno=0; (server.py:122) pr: MainProcess (16805)
^[[31m^[[47m[2022-06-30 02:54:50 PDT] CRITICAL: daphne.server Listen failure: [Errno 88] Socket operation on non-socket; (server.py:160) pr: MainProcess (16806)
INFO: daphne.server Configuring endpoint unix:/home/administrator/hub/data/tmp/daphne1.sock; (server.py:122) pr: MainProcess (16806)
CRITICAL: daphne.server Listen failure: [Errno 88] Socket operation on non-socket; (server.py:160) pr: MainProcess (16805)
INFO: daphne.server Configuring endpoint unix:/home/administrator/hub/data/tmp/daphne0.sock; (server.py:122) pr: MainProcess (16805)

2 replies

ronnievdc Jun 30, 2022

Did you follow the docs: https://github.com/django/channels/blob/main/docs/deploying.rst#nginxsupervisor-ubuntu
Especially the creation of the directory for the socket files with the right permissions

Can you post the output of the following commands
ls -al /home/administrator/hub/data/tmp/ and namei -l /home/administrator/hub/data/tmp/

stevecj Jul 8, 2022

I was using an existing directory owned by the same user that daphne runs as.

I'm testing a different build right now that is not trying to run daphne as fgci, so the output of ls will not be helpful right now, but

$ namei -l /home/administrator/hub/data/tmp/
f: /home/administrator/hub/data/tmp/
dr-xr-xr-x root          root          /
drwxr-xr-x root          root          home
drwx------ administrator administrator administrator
drwxr-xr-x administrator administrator hub
drwxr-xr-x administrator administrator data
drwxrwx--T administrator administrator tmp

The current user and user that supervisord & daphne run as is "administrator"

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CRITICAL Listen failure: [Errno 88] Socket operation on non-socket #420

{{title}}

Replies: 16 comments 2 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

CRITICAL Listen failure: [Errno 88] Socket operation on non-socket #420

Replies: 16 comments · 2 replies

carltongibson Jun 5, 2019 Maintainer

carltongibson Feb 26, 2020 Maintainer

carltongibson Feb 27, 2020 Maintainer

carltongibson Feb 28, 2020 Maintainer

carltongibson Jun 18, 2020 Maintainer

carltongibson Jun 18, 2020 Maintainer

Replies: 16 comments 2 replies

carltongibson
Jun 5, 2019
Maintainer

carltongibson
Feb 26, 2020
Maintainer

carltongibson
Feb 27, 2020
Maintainer

carltongibson
Feb 28, 2020
Maintainer

carltongibson
Jun 18, 2020
Maintainer

carltongibson
Jun 18, 2020
Maintainer