Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BZPOPMIN Errors #408

Open
yuranevmerzhitsky opened this issue Feb 18, 2025 · 1 comment
Open

BZPOPMIN Errors #408

yuranevmerzhitsky opened this issue Feb 18, 2025 · 1 comment

Comments

@yuranevmerzhitsky
Copy link

Hey!
I have a Django application using Django Channels, and I'm observing many errors in Datadog related to the Redis BZPOPMIN command. Does anyone have any ideas why I might be receiving these errors, how to avoid them, and what consequences might be associated with them?

About app:
I'm using a socket connection for real-time updates between the frontend and backend. So, nothing special is happening. The socket connection is established, and the frontend is receiving messages from the backend, and vice versa. The mobile client has the same behavior

Current Setup:

channels==4.0.0
channels-redis==4.2.0
Django==4.2.15
daphne==4.0.0

In settings.py:

CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {"hosts": [os.environ.get("REDIS_URL")],},
},
}

I start daphne with command:

daphne -b 0.0.0.0 -p 8000 app.asgi_ws:application

Infrastructure:

I'm using an AWS Network Load Balancer that redirects connections to a Daphne instance directly. I'm using AWS ElastiCache with a powerful instance. I noticed in the documentation that the library was tested with Redis, not with AWS ElastiCache specifically. So, I tried a self-hosted Redis instance. However, the error in the logs didn't disappear.

Logs

Image Image Image Image Image

I also noticed that the latency increases when I try to simulate many connections.

Questions:

  1. How critical are these errors? Perhaps I missed some settings? How to avoid them? Sometimes, I see in the logs that clients can't connect or reconnect, with status codes 1006 and 403 (Forbidden). Could these connection/reconnection issues with the status codes be related to the BZPOPMIN errors?
  2. I've tried RedisPubSubChannelLayer. With this layer, I don't see any errors (as it uses different Redis commands). I also observed that the latency is lower compared to RedisChannelLayer. However, the documentation states that RedisPubSubChannelLayer is in beta. How safe is it to use in production? Furthermore, since RedisPubSubChannelLayer uses different Redis commands, will the behavior be the same if we use this layer in production? I'm particularly concerned about message ordering, persistent connections, and so on. What are the recommended use cases for this layer?
    Thank you in advance for your help or any ideas you might have.
@carltongibson
Copy link
Member

Just quickly: The PubSub layer is fine to use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants