Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating EventCounter takes a comparatively long time #103

Open
Schlepptop opened this issue Nov 25, 2022 · 2 comments
Open

Updating EventCounter takes a comparatively long time #103

Schlepptop opened this issue Nov 25, 2022 · 2 comments

Comments

@Schlepptop
Copy link

I am trying to improve the performance of my application, and I noticed that updating the eventcounter (which is done for every sent event) performs 3 DB Queries, which on my machine in total take about 7ms. I am sending lots of events, often to many channels, so these quickly add up.
Is there anything I can do to reduce the impact of updating the EventCounter?

@jkarneges
Copy link
Member

The built-in storage implementation (DjangoModelStorage) writes events to the database and may not be optimized for lots of events. Appending takes a row lock on the EventCounter model which hopefully doesn't block other rows, but I have not benchmarked if that's really true and it may depend on the database engine.

You have three options:

a) Try to optimize the current code and submit a PR. Maybe the transaction isn't written optimally.
b) Write your own StorageBase implementation. This would give you the most control for high scale, to do things like sharding or using non-SQL data sources.
c) Don't use event storage and treat event delivery as best-effort.

@Schlepptop
Copy link
Author

Thank you for your reply!
This doesn't help anyone else, but I have decided to solve this and other bottlenecks I faced by rewriting my high frequency async communication backend in golang.
Looking at the EventCounter model implementation, I noticed that it selects the value, increments it, and then updates, instead of doing one relative Update with an F() field, but I'm afraid I don't have an idea how to improve that if we need the value to create the zero_event if necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants