Add support for throttled updates to ProxySQL servers #22
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A few new options will be introduced by the following commits.
If enabled, binlog reader will push updates to ProxySQL clients ever X milliseconds, with all GTID events ocurring in that period.
When update message buffers are too big writes to ProxySQL clients fail, as binlog reader tries to perform these in a single write() call. This behavior is particularly noticeable when -t is enabled, and causes ProxySQL to disconnect, and immediately connect back. This commit reworks writes to batch in chunks of a maximum configurable size - currently 4kb.
23fb45d introduced a hard limit of max 2Kb write queues, which can be (easly) hit with throttled updates. This commit fixes the issue, and also reworks `Client_Data` memory management logic to avoid fragementation.
|
Hi @proton-lisandro-pin . |
Appreciated, thanks! If this PR is accepted, i'm planning to looking into combining GTIDs for batched updates to reduce bandwidth usage as well. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The MySQL Binlog Reader currently pushes an update to every connected ProxySQL server every time it sees a GTID change; for complex database setups, this can have a significant impact in network usage.
This merge request implements a new option (
-t) to batch and write updates every x milliseconds, instead of immediately after every GTID event.