feat: HTTP SenderPool with asyncio support #66
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
A new API to make it easier to work with the sender asynchronously with true parallelism.
Details
flushrepresents an atomic database transaction.API downsides
creates silent network-blocking operations in the API.
Thread safety and Parallelism
pool.next_buffer()andpool.flush()methods are thread safe.N:MconcurrencyNbuffer writer threadsMthreads responsible for concurrently writing to the database (inside the pool).Tasks
SenderPoolinto newquestdb.ingress.poolmodule.TransactionalBuffer.dataframe(..).pool.next_buffer()andpool.flushimplementations.asyncio.wrap_futureindef flush()directly (since that's how it's implemented anyway): https://github.com/python/cpython/blob/8ad88984200b2ccddc0a08229dd2f4c14d1a71fc/Lib/asyncio/base_events.py#L896 - this allows implementing.flush()in terms of.flush_to_future()and cut code duplication.Closes #64