Skip to content

Conversation

snowsignal
Copy link

@snowsignal snowsignal commented Sep 8, 2025

This PR solves two problems with Recordset:

  1. Iteration/collection of scanned records is currently not workable in an async context because the Recordset iterator calls std::thread::yield_now() internally.
  2. An Arc<Recordset> can be sent between threads, and iterated over in multiple places at once, which would leave each thread with a partial iteration over the scanned records. Ideally, it should be impossible to iterate over the scanned records in more than one thread at once.

To solve these, I've implemented a new struct, RecordStream, that acts as an iterable wrapper over an Arc<Recordset>. It can be created by a new into_stream() method on Arc<Recordset>, and crucially, this will make at most one RecordStream available at any one time—trying to instantiate a second RecordStream will return None.

RecordStream implements both the synchronous Iterator trait that was previously implemented for &Recordset, but it also supports asynchronous iteration via futures::Stream.

It might be worth considering whether Client::scan should just return a RecordStream directly, though this can be done in a follow-up PR.

@khaf
Copy link
Collaborator

khaf commented Sep 9, 2025

Thank you for your feedback and PR. I need some clarification though:

  1. We have tests for scan and query that are running in a async context and clearly working. What am I missing? (the std::thread::yield_now() was added to aid sync client in consuming the records. If found to be an issue, I'll need to find another solution.)
  2. If by partial iteration you mean a multi-consumer setup, then that is by design. Or do you mean the iterator gets stuck?

Is possible to provide a gist to help me understand and reproduce the issue on my side? Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants