Description
I'm trying to understand how the Rust client handles management of the last read position in a stream so that readers can continue from the last read position.
Suppose the following scenario:
- Client
A
writes data tos1
. - Client
B
creates reader grouprg1
and readerr1
. B
reads froms1
.B
crashes.- Client
A
continues to write data tos1
. B
restarts, creates reader grouprg1
and readerr1
.B
starts reading from last read position in the stream.
If I understand correctly, I can start consuming a stream from a StreamCut
, but what isn't clear to me from looking at the source code is how I would create a stream cut in the first place. Is there an API for this?
And if I do have a stream cut, is it my responsibility to persist it somehow or does Pravega take care of this for me?
It's also not clear to me how this relates to checkpointing?
If I wanted to add client C
in another process/machine as another reader in reader group rg1
, how can I do this? There isn't an API call for joining an existing reader group. Should I use the create_reader_group
API call, possibly with an existing ReaderGroupConfig
(which I assume it is my responsibility to serialise and transmit from B
to C
somehow) or is there a more correct approach to this?
Any help given in these areas is much appreciated.