Skip to content

[PUB-1825, PUB-1826] Add spec for applying incoming OBJECT messages #343

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 8, 2025
Merged

[PUB-1825, PUB-1826] Add spec for applying incoming OBJECT messages #343

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c091a57
Add Objects spec for applying incoming OBJECT messages
VeskeR Jul 7, 2025
84fc685
Better phrasing
VeskeR Sep 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 88 additions & 12 deletions textile/objects-features.textile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,18 +36,20 @@ h3(#realtime-objects). RealtimeObjects
** @(RTO4b)@ If the @HAS_OBJECTS@ flag is 0 or there is no @flags@ field, the sync sequence must be considered complete immediately, and the client library must perform the following actions in order:
*** @(RTO4b1)@ All objects except the one with id @root@ must be removed from the internal @ObjectsPool@
*** @(RTO4b2)@ The data for the @LiveMap@ with id @root@ must be cleared by setting it to a zero-value per "RTLM4":#RTLM4. Note that the client SDK must not create a new @LiveMap@ instance with id @root@; it must only clear the internal data of the existing @LiveMap@ with id @root@
*** @(RTO4b3)@ The @SyncObjectsPool@ must be cleared
*** @(RTO4b3)@ The @SyncObjectsPool@ list must be cleared
*** @(RTO4b5)@ The @BufferedObjectOperations@ list must be cleared
*** @(RTO4b4)@ Perform the actions for objects sync completion as described in "RTO5c":#RTO5c
* @(RTO5)@ The realtime system reserves the right to initiate an objects sync of the objects on a channel at any point once a channel is attached. A server initiated objects sync provides Ably with a means to send a complete list of objects present on the channel at any point
** @(RTO5d)@ If an @OBJECT_SYNC@ @ProtocolMessage@ is received and "@ObjectMessage.object@":../features#TR4r is null or omitted, the client library should skip processing that @ProtocolMessage@
** @(RTO5a)@ When an @OBJECT_SYNC@ @ProtocolMessage@ is received with a @channel@ attribute matching the channel name, the client library must parse the @channelSerial@ attribute:
*** @(RTO5a1)@ The @channelSerial@ is used as the sync cursor and is a two-part identifier: @<sequence id>:<cursor value>@
*** @(RTO5a2)@ If a new sequence id is sent from Ably, the client library must treat it as the start of a new objects sync sequence, and any previous in-flight sync must be discarded:
**** @(RTO5a2a)@ The current @SyncObjectsPool@ list must be cleared
**** @(RTO5a2a)@ The @SyncObjectsPool@ list must be cleared
**** @(RTO5a2b)@ The @BufferedObjectOperations@ list must be cleared
*** @(RTO5a3)@ If the sequence id matches the previously received sequence id, the client library should continue the sync process
*** @(RTO5a4)@ The objects sync sequence for that sequence identifier is considered complete once the cursor is empty; that is when the @channelSerial@ looks like @<sequence id>:@
*** @(RTO5a5)@ An @OBJECT_SYNC@ may also be sent with no @channelSerial@ attribute. In this case, the sync data is entirely contained within the @ProtocolMessage@
** @(RTO5b)@ During the sync sequence, the "@ObjectMessage.object@":../features#TR4r values from incoming @OBJECT_SYNC@ @ProtocolMessage@s must be temporarily stored in the internal @SyncObjectsPool@ list
** @(RTO5b)@ During the sync sequence, the "@ObjectMessage.object@":../features#TR4r values from incoming @OBJECT_SYNC@ @ProtocolMessages@ must be temporarily stored in the internal @SyncObjectsPool@ list
** @(RTO5c)@ When the objects sync has completed, the client library must perform the following actions in order:
*** @(RTO5c1)@ For each @ObjectState@ in the @SyncObjectsPool@ list:
**** @(RTO5c1a)@ If an object with @ObjectState.objectId@ exists in the internal @ObjectsPool@:
Expand All @@ -59,18 +61,52 @@ h3(#realtime-objects). RealtimeObjects
****** @(RTO5c1b1c)@ Otherwise, log a warning that an unsupported object state message has been received, and discard the current @ObjectState@ without taking any action
*** @(RTO5c2)@ Remove any objects from the internal @ObjectsPool@ for which @objectId@s were not received during the sync sequence
**** @(RTO5c2a)@ The object with ID @root@ must not be removed from @ObjectsPool@, as per "RTO3b":#RTO3b
*** @(RTO5c6)@ @ObjectMessages@ stored in the @BufferedObjectOperations@ list are applied as described in "RTO9":#RTO9
*** @(RTO5c3)@ Clear any stored sync sequence identifiers and cursor values
*** @(RTO5c4)@ The @SyncObjectsPool@ must be cleared
*** @(RTO5c5)@ The @BufferedObjectOperations@ list must be cleared
* @(RTO6)@ Certain object operations may require creating a zero-value object if one does not already exist in the internal @ObjectsPool@ for the given @objectId@. This can be done as follows:
** @(RTO6a)@ If an object with @objectId@ exists in @ObjectsPool@, do not create a new object
** @(RTO6b)@ The expected type of the object can be inferred from the provided @objectId@:
*** @(RTO6b1)@ Split the @objectId@ (formatted as @type:hash&#64;timestamp@) on the separator @:@ and parse the first part as the type string
*** @(RTO6b2)@ If the parsed type is @map@, create a zero-value @LiveMap@ per "RTLM4":#RTLM4 in the @ObjectsPool@
*** @(RTO6b3)@ If the parsed type is @counter@, create a zero-value @LiveCounter@ per "RTLC4":#RTLC4 in the @ObjectsPool@
* @(RTO7)@ The client library may receive @OBJECT@ @ProtocolMessages@ in realtime over the channel concurrently with @OBJECT_SYNC@ @ProtocolMessages@ during the object sync sequence ("RTO5":#RTO5). Some of the incoming @OBJECT@ messages may have already been applied to the objects described in the sync sequence, while others may not. Therefore, the client must buffer @OBJECT@ messages during the sync sequence so that it can determine which of them should be applied to the objects once the sync is complete. See "RTO8":#RTO8
** @(RTO7a)@ An internal @BufferedObjectOperations@ should be used to store the buffered @ObjectMessages@, as described in "RTO8a":#RTO8a. @BufferedObjectOperations@ is an array of @ObjectMessage@ instances
*** @(RTO7a1)@ This array is empty upon @RealtimeObjects@ initialization
* @(RTO8)@ When the library receives a @ProtocolMessage@ with an action of @OBJECT@, each member of the @ProtocolMessage.state@ array (decoded into @ObjectMessage@ objects) is passed to the @RealtimeObjects@ instance per "RTL1":../features#RTL1. Each @ObjectMessage@ from @OBJECT@ @ProtocolMessage@ (also referred to as an @OBJECT@ message) describes an operation to be applied to an object on a channel and must be handled as follows:
** @(RTO8a)@ If an object sync sequence is currently in progress, add the @ObjectMessages@ to the internal @BufferedObjectOperations@ array
** @(RTO8b)@ Otherwise, apply the @ObjectMessages@ as described in "RTO9":#RTO9
* @(RTO9)@ @OBJECT@ messages can be applied to @RealtimeObjects@ in the following way:
** @(RTO9a)@ For each @ObjectMessage@ in the provided list:
*** @(RTO9a1)@ If @ObjectMessage.operation@ is null or omitted, log a warning indicating that an unsupported object operation message has been received, and discard the current @ObjectMessage@ without taking any action
*** @(RTO9a2)@ The @ObjectMessage.operation.action@ field (see "@ObjectOperationAction@":../features#OOP2) determines the type of operation to apply:
**** @(RTO9a2a)@ If @ObjectMessage.operation.action@ is one of the following: @MAP_CREATE@, @MAP_SET@, @MAP_REMOVE@, @COUNTER_CREATE@, @COUNTER_INC@, or @OBJECT_DELETE@, then:
***** @(RTO9a2a1)@ If it does not already exist, create a zero-value @LiveObject@ in the internal @ObjectsPool@ per "RTO6":#RTO6 using the @objectId@ from @ObjectMessage.operation.objectId@
***** @(RTO9a2a2)@ Get the @LiveObject@ instance from the internal @ObjectsPool@ using the @objectId@ from @ObjectMessage.operation.objectId@
***** @(RTO9a2a3)@ Apply the @ObjectMessage.operation@ to the @LiveObject@; see "RTLC7":#RTLC7, "RTLM15":#RTLM15
**** @(RTO9a2b)@ Otherwise, log a warning that an object operation message with an unsupported action has been received, and discard the current @ObjectMessage@ without taking any action

h3(#liveobject). LiveObject

* @(RTLO1)@ The @LiveObject@ represents the common interface and includes shared functionality for concrete object types
* @(RTLO2)@ The client library may choose to implement @LiveObject@ as an abstract class
* @(RTLO3)@ @LiveObject@ properties:
** @(RTLO3a)@ protected @objectId@ string - an Object ID for this object
*** @(RTLO3a1)@ Must be provided and set in the constructor
** @(RTLO3b)@ protected @siteTimeserials@ @Dict<String, String>@ - a map of "serials":../features#OM2h keyed by "siteCode":../features#OM2i, representing the last operations applied to this object
*** @(RTLO3b1)@ Set to an empty map when the @LiveObject@ is initialized, so that any future operation can be applied to this object
** @(RTLO3c)@ protected @createOperationIsMerged@ boolean - a flag indicating whether the corresponding @MAP_CREATE@ or @COUNTER_CREATE@ operation has been applied to this @LiveObject@ instance
*** @(RTLO3c1)@ Set to @false@ when the @LiveObject@ is initialized
* @(RTLO4)@ @LiveObject@ methods:
** @(RTLO4a)@ protected @canApplyOperation@ - a convenience method used to determine whether the @ObjectMessage.operation@ should be applied to this object based on a serial value:
*** @(RTLO4a1)@ Expects the following arguments:
**** @(RTLO4a1a)@ @ObjectMessage@
*** @(RTLO4a2)@ Returns a boolean indicating whether the operation should be applied to this object
*** @(RTLO4a3)@ Both @ObjectMessage.serial@ and @ObjectMessage.siteCode@ must be non-empty strings. Otherwise, log a warning that the object operation message has invalid serial values. The client library must not apply this operation to the object
*** @(RTLO4a4)@ Get the @siteSerial@ value stored for this @LiveObject@ in the @siteTimeserials@ map using the key @ObjectMessage.siteCode@
*** @(RTLO4a5)@ If the @siteSerial@ for this @LiveObject@ is null or an empty string, return true
*** @(RTLO4a6)@ If the @siteSerial@ for this @LiveObject@ is not an empty string, return true if @ObjectMessage.serial@ is greater than @siteSerial@ when compared lexicographically

h3(#livecounter). LiveCounter

Expand All @@ -86,9 +122,29 @@ h3(#livecounter). LiveCounter
** @(RTLC6a)@ Replace the private @siteTimeserials@ of the @LiveCounter@ with the value from @ObjectState.siteTimeserials@
** @(RTLC6b)@ Set the private flag @createOperationIsMerged@ to @false@
** @(RTLC6c)@ Set @data@ to the value of @ObjectState.counter.count@, or to 0 if it does not exist
** @(RTLC6d)@ If @ObjectState.createOp@ is present:
*** @(RTLC6d1)@ Add @ObjectState.createOp.counter.count@ to @data@, if it exists
*** @(RTLC6d2)@ Set the private flag @createOperationIsMerged@ to @true@
** @(RTLC6d)@ If @ObjectState.createOp@ is present, merge the initial value into the @LiveCounter@ as described in "RTLC10":#RTLC10, passing in the @ObjectState.createOp@ instance
*** @(RTLC6d1)@ This clause has been replaced by "RTLC10a":#RTLC10a
*** @(RTLC6d2)@ This clause has been replaced by "RTLC10b":#RTLC10b
* @(RTLC7)@ An @ObjectOperation@ from @ObjectMessage.operation@ can be applied to a @LiveCounter@ in the following way:
** @(RTLC7a)@ A client library may choose to implement this logic as a convenience method named @applyOperation@, which accepts an @ObjectMessage@ instance with an existing @ObjectMessage.operation@ object, with @ObjectMessage.operation.objectId@ matching the Object ID of this @LiveCounter@. This @ObjectMessage@ represents the operation to be applied to this @LiveCounter@
** @(RTLC7b)@ If @ObjectMessage.operation@ cannot be applied based on the result of "@LiveObject.canApplyOperation@":#RTLO4a, log a debug or trace message indicating that the operation cannot be applied because its serial value is not newer than the object's, and discard the @ObjectMessage@ without taking any action
** @(RTLC7c)@ Set the entry in the private @siteTimeserials@ map at the key @ObjectMessage.siteCode@ to equal @ObjectMessage.serial@
** @(RTLC7d)@ The @ObjectMessage.operation.action@ field (see "@ObjectOperationAction@":../features#OOP2) determines the type of operation to apply:
*** @(RTLC7d1)@ If @ObjectMessage.operation.action@ is set to @COUNTER_CREATE@, apply the operation as described in "RTLC8":#RTLC8, passing in @ObjectMessage.operation@
*** @(RTLC7d2)@ If @ObjectMessage.operation.action@ is set to @COUNTER_INC@, apply the operation as described in "RTLC9":#RTLC9, passing in @ObjectMessage.operation.counterOp@
*** @(RTLC7d3)@ Otherwise, log a warning that an object operation message with an unsupported action has been received, and discard the current @ObjectMessage@ without taking any action
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In ably-js, we throw error for spec id RTLC7d3 => https://github.com/ably/ably-js/blob/837afe6585f68bd9fabf6bf8fd14b723258e9b30/src/plugins/objects/livecounter.ts#L215

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ably-js implementation is incorrect, and I intend to change it.
Throwing an error does not adhere to the robustness principle that our SDKs should follow https://sdk.ably.com/builds/ably/specification/main/features/#RTF1, and ably-js LiveObjects implementation violates this principle in multiple places by throwing errors instead of logging a debug/warning message and ignoring the operation.

The spec is correct in this case

Copy link
Collaborator

@sacOO7 sacOO7 Sep 5, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, in case of kotlin, we catch all types of exceptions at one place where processing of incoming liveobject message starts.
https://github.com/ably/ably-java/blob/993b308f93090c4d6ccda4870ac21cf64f41f7b9/liveobjects/src/main/kotlin/io/ably/lib/objects/DefaultRealtimeObjects.kt#L231-L244

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can add an explicit liveobjects spec point for the same.

I don't think we should be that explicit in the spec. It is up to the implementation to implement it in the most convenient/idiomatic for the platform. If the spec instructs to log something at a specific point (and even that might change as a result of #374) it should be as concise as possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can add an explicit liveobjects spec point for the same.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think a spec point like that would be specific to LiveObjects; it would establish generic logging rules for the entire spec.
Feel free to create an issue or open a PR to clarify the logging procedure in the spec, but I don't believe it should be included in this PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ve added the same comment to the previous issue: #362
One of the problems was that ably-js was throwing an internal exception, which ideally should be caught. This could potentially happen in other SDKs unknowingly or runtime exceptions can be thrown. Introducing an explicit spec point for such a mechanism would help improve the overall robustness and consistency of the SDKs.

* @(RTLC8)@ A @COUNTER_CREATE@ operation can be applied to a @LiveCounter@ in the following way:
** @(RTLC8a)@ Expects the following arguments:
*** @(RTLC8a1)@ @ObjectOperation@
** @(RTLC8b)@ If the private flag @createOperationIsMerged@ is @true@, log a debug or trace message indicating that the operation will not be applied because a @COUNTER_CREATE@ operation has already been applied to this @LiveCounter@, and discard the operation without taking any further action
** @(RTLC8c)@ Otherwise merge the initial value into the @LiveCounter@ as described in "RTLC10":#RTLC10, passing in the @ObjectOperation@ instance
* @(RTLC9)@ A @COUNTER_INC@ operation can be applied to a @LiveCounter@ in the following way:
** @(RTLC9a)@ Expects the following arguments:
*** @(RTLC9a1)@ @ObjectsCounterOp@
** @(RTLC9b)@ Add @ObjectsCounterOp.amount@ to @data@, if it exists
* @(RTLC10)@ The initial value from @ObjectOperation.counter@ can be merged into this @LiveCounter@ in the following way:
** @(RTLC10a)@ Add @ObjectOperation.counter.count@ to @data@, if it exists
** @(RTLC10b)@ Set the private flag @createOperationIsMerged@ to @true@

h3(#livemap). LiveMap

Expand Down Expand Up @@ -139,12 +195,27 @@ h3(#livemap). LiveMap
** @(RTLM6a)@ Replace the private @siteTimeserials@ of the @LiveMap@ with the value from @ObjectState.siteTimeserials@
** @(RTLM6b)@ Set the private flag @createOperationIsMerged@ to @false@
** @(RTLM6c)@ Set @data@ to @ObjectState.map.entries@, or to an empty map if it does not exist
** @(RTLM6d)@ If @ObjectState.createOp@ is present:
*** @(RTLM6d1)@ For each key–@ObjectsMapEntry@ pair in @ObjectState.createOp.map.entries@:
**** @(RTLM6d1a)@ If @ObjectsMapEntry.tombstone@ is @false@ or omitted, apply the @MAP_SET@ operation to the current key as described in "RTLM7":#RTLM7, passing in @ObjectsMapEntry.data@ and the current key as @ObjectsMapOp@, and @ObjectsMapEntry.timeserial@ as @serial@
**** @(RTLM6d1b)@ If @ObjectsMapEntry.tombstone@ is @true@, apply the @MAP_REMOVE@ operation to the current key as described in "RTLM8":#RTLM8, passing in the current key as @ObjectsMapOp@, and @ObjectsMapEntry.timeserial@ as @serial@
*** @(RTLM6d2)@ Set the private flag @createOperationIsMerged@ to @true@
* @(RTLM7)@ @MAP_SET@ operation for a key can be applied to a @LiveMap@ in the following way:
** @(RTLM6d)@ If @ObjectState.createOp@ is present, merge the initial value into the @LiveMap@ as described in "RTLM17":#RTLM17, passing in the @ObjectState.createOp@ instance
*** @(RTLM6d1)@ This clause has been replaced by "RTLM17a":#RTLM17a
**** @(RTLM6d1a)@ This clause has been replaced by "RTLM17a1":#RTLM17a1
**** @(RTLM6d1b)@ This clause has been replaced by "RTLM17a2":#RTLM17a2
*** @(RTLM6d2)@ This clause has been replaced by "RTLM17b":#RTLM17b
* @(RTLM15)@ An @ObjectOperation@ from @ObjectMessage.operation@ can be applied to a @LiveMap@ in the following way:
** @(RTLM15a)@ A client library may choose to implement this logic as a convenience method named @applyOperation@, which accepts an @ObjectMessage@ instance with an existing @ObjectMessage.operation@ object, with @ObjectMessage.operation.objectId@ matching the Object ID of this @LiveMap@. This @ObjectMessage@ represents the operation to be applied to this @LiveMap@
** @(RTLM15b)@ If @ObjectMessage.operation@ cannot be applied based on the result of "@LiveObject.canApplyOperation@":#RTLO4a, log a debug or trace message indicating that the operation cannot be applied because its serial value is not newer than the object's, and discard the @ObjectMessage@ without taking any action
** @(RTLM15c)@ Set the entry in the private @siteTimeserials@ map at the key @ObjectMessage.siteCode@ to equal @ObjectMessage.serial@
** @(RTLM15d)@ The @ObjectMessage.operation.action@ field (see "@ObjectOperationAction@":../features#OOP2) determines the type of operation to apply:
*** @(RTLM15d1)@ If @ObjectMessage.operation.action@ is set to @MAP_CREATE@, apply the operation as described in "RTLM16":#RTLM16, passing in @ObjectMessage.operation@
*** @(RTLM15d2)@ If @ObjectMessage.operation.action@ is set to @MAP_SET@, apply the operation as described in "RTLM7":#RTLM7, passing in @ObjectMessage.operation.mapOp@ and @ObjectMessage.serial@
*** @(RTLM15d3)@ If @ObjectMessage.operation.action@ is set to @MAP_REMOVE@, apply the operation as described in "RTLM8":#RTLM8, passing in @ObjectMessage.operation.mapOp@ and @ObjectMessage.serial@
*** @(RTLM15d4)@ Otherwise, log a warning that an object operation message with an unsupported action has been received, and discard the current @ObjectMessage@ without taking any action
* @(RTLM16)@ A @MAP_CREATE@ operation can be applied to a @LiveMap@ in the following way:
** @(RTLM16a)@ Expects the following argument:
*** @(RTLM16a1)@ @ObjectOperation@
** @(RTLM16b)@ If the private flag @createOperationIsMerged@ is @true@, log a debug or trace message indicating that the operation will not be applied because a @MAP_CREATE@ operation has already been applied to this @LiveMap@, and discard the operation without taking any further action
** @(RTLM16c)@ If the private @semantics@ field does not match @ObjectOperation.map.semantics@, log a warning that the operation cannot be applied due to mismatched semantics, and discard the operation without taking any further action
** @(RTLM16d)@ Otherwise merge the initial value into the @LiveMap@ as described in "RTLM17":#RTLM17, passing in the @ObjectOperation@ instance
* @(RTLM7)@ A @MAP_SET@ operation for a key can be applied to a @LiveMap@ in the following way:
** @(RTLM7d)@ Expects the following arguments:
*** @(RTLM7d1)@ @ObjectsMapOp@
*** @(RTLM7d2)@ @serial@ string - operation's serial value
Expand Down Expand Up @@ -178,3 +249,8 @@ h3(#livemap). LiveMap
** @(RTLM9c)@ If only the entry serial exists and is not an empty string, the missing operation serial is considered lower than the existing entry serial, so the operation must not be applied
** @(RTLM9d)@ If only the operation serial exists and is not an empty string, it is considered greater than the missing entry serial, so the operation can be applied
** @(RTLM9e)@ If both serials exist and are not empty strings, compare them lexicographically and allow operation to be applied only if the operation's serial is greater than the entry's serial
* @(RTLM17)@ The initial value from @ObjectOperation.map@ can be merged into this @LiveMap@ in the following way:
** @(RTLM17a)@ For each key–@ObjectsMapEntry@ pair in @ObjectOperation.map.entries@:
*** @(RTLM17a1)@ If @ObjectsMapEntry.tombstone@ is @false@ or omitted, apply the @MAP_SET@ operation to the current key as described in "RTLM7":#RTLM7, passing in @ObjectsMapEntry.data@ and the current key as @ObjectsMapOp@, and @ObjectsMapEntry.timeserial@ as @serial@
*** @(RTLM17a2)@ If @ObjectsMapEntry.tombstone@ is @true@, apply the @MAP_REMOVE@ operation to the current key as described in "RTLM8":#RTLM8, passing in the current key as @ObjectsMapOp@, and @ObjectsMapEntry.timeserial@ as @serial@
** @(RTLM17b)@ Set the private flag @createOperationIsMerged@ to @true@
Loading