From 89b18f6bb138f95b08fef5f0dbc2c88f8a012c79 Mon Sep 17 00:00:00 2001 From: Francis Roberts <111994975+franrob-projects@users.noreply.github.com> Date: Tue, 23 Sep 2025 13:01:33 +0200 Subject: [PATCH 01/22] Can I delete messages which have been persisted through the history API? --- src/pages/docs/storage-history/storage.mdx | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/pages/docs/storage-history/storage.mdx b/src/pages/docs/storage-history/storage.mdx index 049f754672..5eaf4a970d 100644 --- a/src/pages/docs/storage-history/storage.mdx +++ b/src/pages/docs/storage-history/storage.mdx @@ -27,6 +27,10 @@ The time that messages will be stored for depends on your account package: There is a cost associated with storing messages for longer than the minimum time period. [Contact us](https://ably.com/support) to discuss enabling long term storage. +### Message deletion + +Ably does not currently provide an API to delete persisted messages from the history. Once messages are stored with persisted history enabled, they will remain for the entire configured storage period. If you need to delete specific messages from history, please [contact us](https://ably.com/support) to discuss your requirements. + Messages can be retrieved using the [history](/docs/storage-history/history) feature. This is illustrated in the following diagram: ![Persist All Messages](../../../images/content/diagrams/history-persist-all-messages.png) From 8ba78757b07a6d065f593b8ff1f322c9bc7ad15c Mon Sep 17 00:00:00 2001 From: Francis Roberts <111994975+franrob-projects@users.noreply.github.com> Date: Tue, 23 Sep 2025 13:13:26 +0200 Subject: [PATCH 02/22] Is it possible to check if a specific message has been delivered to a device? --- src/pages/docs/messages/index.mdx | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/pages/docs/messages/index.mdx b/src/pages/docs/messages/index.mdx index e30609c3ce..511c4f9306 100644 --- a/src/pages/docs/messages/index.mdx +++ b/src/pages/docs/messages/index.mdx @@ -34,6 +34,12 @@ The following are the properties of a message: | **extras** | A JSON object of arbitrary key-value pairs that may contain metadata, and/or ancillary payloads. Valid payloads include those related to [Push Notifications](/docs/push), [deltas](/docs/channels/options/deltas) and headers | | **encoding** | This is typically empty, as all messages received from Ably are automatically decoded client-side using this value. However, if the message encoding cannot be processed, this attribute contains the remaining transformations not applied to the data payload | +## Message delivery tracking + +You can ensure a message was successfully published by checking the [history](/docs/storage-history/history) of the channel for your message. However, it is only possible to check if a device has received a message from the device itself. + +Ably does not store per-message delivery logs, nor logs of who is subscribed to a channel at any point in time. This means it is not possible to check which users have received messages retroactively. + ## Message conflation Use message conflation to ensure that clients only ever receive the most up-to-date message, by removing redundant and outdated messages. Message conflation will aggregate published messages for a set period of time and evaluate all messages against a [conflation key](#routing). All but the latest message for each conflation key value will be discarded, and the resulting message, or messages, will be delivered to subscribers as a single batch once the period of time elapses. From 2102407072f42819a0f3819bb92c6575f117562d Mon Sep 17 00:00:00 2001 From: Francis Roberts <111994975+franrob-projects@users.noreply.github.com> Date: Tue, 23 Sep 2025 13:18:10 +0200 Subject: [PATCH 03/22] Reliable message ordering for connected clients --- .../architecture/message-ordering.mdx | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/src/pages/docs/platform/architecture/message-ordering.mdx b/src/pages/docs/platform/architecture/message-ordering.mdx index 3cdd3bca52..458799b0a5 100644 --- a/src/pages/docs/platform/architecture/message-ordering.mdx +++ b/src/pages/docs/platform/architecture/message-ordering.mdx @@ -58,3 +58,26 @@ When building globally distributed applications, developers should understand th The dual ordering system allows applications to choose the appropriate consistency model for their needs. For highly interactive applications where responsiveness is critical, realtime order minimizes latency. For applications that require a consistent historical record, the canonical global order provides a stable view that all components can agree on. To support both ordering systems efficiently, messages are stored with metadata that tracks their position in multiple sequences. This allows the platform to rapidly retrieve messages in either realtime or canonical global order as needed for different API operations or connection recovery scenarios. + +## Message ordering guarantees + +Ably ensures that messages are delivered to persistently connected subscribers on a channel in the order they were published when using the Realtime libraries. Each message sent on a realtime connection has a unique incrementing serial number, enabling this ordering guarantee. + +For example, if you publish messages using a Realtime client library, subscribers on that channel anywhere in the world will receive those messages in the same order they were originally published. + +### REST publishing considerations + +When publishing using REST client libraries, Ably still guarantees that the order messages are received will be honoured for all subscribers. However, if you are sending messages at a high rate with separate REST requests, it is possible that a later HTTP request may reach Ably before a previous request due to variable network factors outside of Ably's control. + +If ordering is important to you when using REST, consider these approaches: +- Rate limit your HTTP requests +- [Batch messages](/docs/messages/batch) together in a single request +- Use Realtime client libraries to publish messages instead + +### Connection recovery edge cases + +Ably supports connection state recovery, so even if a connection is lost and re-established, messages replayed when reconnected will be in sequential order. + +However, there is a rare situation where ordering may not be maintained: if a client is disconnected and the server maintaining that client's connection state is terminated or recycled during connection state recovery, messages may potentially be replayed out of order. This behaviour is by design to ensure that the backlog of messages held in the connection state do not prevent the new server from sending realtime messages to clients. + +If message ordering is required in all cases, consider catching the connection disconnected event and either re-establishing a new connection or manually re-ordering incoming messages following disconnection. From 6383ae039a142a3179801547f02005595ca67b8c Mon Sep 17 00:00:00 2001 From: Francis Roberts <111994975+franrob-projects@users.noreply.github.com> Date: Tue, 23 Sep 2025 14:09:45 +0200 Subject: [PATCH 04/22] Am I able to check who has read published messages? --- src/pages/docs/messages/index.mdx | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/src/pages/docs/messages/index.mdx b/src/pages/docs/messages/index.mdx index 511c4f9306..53ed853fe3 100644 --- a/src/pages/docs/messages/index.mdx +++ b/src/pages/docs/messages/index.mdx @@ -40,6 +40,34 @@ You can ensure a message was successfully published by checking the [history](/d Ably does not store per-message delivery logs, nor logs of who is subscribed to a channel at any point in time. This means it is not possible to check which users have received messages retroactively. +### Read receipts and acknowledgments + +Ably does not natively support read receipts or message acknowledgments to track who has read published messages. This aligns with the fundamental pub-sub pattern where publishers are decoupled from subscribers and shouldn't need to know or care about the number of recipients. + +Since Ably doesn't control how many clients are expected on a channel, it's architecturally difficult to determine who has read or received a message. However, you can implement a custom read receipt system by having subscribers publish acknowledgment messages when they consider a message received or read: + +```javascript +// Subscriber acknowledges reading a message +channel.subscribe('content', (message) => { + // Process the message + console.log('Received:', message.data); + + // Send read receipt + channel.publish('read-receipt', { + messageId: message.id, + readBy: clientId, + readAt: Date.now() + }); +}); + +// Publisher listens for read receipts +channel.subscribe('read-receipt', (receipt) => { + console.log(`Message ${receipt.data.messageId} read by ${receipt.data.readBy}`); +}); +``` + +This approach requires application-level implementation and careful consideration of scalability in high-traffic scenarios. + ## Message conflation Use message conflation to ensure that clients only ever receive the most up-to-date message, by removing redundant and outdated messages. Message conflation will aggregate published messages for a set period of time and evaluate all messages against a [conflation key](#routing). All but the latest message for each conflation key value will be discarded, and the resulting message, or messages, will be delivered to subscribers as a single batch once the period of time elapses. From 10e5e3cef88e14568a9265b0e53e56d3df352d32 Mon Sep 17 00:00:00 2001 From: Francis Roberts <111994975+franrob-projects@users.noreply.github.com> Date: Tue, 23 Sep 2025 14:10:28 +0200 Subject: [PATCH 05/22] Client-specified message ID restrictions for multiple messages published atomically --- src/pages/docs/pub-sub/advanced.mdx | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/src/pages/docs/pub-sub/advanced.mdx b/src/pages/docs/pub-sub/advanced.mdx index 46f2c736be..af9e719aa0 100644 --- a/src/pages/docs/pub-sub/advanced.mdx +++ b/src/pages/docs/pub-sub/advanced.mdx @@ -954,7 +954,32 @@ In some cases you may wish to set the unique message ID yourself to achieve idem * To ensure idempotency when a publisher instance might be restarted, and continuous activity cannot be guaranteed. * To integrate with an upstream system that uses message IDs, to ensure idempotency across an entire message processing pipeline. -If setting your own message IDs be aware of the [restrictions](https://faqs.ably.com/client-specified-message-id-restrictions-for-multiple-messages-published-atomically) on its format when publishing messages atomically. +If setting your own message IDs be aware of the restrictions on its format when publishing messages atomically. + +#### Restrictions for atomic publishing + +When publishing multiple messages atomically (e.g., by calling `publish()` with an array of messages), those messages are bundled together and will either all succeed or all fail. For idempotency purposes, there is only a single idempotency key for the entire array of messages. + +To prevent confusion about individual message idempotency, Ably requires that all messages in an atomic publish must have IDs derived from a single base ID. Each message must contain an ID of the form `:` where `idx` is a zero-based index into the array of messages. + +For example, to publish 3 messages atomically with a base ID of `foo`, the messages must have IDs `foo:0`, `foo:1`, and `foo:2`. This emphasizes that the messages share a single idempotency key (`foo`). + +```javascript +// Correct: atomic publishing with derived IDs +await channel.publish([ + { id: 'foo:0', data: 'first' }, + { id: 'foo:1', data: 'second' }, + { id: 'foo:2', data: 'third' } +]); + +// Incorrect: would be rejected +await channel.publish([ + { id: 'foo', data: 'first' }, + { id: 'bar', data: 'second' } // Different base ID not allowed +]); +``` + +If you want messages to be individually idempotent, publish them separately using multiple `publish()` calls, or use the [batch publishing API](/docs/messages/batch#batch-publish) where each message is contained in its own batch specification.