Session workers and dealing with connectivity issues using Ockam Routing

[hairyhum]

Session workers and dealing with connectivity issues:

This document is trying to describe a higher order concept in Ockam Messaging such as Sessions.

The concept of Session serving to create a mental model of communication using Ockam Routing protocol and Ockam Workers.
It may be implemented explicitly by Ockam Implementations, or just serve to guide implementations to design and reason about the messaging easier.

There is no requirement for implementations to explicitly define Sessions, although some protocols behave this way and may
be described this way.

Sessions

A session is established between workers possibly on different sides of the network and can forward messages through.

Session is usually represented as some state negitiated between two workers.

This may be:

• Network connection
• routes to each other
• shared keys
• some additional protocol data

Session workers are the workers storing this state.

The state may be:

• shared (like a shared encryption key)
• symmtrical (like routes to each other)
• asymmetrical (e.g. initialization route, more about routes later)

Sessions may be used for forwarding, like secure channels, or they can be used to just exchange messages between the session workers.

Forwarding sessions may be embedded into a higher order session routes, allowing session layering.

For example a secure channel may use stream session, which in turn use stream client consumer and stream client publisher sessions, which in turn use TCP connection sessions.

A session has two stages:

• session negotiation
• message exchange

Ususally negotiation is the first thing a session worker does when starting up.

Negotiation involves sending messages to the other worker in the session (often creating the this worker) and receiving
a response from it.

After the negotiation both workers would have some knowledge on how to deliver messages to the other end in their state.
When sessions are established with Ockam Routing protocol this knowledge is represented as an Ockam Route.

Session negotiation MAY be implicit in the messaging pipeline. An example of implicit session is transport sessions like in
TCP, when the TCP connection is created when processing a message sent to the TCP Transport address.

Implicit and explicit sessions

Implicit session is a session which is created during message processing (and not an API call) when forwarding a message through.
Implicit sessions are important for message forwarding through intermediate hubs and endpoint discovery.
They are presented to the application code as a step in the routes it's sending messages through.
Implicit sessions are often use as a part of other sessions delivery mechanism

Explicit sessions are started in the application code explicitly and managed by the application code.
There would be an API to create those sessions and application code would usually send messages to the explicit session address

A good example of an explicit session is secure channels.

Session routes

When session uses Ockam Routing protocol to deliver messages the session workers often have two types of routes:

• Initialization route
• Messaging route

Initialization route might be provided when the worker is started while messaging route is negotiated when session is
established.

Messaging route might me transient, so that workers for the route addresses may stop and these addresses may become unaccessible.

Re-negotiating sessions

Session workers may inplement re-negotiation logic using initialization route to re-establish a session and get a new
messaging route.

The session worker still need some signal or trigger to start re-negotiation. This might be an API call, a message or a timeout.

A session which can be re-negotiated while keeping the worker address and state is called a persistent session

A session which does not re-negotiate in case of errors and terminate instead, is called a transient session

In case of TCP networking, the initialization route would contain the TCP transport address and the messaging route would
contain the TCP client local address.
If TCP client fails, the re-negotiation will use the TCP transport and MIGHT create a new TCP client.

Synchronous requests:

When a worker expects to receive a message in response to a message it sends, we can call this synchronous request.

Synchronous requests allow workers to detect interruptions in the message delivery using timeouts and react to that.

Synchronous requests have the following events:

• sending a request
  followed by:
  • receiving a response
  • timeout

Using synchronous request with sessions:

When a session worker is using synchronous requests in its communication, it can set up a session re-negotiation
on request timeouts or error messages.

It's important that in this case re-negotiation messages are send in synchronous requests as well, allowing timeouts and
retry of re-negotiation.

Putting it all together

Let's imagin a session with a synchronous requests and a TCP client in between the session workers.

While initiator and responder establish a persistent explicit session, the TCP client and handler are an implicit transient session.

    initiator   TCP transport                TCP listener                   responder
      |               |                            |                            |
      |               |         TCP client 1       |       TCP handler 1        |
      | negotiate     | create     |   connect     |  create    |               |
      |-------------->|----------->|-------------->| ---------->|-------------->|
      |               |            |               |            |               |
      |               |            |               |            |               |
      |                            |                            | init ok       |
      |<---------------------------|<---------------------------|<--------------| *messaging route*: tcp_handler1 -> initiator
      |                            |                            |               |
      |                            |                            |               |
      | req sync                   |                            |               |
      |--------------------------->|--------------------------->|-------------->| *messaging route*: tcp_client1 -> responder
      |                            |                            |               |
      |                            |                            |  resp sync    |
      |<---------------------------|<---------------------------|<--------------|
      |                            |                            |               |
      |                            |                            |               |
      |                            x         TCP breaks here    x               |
      | req sync                                                                |
      |--------------------------->                                             |
      |                                                                         |
      | timeout                                                                 |
      |               |         TCP client 2       |       TCP handler 2        |
      | re-negotiate  | create     |   connect     |  create    |               |
      |-------------->|----------->|-------------->| ---------->|-------------->| init route: TCP/host:port -> responder
      |               |            |               |            |               |
      |               |            |               |            |               |
      |                            |                            | init ok       |
      |<---------------------------|<---------------------------|<--------------| *messaging route*: tcp_handler2 -> initiator

For example for stream consumer client

initiator: stream client consumer
responder: stream instance (on the hub)

init route: (tcp transport -> stream service)
messaging route (on client): (tcp client* -> stream instance*)

synchronous request: pull messages

• • dynamic addresses not known in advance

Session glossary:

Session - a pair of workers which share some state and knowledge on how to send messages to each other.

Session types:

• Implicit - created as a part of message forwarding from the forwarded message metadata
• Explicit - created using an API with additional arguments

Session endpoints:

• Initiator - a worker which negotiates a session
• Responder - a worker which responds to negotiation
• *Listener - a worker which handles negotiation messages and can spawn or redirect to responder

Session routes (only valid for ockam routing sessions):

• Initialization route - a route used to establish a session
• Messaging route - a route used to send messges whithin a session

Session resiliency:

• Persistent sessions - sessions which can re-negotiate a messaging route keeping the state and their local addresses
• Transient sessions - sessions which do not re-negotiate, usually terminate in case of errors

Using session to achieve reliable delivery with stream protocol

Stream client publisher creates a persistent session with the stream worker and uses synchronous messages to publish.
The publisher then can re-negotiate the session in case of a timeout.

Stream client consumer creates a transient session with the stream worker, but restarts on timeout, which triggers a
re-negotiation of the session.
This is similar to a persistent session, except there is no state to persist between re-negotiations.

Stream worker stores messages in an external storage (kafka) synchronously, which means that it can safely restart
without loosing messages.

Consumer then uses a saved message index, which is updated after the message was processed, which means that it's
guaranteed to process all messages at least once.

Essentially the stream protocol provides a at-least-once delivery guarantee between stream publisher receiving the message
and stream consumer processing it.

Additional thoughts

Because sessions can be embedded into each other using forwarding workers and Ockam Routing, it's possible to model different session workers in the same message pipeline to serve different purposes.

For example if secure channel session uses a stream session to deliver messages, it doesn't have to implement persistent session behaviour, while still getting the benefits of the persistent session like resilience to connection issues.

Alternatively the application code may wrap secure channel session into a different persistent session and handle secure channel failures in there.

[hairyhum]

Persistent sessions may have different "types" of persistence: some sessions can be re-negotiated just from one side, others from both sides.
For example if we implement re-negotiation on the initiator worker - from the initiator side the session is persistent, while for the responder it is not, because responder cannot re-negotiate the session itself.

[hairyhum]

We can draw some inspiration for the names and the patterns from https://www.enterpriseintegrationpatterns.com/patterns/messaging/