Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PLT-8235 Improve traces for chain-seek and chain-query #735

Merged
merged 10 commits into from
Oct 27, 2023

Conversation

jhbertra
Copy link
Contributor

This PR is part of a larger effort to overhaul the tracing of the protocol peers.

  • Introduce a new abstraction (motivated in a comment below): the PeerT indexed monad transformer.
  • Provide PeerT implementation for ChainSeekClient and ChainSeekServer
  • Provide PeerT implementation for QueryClient and Query
  • Introduce a set of application-specific event selectors for these two protocols to replace the generic typed-protocols selectors.
  • Add optional SpanContext propagation to the message types of these two protocols.
  • Change all connections to chain-seek and chain-sync-query within the runtime to use the PeerT implementation instead of the PeerTraced implementation

Pre-submit checklist:

  • Branch
    • Tests are provided (if possible)
    • Test report is updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
    • Relevant tickets are mentioned in commit messages
    • Formatting, PNG optimization, etc. are updated
    • Operables are updated with changes to executable command line options.
    • Deploy charts updated with changes to operables.
  • PR
    • Self-reviewed the diff
    • Useful pull request description
      • Review required
      • Substantial changes to code, test, or documentation
      • Change made to Marlowe validator (@bwbush and @palas must be included as reviewers)
      • Review not required
      • Minor changes to non-critical code, documentation, nix derivations, configuration files, or scripts
      • Formatting, spelling, grammar, or reorganization
    • Reviewer requested

@jhbertra jhbertra requested a review from bwbush October 24, 2023 14:41
@jhbertra jhbertra self-assigned this Oct 24, 2023
@jhbertra jhbertra force-pushed the plt-8235-improve-tracing branch from 4e1026f to dcdd241 Compare October 24, 2023 14:41
@jhbertra
Copy link
Contributor Author

Peer vs PeerT

Traditionally, the DSL for expressing programs running a protocol has been the Peer GADT:

data Peer (pr :: PeerRole) ps (st :: ps) m a where
  Effect :: m (Peer pr ps st m a) -> Peer pr ps st m a
  Yield :: WeHaveAgency pr st -> Message ps st st' -> Peer pr ps st' m a -> Peer pr ps st m a
  Await :: TheyHaveAgency pr st -> (forall (st' :: ps). Message ps st st' -> Peer pr ps st' m a) -> Peer pr ps st m a
  Done :: NobodyHasAgency st -> a -> Peer pr ps st m a

Some essential characteristics of this data interface are:

  • It is an inial encoding.
  • It is not a monad.
  • It is not an applicative or even pointed functor (you can't write a definition for pure)
  • Sequential peer composition is explicit in the structure. This means you have control over the order
    in which sub-actions run but not what happens between or around them. In particular, this means that
    bracket-like functions are impossible to use if they span yield or await operations. This is a consequence
    of it being an initial encoding.
  • Termination is only possible when you are in a terminal protocol state

Unfortunately, functions like withEvent have a bracket-like structure in which you define an explicit scope
during which an event is active. If you want that event to span a call/response cycle from a client, there is no way
to do this with Peer.

This PR introduces an alternative DSL for expressing programs that run protocols, the PeerT indexed monad transformer:

data Channel (pr :: PeerRole) ps (st :: ps) m = Channel
  { sendMessage :: forall st'. WeHaveAgency pr st -> Message ps st st' -> m (Channel pr ps st' m)
  , recvMessage :: TheyHaveAgency pr st -> m (SomeMessageAndChannel pr ps st m)
  }

data SomeMessageAndChannel (pr :: PeerRole) ps (st :: ps) m where
  SomeMessageAndChannel
    :: Message ps st st'
    -> Channel pr ps st' m
    -> SomeMessageAndChannel pr ps st m

newtype PeerT (pr :: PeerRole) ps (i :: ps) (j :: ps) m a = PeerT
  { unPeerT :: Channel pr ps i m -> m (a, Channel pr ps j m)
  }

One way of thinking about this type is to consider it an indexed state monad over a Channel type that can send or receive a message from a particular state. Compared with Peer:

  • It is a final encoding
  • It is a monad (provided the initial and final protocol states are the same)
  • It is an indexed monad (a monad with a (>>=) operation that can change states)
  • Because it is a final encoding, it is possible (with a little bit of encapsulated unsafe usage) to lift
    bracket-like operations into it.

This makes it suitable for event-based tracing. There are some cases in which this is trivial:

{-# LANGUAGE QualifiedDo #-}

import qualified Network.Protocol.Peer.Monad as P

myPeer :: P.PeerT 'AsClient FooProtocol 'StFoo 'StBar m a
myPeer foo = P.withEvent MyEvent \ev -> P.do
    P.yield $ MsgFoo foo
    pure bar

The above starts an event before a message is yielded, yields a message, then ends the event. This would be impossible to do with Peer. await is not as easy in general, because it uses a RankN type for the callback, so some clever alignment of protocols states an use of the indexed join operation is necessary to delimit spans of code, but it is at least possible:

myPeer :: P.PeerT 'AsServer FooProtocol 'StFoo 'StBar m a
myPeer foo = P.do
  P.join $ P.withEvent MyEvent \ev -> P.await \case
    MsgFoo foo -> P.do
      P.lift $ doSomethingWithFoo foo
      pure $ restOfThePeerIDon'tWantInsideTheEvent foo

Why is this significant?

With Peer, the only option was to put generic protocol-agnostic tracing in Peer-eliminating functions. This led to traces that were:

  • Too verbose and noisy
  • Contained too much information
  • Not easy to customize for different protocols.
  • Fragile. Optimizing for one protocol often broke readability for another.

The problem with the last point is that we want different tracing characteristics for different protocols. For example, we don't want to trace every poll message in a sync protocol, and we don't want the whole protocol session to be captured in a single trace that keeps growing, whereas this is acceptable and desirable for simple call-and-response query protocols. Furthermore, there is no way to pass event references to the programs being traced if desired.

With PeerT however, it is possible (though admittedly not always trivial) to move the tracing into the peer code. This allows for the best tracing decisions for the specific protocol or application. This means:

  • Only what needs to be traced is traced
  • The scope of a single span can be decided on a case-by-case basis
  • Changes for one protocol won't affect tracing for another
  • Noisy periodic interactions (like polling) can be omitted.

Scope of this PR

This PR introduces and applies the new abstraction to the ChainSeek and Query protocols. These are utilized in marlowe-chain-sync. Subsequent PRs will to the same for the other protocols.

@jhbertra jhbertra requested a review from shlevy October 24, 2023 18:48
@jhbertra jhbertra force-pushed the plt-8235-improve-tracing branch from 635c525 to 1ee83d6 Compare October 25, 2023 13:05
Copy link
Collaborator

@bwbush bwbush left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks!

- Use `PeerT` instead of `PeerTraced`
- Add trace spans and span context propagation
- Update usages
- Add a peerT implementation
- Add span context info to message types
- Add default trace spans to clients and servers.
@jhbertra jhbertra force-pushed the plt-8235-improve-tracing branch from 1ee83d6 to 2c1b584 Compare October 27, 2023 12:53
@jhbertra jhbertra enabled auto-merge October 27, 2023 12:53
@jhbertra jhbertra merged commit cde7197 into main Oct 27, 2023
5 checks passed
@jhbertra jhbertra deleted the plt-8235-improve-tracing branch October 27, 2023 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants