actor shutdown

[JohnMurray]

Implement a basic shutdown mechanism for an actor. Essentially actors are spawned by parent actors and while they can continue to exist forever (lifetime of the process), they often have lifetimes associated with them.

I'm defining basic shutdown support as:

• Hook on the actor that can be called before the actor is removed from the executor
• A poison-pill, system message
• An immediate shut-down mechanism
• A death-watch mechanism

Additional functionality that should exist as part of actor shut-down, but are a stretch goal of this work:

• Cascading shutdown across children
• System shutdown - when root actor (and all children) are shut-down.

Shutdown Semantics and Message Draining

Actors are single-threaded and process messages from their mailbox serially. So the natural question is, when shutting down, how are existing and incoming messages handled? The answer is that there are two (related) shutdown paths, immediate shutdown and delayed shutdown.

Immediate shutdown is what it sounds like. All messages in the queue are left un-processed, the actor's shutdown hook is called, the actor is removed from the executor, and all death-watches are triggered.

Delayed shutdown is done through the use of a poison pill. This is a special message that can be sent to the actor and sits in the mailbox (message queue) like any other message. What makes it special is that once the poison pill message is processed, the actor's immedaite shutdown logic is triggered. By triggering this code-path through a normal message, all existing messages are processed first (drained) before the actor is terminated. Although it's still possible other messages will be queued up after the poison pill message.

All of the unprocessed messages will go to the dead-letter queue. Actors who's messages are dropped will not receive any sort of notification as message delivery is at-most-once semantics.

---

• Write docs on actor lifecycle
  • Update code-docs to point to the lifecycle docs where it matters (e.g. rustdocs on shutdown functions/types)