Serialization?

[CLOVIS-AI]

Hi!

I'm trying to understand the consequences of multi-shot continuations and what it would allow us to do.

Since this library apparently works by cloning continuations; would it be possible to serialize a continuation so it can be continued on another machine or later in another execution?

(also, what do you think of enabling Discussions on this repository?)

[kyay10]

Serialisation has been attempted before with some people pulling it off. Serialising Continuations is very easy. The tricky part is to deal with everything that might be captured by them. You'll need to replace references to global objects, ensure you capture no file handles, etc. I don't think this library can help much though in serialisation (making its classes serialisable might be a good idea though). I want to explore serialisation eventually because it's such a nice selling point for continuations (even project Loom plans on adding it!).
Now that I think about it more, the continuations we provide (the Cont<T, R>) might be better suited to serialisation since it'll avoid capturing top-level unnecessary things (I've already done significant work on memory leaks). This might help smoothen that journey a bit then.

Multi-shot coroutines can do more things than just serialisation though. It can allow cool things like probabilistic programming and logic programming (maybe even making a Prolog-like DSL?)

Enabling discussions is a great shout! Done!

Edit: accidentally closed the issue lol!

[kyay10]

There's also Section 8 of this paper which touches on the topic briefly. Their approach (storing globals in an array and serialising references to it as indices) is unlikely to be helpful here though.

[CLOVIS-AI]

Honestly, I've never worked with a language with multi-shot continuations, so I don't have an intuition for what becomes possible. So far the use-case I understand the best is the one for the Parameterize library, but that's about it. I'm very curious what people will build with this power.

[kyay10]

I'll try to collect a few usecases soon, perhaps in a discussion thread. For now, you can see a lot of use cases in the tests, especially the effekt directory.
Another use case is the m-word (monads). See this file for an example. Any monad can be used in "direct style" (fancy term that means without using flatMaps everywhere) as long as it supports suspend. Another popular example of this is the Raise DSL from Arrow, which provides direct-style calls over the Either monad. Thinking in terms of monads ends up conflating interface with implementation though, so the better approach is to think in terms of "effect interfaces" or simply "effects". Raise is what's called a single-shot effect, so it doesn't need this library (and in fact, Raise can even be implemented with just suspend and without exceptions). Effects that need multishot behaviour are ones that want to return to the same place multiple times, like Parameterize. You can also, for instance, implement very slick list comprehensions in Kotlin this way (see this file). I especially like the allEightBitPatternsWithOnlyChange test because I struggled to write it neatly in a few lines without comprehensions.
I want to highlight something btw, which is that the value of this library isn't purely the mutishotness. The effekt stuff is a very cool design approach that allows you to make imperative DSLs easily. I'll expand on what "imperative" really means here soon. Good examples of them are Raise and ResourceScope though. It's a style that models effects as "capabilities", or basically just interfaces that we receive through context parameters.

[CLOVIS-AI]

I'm having a really hard time grasping how to read these snippets 😅 I think the difficulty comes from the naming, I don't have an intuition for what Reset etc mean. If that's useful, I'd be interested in helping workshop naming & APIs to make the library more "Kotlin-ready" / easier to understand to the average Kotlin developer who doesn't know the concepts

[kyay10]

Agreed. The naming is from the literature, but it needs more friendly names. I think effekt is friendlier though, with its handle and use naming

[kyay10]

(Note: some examples don't compile because I'm using fun interface with functions that have generics for brevity. Also, most of these examples are singleshot, and all the singleshot cases never use reflection or any trickery, so these could be written using normal coroutines, but I'm sure you'll see that this way is clearer).
The core idea of effects is that you have some scope (think like resourceScope or either) and you can capture a continuation (roughly, all the "rest" of the code that'll run from where you are up to the scope) and get to choose how it's invoked. For instance, you could add finally blocks:

fun interface MyDsl<R> {
  suspend fun onFinally(finalizer: suspend (Result<R>) -> Unit)
}
// assuming a `block: suspend MyDsl<R>.() -> R`
handle {
  block(MyDsl { finalizer ->
    use { continuation -> // continuation is (roughly) a `suspend (Unit) -> R`
      // Unit comes from the return type of `onFinally`, while `R` comes from the return type of `handle`
      val result = runCatching {
        continuation(Unit) // returns R
      }
      finalizer(result)
      result.getOrThrow()
    }
  })
}

Because the continuation is just a function, you can also pass it to other top-level functions, for instance:

fun interface MyDsl {
  suspend fun <A: AutoCloseable> bind(a: A): A
}

// assuming a `block: suspend MyDsl.() -> R`
handle {
  block(MyDsl { a ->
    use { continuation ->
      a.use { // this is stdlib `AutoCloseable.use`
        continuation(a)
      }
    }
  })
}

You could even do this with Raise (note, Raise doesn't play nicely with multishot because it locks itself. The library does have ways of creating multishot Raise instances though, and in the future I'll maybe provide multishot versions of raise builders)

fun interface MyDsl<R> {
  suspend fun <E> applyBuilderLocally(builder: (suspend Raise<E>.() -> R) -> R): Raise<E>
}

// assuming a `block: suspend MyDsl<R>.() -> R`
handle {
  block(MyDsl { builder ->
    use { cont ->
      builder { cont(this) }
    }
  })
}
// usage
applyBuilderLocally(::nullable).raise() // only compiles if the scope is already nullable
applyBuilderLocally<String> { block -> recover(block) { transformStringToRSomehow(it) } } }.raise("I'm outta here!")

Look at that! We just built withError, albeit in a more complicated manner, but while knowing nothing about the internals of Arrow! As long as Arrow provides suspend functions (and, currently, those can't use withContext or other coroutine changers, but I'm working on that!), we can use it to wrap blocks just like that.