IO eagerness fix

The recent IO refactor allowed for eager evaluation of pure lifted values, this causes stack-overflows and other side-issues with StreamT and other infinite recursion techniques. This minor release fixes that.

Fin, Try, and IO applicative behaviours + minor fixes

A question was asked about why Fin doesn't collect errors like Validation when using applicative Apply, seeing as Error (the alternative value for Fin) is a monoid. This seems reasonable and has now been added for the following types:

• Fin<A>
• FinT<M, A>
• Try<A>
• TryT<M, A>
• IO<A>
• Eff<A>
• Eff<RT, A>

I extended this for Try, IO, and Eff because their alternative value is also Error, so it makes sense in applicative scenarios.

The IO monad has also had its Apply internals updated to work with the new underlying IOAsync, IOSync, ... types. It now uses regular Task.WhenAll instead of forking to achieve concurrent execution. To achieve genuine parallel execution you can still call Fork on the operands.

IO has also had its Zip functions updated to use Apply instead of forking for the same reasons. That means forking of an IO operation is a choice by the programmer rather than something that is imposed in certain functions.

Because Eff<RT, A> and Eff<A> are both based on the IO monad they're also updated to this new behaviour.

Domain Type traits

Minor fixes to the Domain-Type interfaces:

In Locus<SELF, SCALAR, DISTANCE>, I have reordered the SCALAR and DISTANCE types and renamed SCALAR to SCALAR_DISTANCE; that means the new type is: Locus<SELF, DISTANCE, DISTANCE_SCALAR> -- so it's obvious that it's a scalar value for the distance rather than SELF. Also, removed Origin and now rely on the AdditiveIdentity from IAdditiveIdentity.

Credit card validation sample

Added a new Credit Card Validation sample, this is the example built in my Higher Kinds in C# series with all of the data-types converted to use the Domain Type traits.

IO performance improvements

In one of the proposals leading up to the big v5 refactor, I discussed the idea of using SpinWait as a lightweight waiting technique to avoid the use of the async/await machinery everywhere. I also mentioned that the idea might be too primitive. Well, it was.

So, I have modified the internals of the IO monad (which is where all async code lives now) to have four possible states: IOSync, IOAsync, IOPure, and IOFail. These are just types derived from IO (you never see them).

The idea is that any actual asynchronous IO will just use the regular async/await machinery (internally in IOAsync), any synchronous IO will be free of async/await (in IOSync), and any pure or failure values will have a super simplified implementation that has no laziness at all and just can pre-compute.

The TestBed.Web sample with the TestBed.Web.Runner NBomber test now runs both the sync and async versions with exactly the same performance and with no thread starvation; and without any special need to fork the IO operation on the sync version.

I consider that a big win which will allow users to avoid async/await entirely (if they so wish), one of the goals of 'Drop all Async variants' proposal.

app.MapGet("/sync", 
    () => {
        var effect = liftIO(async () =>
                            {
                                await Task.Delay(1000);
                                return "Hello, World";
                            });

        return effect.Run();
    });

app.MapGet("/async", 
    async () => {
        var effect = liftIO(async () =>
                            {
                                await Task.Delay(1000);
                                return "Hello, World";
                            });
        
        return await effect.RunAsync();
    });

Issue fix

• AsyncEnumerable extensions introduced in 5.0.0-beta-08 conflict with System.Linq.Async #1360 - thanks to @guythetechie

Domain-types update

Domain-types are still a relatively nascent idea for v5 that I am playing around with. I wouldn't use them in anger unless you're ok with updating your code when I change them. Because I will!

Anyway, the updates are:

• Improved documentation
• Changed their inheritance hierarchy so don't see so many where constraints
• DomainType<SELF, REPR> has a base DomainType<SELF>. The derived domain types (Identifier, Locus, VectorSpace, and Amount) inherit from DomainType<SELF>.
  • So, they don't need to specify a REPR type, simplifying the traits.
  • It does however mean that you will need to specify the DomainType<SELF, REPR> type as well as whatever derived domain type to gain a constructable value (see the Length example later)
• Changed From in DomainType<SELF, REPR> to return a Fin<SELF. This allows for validation when constructing the domain-type.
  • Because this isn't always desired, you can use an explicitly implemented interface method to override it.
    • See the Length example below
• Dropped the Quantity domain-type for now
  • I need to find a better approach with C#'s type system

public readonly record struct Length(double Value) :
    DomainType<Length, double>, //< note this is now needed, because Amount only impl DomainType<Length>
    Amount<Length, double> 
{
    public static Length From(double repr) => 
        new (repr);
    
    public double To() =>
        Value;

    // explicitly implemented `From`, so it's not part of the Length public interface
    static Fin<Length> DomainType<Length, double>.From(double repr) =>
        new Length(repr);

    public static Length operator -(Length value) => 
        new (-value.Value);

    public static Length operator +(Length left, Length right) => 
        new (left.Value + right.Value);

    public static Length operator -(Length left, Length right) => 
        new (left.Value - right.Value);

    public static Length operator *(Length left, double right) => 
        new (left.Value * right);

    public static Length operator /(Length left, double right) => 
        new (left.Value / right);

    public int CompareTo(Length other) => 
        Value.CompareTo(other.Value);

    public static bool operator >(Length left, Length right) =>
        left.CompareTo(right) > 0;

    public static bool operator >=(Length left, Length right) => 
        left.CompareTo(right) >= 0;

    public static bool operator <(Length left, Length right) => 
        left.CompareTo(right) < 0;

    public static bool operator <=(Length left, Length right) => 
        left.CompareTo(right) <= 0;
}

StreamT merging and zipping + parsing updates

This release follows on from the last release (which featured the new StreamT type): we can now merge and zip multiple streams. There's also an update to the Prelude.parse* functions (like the Option<int> returning parseInt).

Merging

Merging multiple StreamT streams has the following behaviours:

• async & async stream: the items merge and yield as they happen
• async & sync stream: as each async item is yielded, a sync item is immediately yielded after
• sync & async stream: each sync item is yielded immediately before each async item is yielded
• sync & sync stream: each stream is perfectly interleaved

If either stream finishes first, the rest of the stream that still has items keeps yielding its own items.

There is an example of merging on in the Streams sample:

public static class Merging
{
    public static IO<Unit> run =>
        example(20).Iter().As() >>
        emptyLine;

    static StreamT<IO, Unit> example(int n) =>
        from v in evens(n) & odds(n)
        where false
        select unit;
    
    static StreamT<IO, int> evens(int n) =>
        from x in Range(0, n).AsStream<IO>()
        where isEven(x)
        from _ in magenta >> write($"{x} ")
        select x;

    static StreamT<IO, int> odds(int n) =>
        from x in Range(0, n).AsStream<IO>()
        where isOdd(x)
        from _ in yellow >> write($"{x} ")
        select x;
    
    static bool isOdd(int n) =>
        (n & 1) == 1;

    static bool isEven(int n) =>
        !isOdd(n);
}

This creates two streams: odds and evens and them merges them into a single stream using:

evens(n) & odds(n)

The output looks like this:

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

With differing colours depending on whether odd or even.

You can merge any number of streams with the & operator, or concatenate streams with the + operator.

Other ways to merge:

var s = stream1.Merge(stream2, ...);
var s = StreamT.merge(stream1, stream2, ...);
var s = merge(stream1, stream2, ...);  // in the Prelude

Zipping

You can zip up to four streams and the result is a stream of tuples.

Obviously, to create a tuple all of the streams need to have yielded a value and so must wait for them on each stream. But, be sure that the async streams are running independently and not blocking before being tupled.

That also means the length of the tuple stream is clamped to the shortest stream length.

Useful aspects of zipping sync and async is that you can pair async events with identifiers:

For example, imagine you have a stream of messages coming from an external source (async):

static StreamT<IO, Message> messages =>
    // create an async message stream

And a stream of natural numbers, playing the role of an identifier (sync):

static StreamT<IO, long> ids =>
    Range(0, long.MaxValue).AsStream<IO>();

Then you can tag each message with a unique identifier like so:

static StreamT<IO, (long Id, Message)> incoming =>
    ids.Zip(messages);

There's also an example in the Streams sample. It's similar to the merging example, except, instead of interleaving the odd and even streams, it tuples them:

public static class Zipping
{
    public static IO<Unit> run =>
        from x in example(10).Iter().As()
        select unit;

    static StreamT<IO, Unit> example(int n) =>
        from v in evens(n).Zip(odds(n))
        from _ in writeLine(v)
        where false
        select unit;

    static StreamT<IO, int> evens(int n) =>
        from x in Range(0, n).AsStream<IO>()
        where isEven(x)
        select x;

    static StreamT<IO, int> odds(int n) =>
        from x in Range(0, n).AsStream<IO>()
        where isOdd(x)
        select x;
    
    static bool isOdd(int n) =>
        (n & 1) == 1;

    static bool isEven(int n) =>
        !isOdd(n);
}

The output looks like this:

(0, 1)
(2, 3)
(4, 5)
(6, 7)
(8, 9)

There are no operators for zipping (because operators don't support generics), these are the options:

var s = stream1.Zip(stream2, .., stream4);
var s = StreamT.zip(stream1, .., stream4);
var s = zip(stream1, .., stream4);  // in the Prelude

Parsing

parseInt and its variants (parseLong, parseGuid, etc.) all return Option<A> where A is the type being generated from the parse. With the advent of the trait-types - in particular the Alternative<M> trait - we can now parse to any type that implements the Alternative<M> trait.

Alternative<M> is like a monoid for higher-kinds and it has an Empty<A>() function that allows us to construct a 'zero' version of higher-kind (think None in Option, but also Errors.None in types with an alternative value of Error).

The original parse* functions (that return Option), remain unchanged, but there is now an extra overload for each variant that takes the trait-implementation type as a generic parameter:

Here's the original parseInt with the new parseInt<M>:

public static Option<int> parseInt(string value) =>
    Parse<int>(int.TryParse, value);

public static K<M, int> parseInt<M>(string value)
    where M : Alternative<M> =>
    Parse<M, int>(int.TryParse, value);

To see how this helps, take a look at the run function from the SumOfSquares example:

Before:

public static class SumOfSquares
{
    public static IO<Unit> run =>
        from _ in writeLine("Enter a number to find the sum of squares")
        from s in readLine
        from n in parseInt(s).Match(Some: IO.pure, None: IO.fail<int>("expected a number!"))
        from x in example(n).Iter().As()
        select unit;

    ..
}

After

public static class SumOfSquares
{
    public static IO<Unit> run =>
        from _ in writeLine("Enter a number to find the sum of squares")
        from s in readLine
        from n in parseInt<IO>(s)
        from x in example(n).Iter().As()
        select unit;
   
    ..
}

We lift directly into the IO monad instead of into Option first (only to have to match on it straight away).

Obviously, the default alternative value might not be right, and so you can then use the | operator to catch the failure:

public static class SumOfSquares
{
    public static IO<Unit> run =>
        from _ in writeLine("Enter a number to find the sum of squares")
        from s in readLine
        from n in parseInt<IO>(s) | IO.fail<int>("expected a number!")
        from x in example(n).Iter().As()
        select unit;

    ..
}

Instead of raising an error, you could also provide a default if the parse fails:

parseInt<IO>(s) | IO.pure(0)

This is nice and elegant and, I think, shows the usefulness of the traits. I wouldn't mind removing the Option bearing parse* functions, but I don't think it hurts to keep them in.

As always, any questions or comments, please reply below.

New features: Monadic action operators, StreamT, and Iterable

Features:

• Monadic action operators
• New Iterable monad
• New StreamT monad-transformer
  • Support for recursive IO with zero space leaks
• Typed operators for |
• Atom rationalisation
• FoldOption
• Async helper
• IAsyncEnumerable LINQ extensions

Monadic action operators

The monadic action operator >> allow the chaining of two monadic actions together (like a regular bind operation), but we discard the result of the first.

A good example of why we want this is the LINQ discards that end up looking like BASIC:

public static Game<Unit> play =>
    from _0 in Display.askPlayerNames
    from _1 in enterPlayerNames
    from _2 in Display.introduction
    from _3 in Deck.shuffle
    from _4 in playHands
    select unit;

We are always discarding the result because each operation is a side-effecting IO and/or state operation.

Instead, we can now use the monadic action operator:

public static Game<Unit> play =>
    Display.askPlayerNames >>
    enterPlayerNames       >>
    Display.introduction   >>
    Deck.shuffle           >>
    playHands;

Here's another example:

static Game<Unit> playHands =>
    from _   in initPlayers >>
                playHand >>
                Display.askPlayAgain
    from key in Console.readKey
    from __  in when(key.Key == ConsoleKey.Y, playHands)
    select unit;

In the above example you could just write:

static Game<Unit> playHands =>
    initPlayers >>
    playHand >>
    Display.askPlayAgain >>
    from key in Console.readKey
    from __  in when(key.Key == ConsoleKey.Y, playHands)
    select unit;

It's really down to taste. I like things to line up!

Because operators can't have generics, we can only combine operands where the types are all available. For example:

public static IO<A> operator >> (IO<A> lhs, IO<A> rhs) =>
    lhs.Bind(_ => rhs);

But, we can de-abstract the K versions:

public static IO<A> operator >> (IO<A> lhs, K<IO, A> rhs) =>
    lhs.Bind(_ => rhs);

And, also do quite a neat trick with Unit:

public static IO<A> operator >> (IO<A> lhs, IO<Unit> rhs) =>
    lhs.Bind(x => rhs.Map(_ => x));

That propagates the result from the first operation, runs the second (unit returning) operation, and then returns the first-result. This is actually incredibly useful, I find.

Because, it's not completely general case, there will be times when your types don't line up, but it's definitely useful enough, and can drastically reduce the amount of numbered-discards! I also realise some might not like the repurposing of the shift-operator, but I chose that because it's the same operator used for the same purpose in Haskell. Another option may have been to use &, which would be more flexible, but in my mind, less elegant. I'm happy to take soundings on this.

The CardGame sample has more examples.

New Iterable monad

The EnumerableM type that was a wrapper for IEnumerable (that enabled traits like foldable, traversable, etc.) is now Iterable. It's now more advanced than the simple wrapper that existed before. You can Add an item to an Iterable, or prepend an item with Cons and it won't force a re-evaluation of the lazy sequence, which I think is pretty cool. The same is true for concatenation.

Lots of the AsEnumerable have been renamed to AsIterable (I'll probably add AsEnumerable() back later (to return IEnumerable again). Just haven't gotten around to it yet, so watch out for compilation failures due to missing AsEnumerable.

The type is relatively young, but is already has lots of features that IEnumerble doesn't.

New StreamT monad-transformer

If lists are monads (Seq<A>, Lst<A>, Iterable<A>, etc.) then why can't we have list monad-transformers? Well, we can, and that's StreamT. For those that know ListT from Haskell, it's considered to be done wrong. It is formulated like this:

   K<M, Seq<A>>

So, the lifted monad wraps the collection. This has problems because it's not associative, which is one of the rules of monads. It also feels instinctively the wrong way around. Do we want a single effect that evaluates to a collection, or do we want a collection of effects? I'd argue a collection of effects is much more useful, if each entry in a collection can run an IO operation then we have streams.

So, we want something like this:

Seq<K<M, A>>

In reality, it's quite a bit more complicated than this (for boring reasons I won't go into here), but a Seq of effects is a good way to picture it.

It's easy to see how that leads to reactive event systems and the like.

Anyway, that's what StreamT is, it's ListT done right.

Here's a simple example of IO being lifted into StreamT:

StreamT<IO, long> naturals =>
    Range(0, long.MaxValue).AsStream<IO>();

static StreamT<IO, Unit> example =>
    from v in naturals
    where v % 10000 == 0
    from _ in writeLine($"{v:N0}")
    where false
    select unit;

So, naturals is an infinite lazy stream (well, up to long.MaxValue). The example computation iterates every item in naturals, but it uses the where clause to decide what to let through to the rest of the expression. So, where v % 10000 means we only let through every 10,000th value. We then call Console.writeLine to put that number to the screen and finally, we do where false which forces the continuation of the stream.

The output looks like this:

10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
90,000
100,000
110,000
120,000
130,000
140,000
150,000
...

That where false might seem weird at first, but if it wasn't there, then we would exit the computation after the first item. false is essentially saying "don't let anything thorugh" and select is saying "we're done". So, if we never get to the select then we'll keep streaming the values (and running the writeLine side effect).

We can also lift IAsyncEnumerable collections into a StreamT (although you must have an IO monad at the base of the transformer stack -- it needs this to get the cancellation token).

static StreamT<IO, long> naturals =>
    naturalsEnum().AsStream<IO, long>();

static StreamT<IO, Unit> example =>
    from v in naturals
    from _ in writeLine($"{v:N0}")
    where false
    select unit;

static async IAsyncEnumerable<long> naturalsEnum()
{
    for (var i = 0L; i < long.MaxValue; i++)
    {
        yield return i;
        await Task.Delay(1000);
    }
}

We can also fold and yield the folded states as its own stream:

static StreamT<IO, int> naturals(int n) =>
    Range(0, n).AsStream<IO>();

static StreamT<IO, Unit> example(int n) =>
    from v in naturals(n).FoldUntil(0, (s, x) => s + x, (_, x) => x % 10 == 0)
    from _ in writeLine(v.ToString())
    where false
    select unit;

Here, FoldUntil will take each number in the stream and sum it. In its predicate it returns true every 10th item. We then write the state to the console. The output looks like so:

0
55
210
465
820
1275
1830
2485
3240
4095
..

Support for recursive IO with zero space leaks

I have run the first StreamT example (that printed every 10,00th entry forever) to the point that this has counted over 4 billion. The internal implementation is recursive, so normally we'd expect a stack-overflow, but for lifted IO there's a special trampoline in there that allows it to recurse forever (without space leaks either). What this means is we can use it for long lived event streams without worrying about memory leaks or stack-overflows.

To an extent I see StreamT as a much simpler pipes system. It doesn't have all of the features of pipes, but it is much, much easier to use.

To see more examples, there's a 'Streams' project in the Samples folder.

Typed operators for |

I've added lots of operators for | that keeps the .As() away when doing failure coalescing with the core types.

Atom rationalisation

I've simplified the Atom type:

• No more effects inside the Swap functions (so, no SwapEff, or the like).
• Swap doesn't return an Option any more. This was only needed for atoms with validators. Instead, if a validator fails then we just return the original unchanged item. You can still use the Changed event to see if an actual change has happened. This makes working with atoms a bit more elegant.
• New Prelude functions for using atoms with IO:
  • atomIO to construct an atom
  • swapIO to swap an item in an atom while in an IO monad
  • valueIO to access a snapshot of the Atom
  • writeIO to overwrite the value in the Atom (should be used with care as the update is not based on the previous value)

FoldOption

New FoldOption and FoldBackOption functions for the Foldable trait. These are like FoldUntil, but instead of a predicate function to test for the end of the fold, the folder function itself can return an Option. If None the fold ends with the latest state.

Async helper

• Async.await(Task<A>) - turns a Task into a synchronous process. This is a little bit like Task.Result but without the baggage. The idea here is that you'd use it where you're already in an IO operation, or something that is within its own asynchronous state, to pass a value to a method that doesn't accept Task.
• Async.fork(Func<A>, TimeSpan) and `Async.fork(Func<Task>, TimeS...

IO and effects refactoring

In the last release I wrote this:

"Because the traits are all interfaces we can't use operator | for error handling"

I realised that because @catch creates a temporary struct (the various Catch* record structs) that I could attach operator | to those types and make @catch work for K<F, A> types that are Fallible<F> or Fallible<E, F>.

This took me down a massive rabbit hole! So this release has quite a few changes. If you're using v5 then you'll need to pay attention. And, if you're using runtimes, you'll really need to pay attention!

@catch

As, many of you know, in v4 we can @catch errors raised in the Eff<RT, A> and Eff<A> types by using the | operator like a coalescing operator. For example:

   public static Eff<RT, Unit> main =>
        from _1 in timeout(60 * seconds, longRunning)
                 | @catch(Errors.TimedOut, unit)
        from _2 in Console<Eff<RT>, RT>.writeLine("done")
        select unit;

This imposes a time-limit on the longRunning operation, which throws a TimedOut error if it doesn't finish in time. It then catches the timeout and continues safely by returning a default value of unit.

There were a number of types that @catch (depending on the overload) could create:

• CatchValue<A> - for returning default values (as above)
• CatchValue<E, A> - for returning default values (with generic error type)
• CatchError - for returning an alternative error
• CatchError<E> - for returning an alternative error (with generic error type)
• CatchIO<A> - for returning and lifting an IO<A> as the result
• CatchIO<E, A> - for returning and lifting an IO<A> as the result (with generic error type)
• CatchM<M, A> - for returning and lifting an K<M, A> as the result
• CatchM<E, M, A> - for returning and lifting an K<M, A> as the result (with generic error type)

Each one carries a predicate function and an action function. If the predicate returns true for the error raised then the action is run, otherwise the result it left alone. This means a chain of | catch(...) operations can effectively pattern match the errors raised.

Most importantly: the arguments to @catch can make the inference of the generic parameters automatic, so we don't have to manually write @catch<Error, Eff, int>(...) -- this makes catch usable.

Back to the idea that we have a Fallible<E, F> (and Fallible<F> which is equivalent to Fallible<Error, F>). Because, operator declarations can't have generic parameters, all generic parameters must come from the type.

To be able to leverage the Fallible<E, F> trait then we need F (the trait type), E (the error type), and A (the bound value type):

public interface Fallible<E, F>
{
    public static abstract K<F, A> Catch<A>(
        K<F, A> fa,
        Func<E, bool> Predicate, 
        Func<E, K<F, A>> Fail);

   ...
}

Only one of the Catch* record structs has all of those generics:

• CatchM<E, M, A> - for returning and lifting an K<M, A> as the result (with generic error type)

So, that's the only type that can support an operator | that can work with Fallible<E, M>:

public readonly record struct CatchM<E, M, A>(Func<E, bool> Match, Func<E, K<M, A>> Value)
    where M : Fallible<E, M>
{
    public static K<M, A> operator |(K<M, A> lhs, CatchM<E, M, A> rhs) =>
        lhs.Catch(rhs.Match, rhs.Value);
}

So, I had a couple of options:

1. Add support only to CatchM and leave the other Catch* types as non-Fallible supporting
2. Remove all of the other Catch* types that can't support Fallible

Option 1 would mean that some usages of @catch would work with Eff<A> but not K<Eff, A>. This felt unsatisfactory.
Option 2 would mean that some of the convenience @catch overrides would have to be removed. So, you couldn't write this anymore:

   @catch(Errors.TimedOut, unit)

You'd have to write (one of):

   @catch(Errors.TimedOut, SuccessEff(unit))
   @catch(Errors.TimedOut, pure<Eff, Unit>(unit))
   @catch(Errors.TimedOut, unitEff)  // unitEff is a static readonly of SuccessEff

Option 2 is the option I've gone with. The reasons for this are primarily for consistency between the concrete types (Eff<A>) and their abstract pairs (K<Eff, A>), but also...

Every single Fallible type gets to use @catch!

So, previously, @catch only worked for Eff<RT, A>, Eff<A>, and IO<A>. It now works for:

• IO<A>
• Eff<RT, A>
• Eff<A>
• Either<L, R>
• EitherT<L, M, R>
• Fin<A>
• FinT<M, A> - more on this later
• Option<A>
• OptionT<M, A>
• Try<A>
• TryT<A>
• Validation<F, A>
• ValidationT<F, M, A>

So now all Fallible types get to use @catch and they all get to use the same set (well, some are specifically for the Error type, like @expected and @exceptional, but other than that they're all the same).

Things to note about this change:

• Because @catch is now entirely generic and based around Fallible types, the | operator can only return K<M, A>, so you may need to use .As() if you need to get back to the concrete type.
• For catch-all situations, it's better to not use @catch at all, unless you need access to the error value.

MonadIO refactor

The generalisation of catching any errors from Fallible led to me doing some refactoring of the Eff<RT, A> and Eff<A> types. I realised not all errors were being caught. It appeared to be to do with how the IO monad was lifted into the Eff types. In the Monad<M> trait was a function: WithRunInIO which is directly taken from the equivalent function in Haskell's IO.Unlift package.

I decided that was too complicated to use. Every time I used it, it was turning my head inside out, and if it's like that for me then it's probably unusable for others who are not fully aware of unlifting and what it's about. So, I removed it, and UnliftIO (which depended on it).

I have now moved all lifting and unlifting functions to MonadIO:

public interface MonadIO<M>
    where M : MonadIO<M>, Monad<M>
{
    public static virtual K<M, A> LiftIO<A>(IO<A> ma) =>
        throw new ExceptionalException(Errors.LiftIONotSupported);

    public static virtual K<M, A> LiftIO<A>(K<IO, A> ma) =>
        M.LiftIO(ma.As());

    public static virtual K<M, IO<A>> ToIO<A>(K<M, A> ma) =>
        throw new ExceptionalException(Errors.UnliftIONotSupported);

    public static virtual K<M, B> MapIO<A, B>(K<M, A> ma, Func<IO<A>, IO<B>> f) =>
        M.ToIO(ma).Bind(io => M.LiftIO(f(io)));
}

Monad<M> inherits MonadIO<M>, which isn't how it should be, but because of the limitations of C#'s type system we have all monads expose the MonadIO functionality (otherwise monad-transformers won't work). I'm still thinking through alternative approaches, but I'm a little stumped at the moment. So, for now, there are default implementations for LiftIO and ToIO that throw exceptions. You only implement them if your type supports IO.

• LiftIO as most will know, will lift an IO<A> into your monad-transformer.
• ToIO is the opposite and will unpack the monad-transformer until it gets to the IO monad and will then return that as the bound value.

For example, this is the implementation for ReaderT:

    public static ToIO<A>(K<ReaderT<Env, M>, A> ma) =>
        new ReaderT<Env, M, IO<A>>(env => ma.As().runReader(env).ToIO());

So, we run the reader function with the env environment-value, it will return a K<M, A> which we then call ToIO() on to pass it down the transformer stack. Eventually it reaches the IO monad that just returns itself. This means we run the outer shell of the stack and not the inner IO.

That allows methods like MapIO to operate on the IO<A> monad, rather than the <A> within it:

M.ToIO(ma).Bind(io => M.LiftIO(f(io)));

What does this mean?

• It means you can call .MapIO(...) on any monad that has an IO monad within it (as long as ToIO has been implemented for the whole stack)
• Once we can map the IO we can generalise all of the IO behaviours...

Generalised IO behaviours

The IO<A> monad has many behaviours attached to it:

• Local - for creating a local cancellation environment
• Post - to make the IO computation run on the SynchronizationContext that was captured at the start of the IO operation
• Fork - to make an IO computation run on its own thread
• Await - for awaiting a forked IO operation's completion
• Timeout - to timeout an IO operation if it takes too long
• Bracket - to automatically track resource usage and clean it up when done
• Repeat, RepeatWhile, RepeatUntil - to repeat an IO operation until conditions cause the loop to end
• Retry, RetryWhile, RetryUntil - to retry an IO operation until successful or conditions cause the loop to end
• Fold, FoldWhile, FoldUntil - to repeatedly run an IO operation and aggregating a result until conditions cause the loop to end
• Zip - the ability to run multiple IO effects in parallel and join them in a tuppled result.

Many of the above had multiple overrides, meaning a few thousand lines of code. But, then we put our IO monad inside monad-transformers, or encapsulate them inside types like Eff<A> and suddenly those functions above are not available to us at all. We can't get at the IO<A> monad within to pass as arguments to the IO behaviours.

That's where MapIO comes in. Any monadic type or transformer type that has implemented ToIO (and has an IO<A> monad encapsulated within) can now directly invoke ...

New trait: Fallible

In Haskell there's a trait called MonadFail for monadic types to raise errors. It's not particularly effective as most tend to avoid it. I wanted to create a trait (for types that can fail) that's effective and could help standardise error handling.

That's the new Fallible<E, F> and Fallible<F> trait type...

Fallible

Traits that are fallible can fail (I'm quite smug about the name, I think it's pretty cool, haha)!

• Fallible<E, F> - can have a parameterised failure value E for structure F (usually a functor, applicative, or monad)
• Fallible<F> is equivalent to Fallible<Error, F> - which simplifies usage for the commonly used Error type

Anything that is fallible must implement:

public static abstract K<F, A> Fail<A>(E error);

public static abstract K<F, A> Catch<A>(
    K<F, A> fa,
    Func<E, bool> predicate, 
    Func<E, K<F, A>> fail);

• Fail is for the raising of errors
• Catch can be used to catch an error if it matches a predicate; and if so, it runs the fail function to produce a new structure (which may also be failing, but could be used to rescue the operation and provide a sensible succeeding default).

Fallible module

In the Fallible module there are functions to raise failures:

public static class Fallible
{
    public static K<F, A> fail<E, F, A>(E error)
        where F : Fallible<E, F> =>
        F.Fail<A>(error);
    
    public static K<F, Unit> fail<E, F>(E error)
        where F : Fallible<E, F> =>
        F.Fail<Unit>(error);    
    
    public static K<F, A> error<F, A>(Error error)
        where F : Fallible<Error, F> =>
        F.Fail<A>(error);
    
    public static K<F, Unit> error<F>(Error error)
        where F : Fallible<Error, F> =>
        F.Fail<Unit>(error);    
}

• fail raises the parameterised error types
• error raises the Error type

Because the traits are all interfaces we can't use operator | for error handling (the operators can still be used for concrete types, like Eff<A>, IO<A>, etc.) -- and so there are now lots of Catch extension methods for catching errors in Fallible structures. You can view them here.

Prelude

The Prelude now has:

public static K<F, A> pure<F, A>(A value)
    where F : Applicative<F>;

public static K<F, A> fail<E, F, A>(E error)
    where F : Fallible<E, F>;

public static K<F, Unit> fail<E, F>(E error)
    where F : Fallible<E, F>;

public static K<F, A> error<F, A>(Error error)
    where F : Fallible<F>;

public static K<F, Unit> error<F>(Error error)
    where F : Fallible<F>;

So, for example, you can now construct any type (as long as it implements the Applicative trait) using pure:

var effect = pure<Eff, int>(100);
var option = pure<Option, string>("Hello");
var either = pure<Either<Error>, bool>(true);

And you can construct any type (as long as it implements the Fallible<E, F> trait) using fail or with error (when Fallible<F>):

var effect = error<Eff, int>(Errors.SequenceEmpty);
var trying = error<Try, int>(Errors.EndOfStream);
var option = fail<Unit, Option, string>(unit);
var either = fail<string, Either<string>, bool>("failed!");

Types that have been made Fallible

• IO<A>
• Eff<RT, A>
• Eff<A>
• Either<L, R>
• EitherT<L, M, R>
• Fin<A>
• Option<A>
• OptionT<M, A>
• Try<A>
• TryT<A>
• Validation<F, A>
• ValidationT<F, M, A>

Which means you can use .Catch(...) on all of those types now. For example:

var res = error<Eff, int>(Errors.Cancelled)
            .Catch(Errors.EndOfStream, _ => 0)             // Catch a specific error and ignore with a default 
            .Catch(Errors.Closed, _ => pure<Eff, int>(-1)) // Run an alternative effect
            .Catch(Errors.Cancelled, _ => IO.pure(-2))     // For monads that support IO, launch an IO operation
            .Catch(e => Errors.ParseError(e.ToString()));  // Catch-all mapping to another error

IO changes

• IO.Pure has been renamed IO.pure
  • Capital letters are used for constructor cases (like Some and None in a discriminated union)
  • IO.Pure doesn't create an IO<A> type with a Pure case, it constructs a lambda that returns an A
  • So, I'm going to change the construction functions where they're not really doing what they claim
• IO.Fail has been renamed IO.fail (see above)

Eff running extensions

The various .Run* methods for Eff<A> and Eff<RT, A> have been made into extension methods that work with K<Eff, A> and K<Eff<RT>, A>. That means if you end up with the more abstract representation of Eff you can run it without calling .As() first.

I'll be doing this for other types that are 'run'.

I have also tidied up some of the artefacts around the MinRT runtime used by Eff<A>. Because Eff<A> is now backed by a transformer stack with IO<A> as its inner monad, the MinRT doesn't need to carry any IO environment any more, so I've removed it from MinRT, making MinRT into a completely empty struct. This removes some constraints from the Run* extensions.

Prelude.ignoreF and Functor.IgnoreF

The prelude function ignoreF and equivalent extension method to Functor<F>, IgnoreF are the equivalent of calling .Map(_ => unit) to ignore the bound-value of a structure and instead return unit.

Transducers removed

I have removed the Transducers completely from v5. They were originally going to be the building blocks of higher-kinds, but with the new trait-system I don't think they add enough value, and frankly I do not have the time to bring them through this v5 release process (which is already a mammoth one)! As much as I like transducers, I think we can do better with the traits system now.

Not needed traits removed

The following traits have been removed:

• HasCancel - This was used in the Aff monad and now isn't needed because the IO monad has its own environment which carries the cancellation token
• HasFromError - was used by the transducers, so not needed
• HasIO - was used by the MinRT runtime, which isn't needed anymore
• HasSyncContextIO - as above

New Sample

Those of you who are subscribed to my blog at paullouth.com will have seen the first newsletter this week. It took a while to get off the ground because I refused to use the terrible Mailgun integration in GhostCMS.

Instead I rolled my own, which I've been working on the past few days. So it was an opportunity to test out the effect system and trait system. I took it as far as it can go and the entire application is trait driven. Only when you invoke the application do you specify what monad and runtime to use.

This is the main operation for generating the newsletter and emailing it out to all of the members:

public static class Send<M, RT>
    where RT : 
        Has<M, WebIO>,
        Has<M, JsonIO>,
        Has<M, FileIO>,
        Has<M, EmailIO>,
        Has<M, ConsoleIO>,
        Has<M, EncodingIO>,
        Has<M, DirectoryIO>,
        Reads<M, RT, Config>,
        Reads<M, RT, HttpClient>
    where M :
        Monad<M>,
        Fallible<M>,
        Stateful<M, RT>
{
    public static K<M, Unit> newsletter =>
        from posts     in Posts<M, RT>.readLastFromApi(4)
        from members   in Members<M, RT>.readAll
        from templates in Templates<M, RT>.loadDefault
        from letter    in Newsletter<M, RT>.make(posts, templates)
        from _1        in Newsletter<M, RT>.save(letter)
        from _2        in Display<M, RT>.showWhatsAboutToHappen(members)
        from _3        in askUserToConfirmSend
        from _4        in Email<M, RT>.sendToAll(members, letter)
        from _5        in Display<M, RT>.confirmSent
        select unit;
  
    ..
}

Note how the computation being run is entirely generic: M. Which is constrained to be a Monad, Fallible, and Stateful. The state is RT, also generic, which is constrained to have various IO traits as well as a Config and HttpClient state. This can be run with any type that supports those traits. Completely generic and abstract from the underlying implementation.

Only when we we pass the generic argument to Send<> do we get a concrete implementation:

var result = Send<Eff<Runtime>, Runtime>.newsletter.Run(runtime);

Here, we run the newsletter operation with an Eff<Runtime> monad. But, it could be with any monad we build.

Importantly, it works, so that's good :)

Source code is here . Any questions, ask in the comments below...

StateT bug fix + monadic conditionals

• Fixed: a bug in the StateT monad-transformer. One of the SelectMany overloads wasn't propagating the state correctly.
• Changed: Prelude.local that creates a local IO and resource environment renamed to localIO to avoid conflicts with ReaderT.local and Reader.local
• Added: general purpose liftIO in Prelude
• Added: variants of when and unless that take a K<M, bool> as the source of the flag. Means any monad that binds a bool can be used directly in when and unless, rather than having to lower it first.
• Added: new monadic conditional: iff - works like when and unless, but has an else case. K<M, bool> can be used directly also, meaning that if/then/else monadic expressions can be built without lowering.
• Added: applicative actions to the Prelude. Allows for chaining n applicative actions, discarding their results, apart from the last one, which is returned

Fix for: use of custom sub-type errors in IO monads

This is a minor release to fix: issue 1340.

Thanks to @HernanFAR for raising the issue with concise repro steps 👍