Skip to content

More capable Type objects #4200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
eernstg opened this issue Dec 9, 2024 · 144 comments
Open

More capable Type objects #4200

eernstg opened this issue Dec 9, 2024 · 144 comments
Labels
feature Proposed language feature that solves one or more problems meta-classes

Comments

@eernstg
Copy link
Member

eernstg commented Dec 9, 2024

This issue is a response to #356 and other issues requesting virtual static methods or the ability to create a new instance based on a type variable, and similar features.

Static substitutability is hard

The main difficulty with existing proposals in this area is that the set of static members and constructors declared by any given class/mixin/enum/extension type declaration has no interface and no subtype relationships:

class A {
  A();
  A.named(): this();
  static int foo => 1;
  static int bar => 2;
}

class B extends A {
  B();
  static int foo => -1;
  static int baz => -3;
}

As a fundamental OO fact, B is an enhanced version of A when it comes to instance members (even in this case where we don't enhance anything), but it is simply completely unrelated when it comes to constructors and static members.

In particular, the relationship between the constructors A(); and B(); is very different from an override relationship. A has a constructor named A.named but B doesn't have a constructor named B.named. The static member B.foo does not override A.foo. B does not inherit A.bar. In general, none of the mechanisms and constraints that allow subtype substitutability when it comes to instance members are available when it comes to "class members" (that is, static members and constructors).

Consequently, it would be a massively breaking change to introduce a rule that requires subtype substitutability with respect to class members (apart from the fact that we would need to come up with a precise definition of what that means). This means that it is a highly non-trivial effort to introduce the concept which has been known as a 'static interface' in #356.

This comment mentions the approach which has been taken in C#. This issue suggests going in a different direction that seems like a better match for Dart. The main difference is that the C# approach introduces an actual static interface (static members in an interface that must be implemented by the class that claims to be an implementation of that interface). The approach proposed here transforms the static members into instance members, which means that we immediately have the entire language and all the normal subsumption mechanisms, we don't have to build an entirely new machine for static members.

What's the benefit?

It has been proposed many times, going back to 2013 at least, that an instance of Type that reifies a class C should be able to do a number of things that C can do. E.g., if we can do C() in order to obtain a new instance of the class C then we should also be able to do MyType() to obtain such an instance when we have var MyType = C;. Similarly for T() when T is a type variable whose value is C.

Another set of requests in this topic area is that static members should be virtual. This is trivially true with this proposal because we're using instance members of the reified Type objects to manage the access to the static members.

There are several different use cases. A major one is serialization/deserialization where we may frequently need to create instances of a class which is not statically known, and we may wish to call a "virtual static method".

Proposal

We introduce a new kind of type declaration header clause, static implements, which is used to indicate that the given declaration must satisfy some subtype-like constraints on the set of static members and constructors.

The operand(s) of this clause are regular class/mixin/mixin-class declarations, and the subtype constraints are based on the instance members of these operands. In other words, they are supertypes (of "something"!) in a completely standard way (and the novelty arises because of that "something").

The core idea is that this "something" is a computed set of instance members, amounting to a correct override of each of the instance members of the combined interface of the static implements types.

abstract class A<X> {
  int get foo;
  void bar();
  X call(int _);
  X named(int _, int _);
}

class B static implements A<B> {
  final int i;

  B(this.i);
  B.named(int i, int j): this(i + j);
  
  static int get foo => 1;
  static void bar() {}
}

These declarations have no compile-time errors. The static analysis notes the static implements clause, computes the corresponding meta-member for each static member and for each constructor, and checks that the resulting set of meta-members amount to a correct and complete set of instance members for a class that implements A<B>. Here is the set of meta-members (note that they are implicitly created by the tools, not written by a person):

mixin MetaMembers_Of_B on Type implements A<B> {
  B call(int i) => B(i);
  B named(int i, int j) => B.named(i, j);
  int get foo => B.foo;
  void bar() => B.bar();
}

The constructor named B becomes an instance method named call that takes the same arguments and returns a B. Similarly, the constructor named B.named becomes an instance method named named. Static members become instance members, with the same name and the same signature.

The point is that we can now change the result of type Type which is returned by evaluating B such that it includes this mixin.

This implies that for each constructor and static member of B, we can call a corresponding instance member of its Type:

void main() {
  dynamic t = B; // `t` is the `Type` that reifies `B`.
  t(10); // Similar to `B(10)`, yields a fresh `B`.
  t.named(20, 30); // Ditto, for `B.named(20, 30)`.
  t.foo; // Similar to `B.foo`.
  t.bar(); // Similar to `B.bar()`.
}

This shows that the given Type object has the required instance members, and we can use them to get the same effect as that of calling constructors and static members of B.

We used the type dynamic above because those methods are not members of the interface of Type. However, we could change the typing of type literal expressions such that are not just Type. They could be Type & M in every situation where it is known that the reified type has a given mixin M. We would then be able to use the following typed approach:

class C static implements A<C> {
  final int i, j;

  C(int i): this(i, i);
  C.named(this.i, this.j): assert(i < j);
  
  static int get foo => 1000;
  static void bar() {}
}

void main() {
  var t = B; // `T` has type `Type & MetaMembers_Of_B`.

  // With that in place, all of these are now statically checked.
  t(10); t.named(20, 30); t.foo; t.bar();

  // We can also use the type `A` in order to abstract the concrete class away.
  X f<X>(A<X> a) {
    a.bar();
    return switch (a.foo) {
      1 => a(),
      _ => a.named(),
    };
  }

  B b = f(B);
  C c = f(C);
}

Next, we could treat members invoked on type variables specially, such that T.baz() means (T).baz(). This turns T into an instance of Type, which means that we have access to all the meta members of the type. This is a plausible treatment because type variables don't have static members (not even if and when we get static extensions), so T.baz() is definitely an error today.

We would need to consider exactly how to characterize a type variable as having a reified representation that has a certain interface. Let us use the following, based on the syntax of regular type parameter bounds:

X f<X static extends A<X>>() { // New relationship that `B` and `C` satisfy.
  X.bar();
  return switch (X.foo) {
    1 => X(),
    _ => X.named(),
  };
}

void main() {
  B b = f(); // Inferred as `f<B>();`.
  C c = f(); // Inferred as `f<C>();`.
}

Even if it turns out to be hard to handle type variables so smoothly, we could of course test it at run time:

X g<X>() { // No specialized bound.
  var Xreified = X;
  if (Xreified is! A<X>) throw "Ouch!";
  Xreified.bar();
  return switch (Xreified.foo) {
    1 => Xreified(),
    _ => Xreified.named(),
  };
}

void main() {
  B b = f(); // Inferred as `f<B>();`.
  C c = f(); // Inferred as `f<C>();`.
}

Customized behavior

The behavior of the reified type objects can be customized, that is, they can do other things than just forwarding a call to a static member or a constructor.

One possible approach could be to have static extends C with M1 .. Mk in addition to static implements T1 .. Tn on type introducing declarations (like classes and mixins), and then generate the code for the reified type object such that it becomes a subclass that extends C with M1 .. Mk and also implements T1 .. Tn. However, we could also apply those mixins outside the static extends clause, so we only consider a simpler mechanism:

A type introducing declaration can include a static extends C clause. This implies that the reified type object will be generated such that the given C is the superclass. Compile-time errors occur according to this treatment. E.g., if C is a sealed class from a different library then static extends C is an error, based on the fact that it would give rise to a subclass relation to that class, which is an error.

This mechanism allows the reified type object to have arbitrary behaviors which can be written as code in the class which is being used as the static extends operand.

Use case: An existential open mechanism

One particular kind of feature which could be very useful is a flexible mechanism that delivers the behaviors otherwise obtained by an existential open mechanism. In other words, a mechanism that allows the actual type arguments of a given class to be accessed as types.

This would not involve changes to the type system (so it's a much smaller feature than a real existential open would be). It is less strictly checked at compile time, but it will do the job—and it will presumably be used sparingly, in just that crucial bit of code that allows a given API to be more convenient to use, and the API itself would be statically typed just like any other part of the system.

For example, the reified type object could implement this interface:

abstract class CallWithTypeParameters {
  int get numberOfTypeParameters;
  R callWithTypeParameter<R>(int index, R Function<T>() callback);
}

A class C with a single type parameter X could use static extends _CallWithOneTypeParameter<X>, which would make it implement CallWithTypeParameters:

abstract class _CallWithOneTypeParameter<E> implements CallWithTypeParameters {
  int get numberOfTypeParameters => 1;
  R callWithTypeParameter<R>(int index, R Function<Y>() callback) {
    if (index != 1) {
      throw ArgumentError("Index 1 expected, got $index");
    }
    return callback<E>();
  }
}

For example, assume that the standard collection classes will use this (note that this is a non-breaking change):

abstract mixin class Iterable<E> static extends _CallWithOneTypeParameter<E> {
  ... 
}

abstract interface class List<E> static extends _CallWithOneTypeParameter<E>
    implements Iterable<E>, ... {
  ...
}
...

The 'existential open' feature is so general that it would make sense to expect system provided classes to support it. It also makes sense for this feature to be associated with a very general interface like CallWithTypeParameters, such that all the locations in code where this kind of feature is needed can rely on a well-known and widely available interface.

If we have this feature then we can deconstruct a type of the form List<T>, Set<T>, ..., and use the actual type argument:

X build<X, Y>(Y y) {
  if (<X>[] is List<List>) {
    // `X` is `List<Z>` for some `Z`.
    final reifiedX = X as CallWithTypeParameters;
    return reifiedX.callWithTypeParameter(1, <Z>() {
      return <Z>[build<Z, Y>(y)];
    }) as X;
  } else if (<X>[] is List<Set>) {
    // `X` is `Set<Z>` for some `Z`.
    final reifiedX = X as CallWithTypeParameters;
    return reifiedX.callWithTypeParameter(1, <Z>() {
      return <Z>{build<Z, Y>(y)};
    }) as X;    
  } else if (<Y>[] is List<X>) {
    // `Y <: X`, so we can return `y`.
    return y as X;
  } else {
    throw ArgumentError("Inconsistent call `build<$X, $Y>(_)");
  }
}

void main() {
  String v1 = build('Hello!'); // Passthrough, 'Hello!'.
  List<num> v2 = build(1); // `<num>[1]`.
  Set<List<int>> v3 = build(2); // `<List<int>>{<int>[2]}`.
  print('v1: ${v1.runtimeType} = "$v1"');
  print('v2: ${v2.runtimeType} = $v2');
  print('v3: ${v3.runtimeType} = $v3');
}

Obviously, this involves some delicate low-level coding, and it needs to be done carefully. However, the resulting API may be considerably more convenient than the alternatives.

In particular, an API could use regular objects that "represent" the composite types and their type arguments. Those type representations would then be handled by an interpreter inside build. For example, with this kind of approach it is probably not possible to obtain a helpful return type, and it is certainly not possible to use type inference to obtain an object that "represents" a type like Set<List<int>>.

Running code, emulating the example above.
abstract class CallWithTypeParameters {
  int get numberOfTypeParameters;
  R callWithTypeParameter<R>(int index, R Function<T>() callback);
}

abstract class _CallWithOneTypeParameter<E> implements CallWithTypeParameters {
  int get numberOfTypeParameters => 1;
  R callWithTypeParameter<R>(int index, R Function<Y>() callback) {
    if (index != 1) {
      throw ArgumentError("Index 1 expected, got $index");
    }
    return callback<E>();
  }
}

// Assume `static extends _CallWithOneTypeParameter<E>` in collection types.

class ReifiedTypeForList<E> extends _CallWithOneTypeParameter<E> {}
class ReifiedTypeForSet<E> extends _CallWithOneTypeParameter<E> {}

// Workaround: Allow the following types to be used as an expression.
typedef _ListNum = List<num>;
typedef _ListInt = List<int>;
typedef _SetListInt = Set<List<int>>;

CallWithTypeParameters? emulateFeature<X>() {
  return switch (X) {
    const (_ListNum) => ReifiedTypeForList<num>(),
    const (_ListInt) => ReifiedTypeForList<int>(),
    const (_SetListInt) => ReifiedTypeForSet<List<int>>(),
    _ => null,
  };
}

X build<X, Y>(Y y) {
  if (<X>[] is List<List>) {
    // `X` is `List<Z>` for some `Z`.
    final reifiedX = emulateFeature<X>() as CallWithTypeParameters;
    return reifiedX.callWithTypeParameter(1, <Z>() {
      return <Z>[build<Z, Y>(y)];
    }) as X;
  } else if (<X>[] is List<Set>) {
    // `X` is `Set<Z>` for some `Z`.
    final reifiedX = emulateFeature<X>() as CallWithTypeParameters;
    return reifiedX.callWithTypeParameter(1, <Z>() {
      return <Z>{build<Z, Y>(y)};
    }) as X;    
  } else if (<Y>[] is List<X>) {
    // `Y <: X`, so we can return `y`.
    return y as X;
  } else {
    throw ArgumentError("Inconsistent call `build<$X, $Y>(_)");
  }
}

void main() {
  String v1 = build('Hello!'); // Passthrough, 'Hello!'.
  List<num> v2 = build(1); // `<num>[1]`.
  Set<List<int>> v3 = build(2); // `<List<int>>{<int>[2]}`.
  print('v1: ${v1.runtimeType} = "$v1"');
  print('v2: ${v2.runtimeType} = $v2');
  print('v3: ${v3.runtimeType} = $v3');
}

Type parameter management

With this mechanism, a number of classes will be implicitly induced (that is, the compiler will generate them), and they will be used to create the reified type object which is obtained by evaluating the corresponding type as an expression.

The generated class will always have exactly the same type parameter declarations as the target class: If class C has 3 type parameters with specific bounds then the generated class will have the same type parameter declarations with the same bounds. This implies that T in static implements T or static extends T is well defined.

abstract class A<Y> {
  void foo();
  List<Y> get aList => <Y>[];
}

class C<X1 extends B1, X2 extends B2> static extends A<X2> {
  static void foo() => print('C.foo running!');
}

// Implicitly generated class.
class ReifiedTypeForC<X1 extends B1, X2 extends B2> extends A<X2> {
  void foo() => C.foo();
  // Inherited: `List<X2> get aList => <X2>[];`
}

// Example, where `String` and `int` are assumed to satisfy the bounds.
void main() {
  void f<X static extends A>() {
    X.foo(); // Prints 'C.foo running!'.
    print(X.aList.runtimeType); // 'List<int>'.
  }
  
  f<C<String, int>>();
}

More capable type objects as an extension

We may well wish to equip an existing type that we aren't able to edit (say, String) with reified type object support for a particular class.

We can do this by adding a "magic" static member on the class Type as follows:

class Type {
  ...
  static DesiredType? reify<TypeToReify, DesiredType>() {...}
}

This is, at first, just a less convenient way to reify a type (passed as the type argument TypeToReify) into a reified type object. With an actual type argument of Type (or any supertype thereof), the returned object is simply going to be the reified type object that you would also get by simply evaluating the given TypeToReify as an expression.

However, if DesiredType is not a supertype of Type then the reified type object may or may not satisfy the type constraint (that is, it may or may not be an instance of a subtype of DesiredType). If it is not an instance of the specified DesiredType then reify will attempt to use an extension of the reified type of TypeToReify.

Such extensions can be declared in an extension declaration. For example:

extension E<X> on List<X> {
  static extends _CallWithOneTypeParameter<X>;
}

At the location where Type.reify<List<int>, CallWithTypeParameters>() is invoked, we gather all extensions in scope (declared in the same library, or imported from some other library), whose on-type can be instantiated to be the given TypeToReify. In an example where the given TypeToReify is List<int>, we're matching it with List<X>.

For this matching process, the first step is to search the superinterface graph of the TypeToReify to find the class which is the on-type of the extension. In the example there is no need to go to a superinterface, the TypeToReify and the on-type of the extension are already both of the form List<_>.

Next, the value of the actual type arguments in the chosen superinterface of the TypeToReify is bound to the corresponding type parameter of the extension. With List<int> matched to List<X>, X is bound to int.

Next, it is checked whether there is a static implements T or static extends T clause in the extension such that the result of substituting the actual type arguments for the type parameters in T is a subtype of DesiredType.

In the example where TypeToReify is List<int> and DesiredType is CallWithTypeParameters, we find the substituted static implements type to be _CallWithOneTypeParameter<int>, which is indeed a subtype of CallWithTypeParameters.

If more than one extension provides a candidate for the result, a compile-time error occurs.

Otherwise, the given reified type object is returned. In the example, this will be an instance of the implicitly generated class for this static implements clause:

class ExtensionE_ReifiedTypeForList<X> extends _CallWithOneTypeParameter<X>
    implements Type {}

The result is that we can write static implements and static extends clauses in extensions, and as long as we're using the long-form reification Type.reify<MyTypeVariable, CallWithTypeParameters>(), we can obtain an "alternative reified object" which is specifically tailored to handle the task that CallWithTypeParameters was designed for.

If we're asking for some other type then we might get it from the reified type object of the class itself, or perhaps from some other extension.

Finally, if the reified type object of the class/mixin/etc. itself doesn't have the DesiredType, and no extensions will provide one, then Type.reify returns null.

It is going to be slightly less convenient to use Type.reify than it is to simply evaluate the type literal as an expression, but the added expressive power will probably imply that Type.reify will be used for all the more complex cases.

Revisions

  • Feb 21, 2025: Added a section about extension-like reified type objects. Added a section about how to manage type parameters in the implicitly induced class that defines the reified type object.
  • Feb 20, 2025: Further developed the ideas about customized behavior.
  • Feb 14, 2025: Add the section about customized behavior.
  • Dec 10, 2024: Adjust the mixin to be on Type.
  • Dec 9, 2024: First version.
@eernstg eernstg added feature Proposed language feature that solves one or more problems meta-classes labels Dec 9, 2024
@lrhn
Copy link
Member

lrhn commented Dec 9, 2024

The approach proposed here transforms the static members into instance members,

That sounds like a Kotlin companion object.
I don't claim to understand Kotlin, but my understanding is that Kotlin doesn't have static members as such, only instance members on companion objects, which can be called just like static members otherwise would. The difference is that you can add interfaces to the companion object (otherwise each companion object is a singleton class instance with no relations to other classes) and that you can access the companion object as an object, and pass it around.

static implements

Can we have static extends to inherit static members? static with to add mixins?
(Or should we go the companion object way and require you to use an embedded declaration if you want more control,
like:

class C {
  // Before any static member:
  static class extends A.class with HelperMixin1 implements HelperType2;
  // Following static members are membes of this static class.
}     

and you can access the type as C.class, or C.static. Or something.

You can only extend or implement a Name.class type in another static companion class, to ensure that they all extend the real Type.

The constructor named B becomes an instance method named call

We probably still want to ensure that tear-offs from a static-member type is a canonicalized constant. Somehow. (That's one of the reasons I don't want normal classes to be able to implement the special singleton classes.)

To make this equivalent to the current behavior, I assume constructors have access to the type parameters of the
type object, so var l = List<num>; var vs = l.filled(24, 42); will be the same as var vs = List<num>.filled(24, 42);.
The other static members have no access to type variables.
And we want List<int>.copyRange and List<String>.copyRange (let's assume there is a static helper function with that name on List) to be identical, even if they are torn off from different instances. So some care is needed for that.

(Also, it's currently possible to have class C { C(); static void call() {} }, so just converting every static member to an instance member, and unnamed constructor to a call method, can be a conflict. One of them have to surrende.
Alternatively we can allow new as a member name that can only be declared indirectly using a constructor, but that still means you can't write (C)() and invoke the constructor. Or the call method. The ambiguity is still there.)

Type & M

Rather than needing this intersection, just let the generated mixin be on Type and have it actually extend the real Type type.
All such types implement Type, and their own companion interface MyName.class.

X static extends A

Probably want both a non-static and static type bound, say X extends Widget static extends Jsonable.
Having to throw away one of the types to use the other is going to be a problem.


This will be yet another reason to allow static and instance members with the same name.
I'd like to declare the toString of my static member objects, or have it be comparable, while the class itself is also comparable.
It'll happen. If not with this, then with static extensions, or any other feature that makes statics more important.

How will this interact with runtimeType?
Will A().runtimeType return the static Type object. (Yes, what else?)
Will the return type of runtimeType default to A.class when it's not overridden? (No, it's a virtual getter, so subclasses must override correctly, and there is no default type relation between the static member object's types.)

So, runtimeType stays useless, but if anyone does:

dynamic x;
// ...
 ...  (Object? o) { 
    ...
    x = o.runtimeType;   
    ...
...
x.foo();

we may have to retain a lot of static methods that could be tree-shaken today, because we can't tell statically whether they're invoked or not.
That's one advantage with static declarations today: they're either visibly used, or they're dead code.
When we turn static declarations into instance members, and allow casting the instance to Object and then dynamic, they're no longer static methods, and resolution is only going to be an approximation.

@ghost
Copy link

ghost commented Dec 9, 2024

print((int).runtimeType); // _Type
print((int).runtimeType.runtimeType); // _Type
print((String).runtimeType); // _Type

But... if int and String, as type objects, have the same type (_Type), they should have the same methods. But, according to my reading of the proposal, (int).parse("0") will be valid, but (String).parse("0") won't. 😕

(I think, the explanation is that (String).runtimeType is _Type, but String.runtimeType is not defined - but this difference is just an artifact of syntax. Will "Hello".runtimeType.runtimeType be different from ("Hello".runtimeType).runtimeType? Probably not. Not sure what to make of this 😄)

Maybe a function like static(String) can solve the problem? It will return something like _Static$String, which will be a regular object that can be a part of expression like static(MyClass) is MyInterface, static(MyClass) as MyInterface etc.

@lrhn
Copy link
Member

lrhn commented Dec 10, 2024

I would say that int.runtimType returns the same value as the expression int, but with static type Type.
The runtime type would be the type denoted by, fx, int.class, which is a subtype of Type.

Then int.runtimeType.runtimeType has the runtime type int.class.class.
We may want to limit the recursion here. Maybe say that the second-level runtime type is always the same type, plain Type, since the second level runtime types have no members at all. Any static member object with no members is a Type with no extra mixin, and since you can't declare a static static member, only the first level of .class can be proper subtypes of Type.

Or maybe that's a bad idea, because of what it does to tree shaking.
Maybe it's better to let runtimeType return a plain Type object representing the type, without the extra members from the static declarations.
It must still be equal to the object with the static members, but may not have the same runtimeType.

That is:

  • (strawman syntax) if A denotes a type declaration, then A.class denotes the static and runtime type of the expression A or A<T1,…,T2>.
  • If A declares no static members, and either has no constructors or the declaration is abstract, then A.class is the type Type
  • If A does declare a static member, or a constructor and is not abstract, then A.class denotes an implicit, unnamed subclass of Type which has the static members as instance members. That class has no static members or constructors. It may or may not be allowed to declare static operators, maybe including operator==
  • The object returned by Object.runtimeType is a plain Type object, with no extra instance members. If A.class does not override Type.==, then A().runtimeType == A, even of the latter has more members.

A generic type's class objects have instances for each instantiation, and the constructor members can access those.
The object for List<int>.class differs from List<String>.class as of they were different instantiations of a generic class (same generic mixin applied to Type, but different instantiation, so different runtime types, and constructor methods have different return types.)

The getters and setters of static variables are not represented by instance variables, they all access the same global variable's state.

@eernstg
Copy link
Member Author

eernstg commented Dec 10, 2024

Great comments and questions, @lrhn!

That sounds like a Kotlin companion object.

It's similar to Kotlin (and Scala) companion objects, but also different in other ways:

I'm not suggesting that Dart classes should stop having the static members and constructors that we know today, I'm just proposing that we should allow developers to request access to some of the static members and constructors (determined by the interface of the T in static implements T) using the reified types that we already have.

This differs from the companion objects in that there is no support for obtaining a companion object from a given actual type argument (because the type argument was erased). In contrast, a Dart type argument will always be able to deliver the corresponding Type object (just evaluate the corresponding type parameter as an expression). We may then invoke the (forwarding methods to the) static members and constructors of the underlying type, if this access has been provided.

For example:

abstract class A<X> { X call(); }

class B1 static implements A<B1> {}
class B2 {}

void main() {
  var type = someExpression ? B1 : B2;
  if (type is A) {
    var newObject = type();
  }
}

This means that the Kotlin/Scala developer must predict the need to invoke static members or constructors up front (when the concrete type is known) and must pass a reference to the required companion object(s) along with the base data. In contrast, Dart programs can pass type arguments along in the same way they do today, and then it will be possible to get access to the static members and constructors arbitrarily late, as long as the given type is available as the value of a type parameter.

In the case where a class does not have a static implements clause, the reified Type will be exactly the same as today, it's only the classes that are explicitly requesting this feature which will have a somewhat more expensive reified type.

Can we have static extends to inherit static members? static with to add mixins?

This seems to imply that we would implicitly generate a static member or a constructor from the instance method implementations. This may or may not be doable, but I'm not quite sure what the desired semantics should be.

When it comes to code reuse I would prefer to have good support for forwarding, possibly including some abstraction mechanisms (such that we could introduce sets of forwarding methods rather than specifying them one by one). This could then be used to populate the actual class with actual static members as needed, such that it is able to have the desired static implements clause.

We probably still want to ensure that tear-offs from a static-member type is a canonicalized constant

I think the current approach to constants based on constructors and static members is working quite well. The ability to access static members and constructors indirectly via a reified type is an inherently dynamic feature, and I don't think it's useful to try to shoehorn it into a shape where it can yield constant expressions. There's no point in abstracting over something that you already know at compile time anyway.

Rather than needing this intersection, just let the generated mixin be on Type and have it actually extend the real Type type.

Good point! Done.

Will A().runtimeType return the static Type object. (Yes, what else?)

I'm not quite sure what A means here. In my examples A is an abstract class which is used as the operand of static implements (that is, it's the "static interface" that the class B and C declare the required static members to "statically implement"), but it might just as well have been a concrete class (B and C don't care).

In that case, A().runtimeType would evaluate to the reified instance of Type (or a subtype) that represents the class A. It may or may not have some instance forwarders to static/constructor members of A, just like any other reified type. The whole thing is "getting rather meta" really quickly if A is used in static implements clauses of other classes, and also has its own static implements clause, but I don't see why it wouldn't work.

That's one advantage with static declarations today: they're either visibly used, or they're dead code.

A class that doesn't have a static implements clause doesn't have any new ways to invoke its static members or constructors, so they can be tree-shaken just as well as today.

A class that does have a static implements clause makes some of its static members and constructors callable from the implicitly generated forwarding instance members, but this is no worse than the following:

class A {
  static int get foo => 1;
}

class B {
  void foo() {
    print(A.foo);
  }
}

We may still be able to detect that B.foo is never invoked, and no other call sites for B.foo exist, so B.foo can be tree-shaken out of the program.

With "more capable Type objects" we need to track one more thing: Does it ever happen that a reified type object for a given class/mixin/etc. is created? If this may happen then we may obtain an object which is capable of calling a static method (via a forwarding instance member of that reified type object).

It may be harder, but it does sound like a task whose complexity is similar to that of tracking invocations of instance members. That is, if we're able to determine that B.foo above is definitely not called, couldn't we also determine that it is never going to be the case that a reified type object for a given class is created?

@eernstg
Copy link
Member Author

eernstg commented Dec 10, 2024

@tatumizer wrote:

But... if int and String, as type objects, have the same type (_Type), they should have the same methods. But, according to my reading of the proposal, (int).parse("0") will be valid, but (String).parse("0") won't. 😕

If you evaluate a type literal (such as int or String, but let's just use an example where we can decide whether or not there is a static implements clause on the class), the result will have static type Type. The run-time type is not specified currently, but the actual implementations may use a specific private _Type, or whatever they want. Nobody has a lower bound on this run-time type, just like it is with almost any other run-time type.

In particular, it is certainly possible for an implementation to evaluate B and obtain a reified type whose run-time type is of the form MetaMembers_Of_B (which is a subtype of Type and a subtype of A<B>, in the example).

Those reified objects may then have different interfaces, that is, they support invocations of different sets of members, so certainly it's possible for (int).parse("0") to be (1) statically type correct, and (2) supported by the expected implementation (which is the static method int.parse) at run time.

On the other hand, the reified String type does not have a parse instance method, because there is no parse static method in String to forward to (and hence it would be a compile-time error for String to have a static implements Something where Something has a String parse(); member). So (String).parse("0") is a compile-time error, and (String as dynamic).parse("0") throws at run time.

There's nothing special about this (and that's basically the point: I want this mechanism to use all the well-known OO mechanisms to provide flexible/abstract access to the static members and constructors which are otherwise oddly inflexible, from an OO perspective).

Maybe a function like static(String) can solve the problem? It will return something like _Static$String, which will be a regular object that can be a part of expression like static(MyClass) is MyInterface, static(MyClass) as MyInterface etc.

If we have B and C with a static implements A<...> clause then B evaluated as an expression is a regular object. It is also a reified type, but that doesn't prevent that it can be a perfectly normal object with normal semantics and applicability.

So we can certainly do B is MyInterface in order to detect whether the class B has MyInterface as a static interface. We would presumably evaluate the type literal and get a Type object and store it in a local variable in order to be able to promote the reified type such that we can use this fact:

abstract interface class MyInterface {
  int get foo;
}

class D static implements MyInterface {
  static int get foo => 1;
}

void f<X extends D>() {
  var reifiedX = X;
  if (reifiedX is MyInterface) {
    print(reifiedX.foo);
  }
}

@Wdestroier
Copy link

Wdestroier commented Dec 10, 2024

Could the following code similar to C#'s syntax:

abstract class Json {
  static fromJson(Map<String, dynamic> json);
  Map<String, dynamic> toJson();
}

class User implements Json { ... }

be syntatic sugar for this proposal's syntax?

abstract class Json$1 {
  Map<String, dynamic> toJson();
}

abstract class Json$2 {
  fromJson(Map<String, dynamic> json);
}

class User implements Json$1 static implements Json$2 { ... }

To keep static implements as an internal Dart abstraction / implementation detail, but I may be missing edge cases.

EDIT:
I chatted with mateusfccp and he commented "This would be the same as just making static part of the interface, which would basically break the entire universe". To avoid this problem, the base class must have the static method without a body (or marked as abstract). Another point was "you wouldn't be able to provide a default implementation, or else it would become a regular static method". If a static method with an implementation has to be abstract (probably rare), then it could have an abstract modifier imo.

I can't imagine any piece of code implementing the same interface for instance methods and static methods. However, I can imagine most use cases implementing an interface with instance methods and static methods. That's why I would prefer if instance and static interfaces were merged.

@ghost
Copy link

ghost commented Dec 10, 2024

@eernstg wrote:

So we can certainly do B is MyInterface in order to detect whether the class B has MyInterface as a static interface

This can't be! Today String is Object is tautologically true, but under the proposed reforms, the predicate will acquire a new meaning: "String implements static interface of Object", which is not true: static interface of Object includes the method hash and a couple of others, but these methods are not inherited by String. No, we need a different hierarchy and a different syntax for the "companions". Maybe String.class will do, not sure.
The way it's defined above is difficult to wrap my head around. 😄

(An argument against String.class syntax is that we will now have the notions of type and class, which might be very confusiing. We need a syntax for "get static interface of type B", so I think static(B) or B.static would be more appropriate)

@lrhn
Copy link
Member

lrhn commented Dec 10, 2024

String is Object is tautologically true because the expression String evaluates to a Type object, and Type is a subtype of Object.

The Type object that Foo of class Foo static implements Bar evaluates to is an object that implements Type and Bar. It will certainly have a toString and hashCode implementation because it's an object (and an Object).

That doesn't mean that Foo has a static toString, but it does absolutely mean that (Foo).toString() will be allowed, the "static interface implementation object" (or whatever we'll call it) that the expression Foo evaluates to does implement Object, and Type, and in this case also Bar.

(I chose String.class as strawman syntax because it's for accessing "class members", but also mainly because static is not a reserved word, so Foo.static might mean something. It better not, but it could.
You can do some weird stuff with classes and extensions if you really want to.
Like the expression static(C).static(C.static.static.C).static(C).static.)

Spoiler:

void main() {
  static(C).static(C.static.static.C).static(C).static;
}

class C {
  static C get static => C();
  C call(Object? value) => this;
}

C static(Type value) => C();

typedef _C = C;
extension on _C {
  _C get static => this;
  _C get C => this;
}

@ghost
Copy link

ghost commented Dec 10, 2024

Just to clarify: speaking of Object methods, I didn't mean hashCode or toString. I meant static methods of Object, which are:
hash, hashAll and hashAllUnordered. But I see your point: class String.static certainly implements Object, but it doesn't implement Object.static. To implement Object.static, class A has to say class A static implements Object, right?

static is not a reserved word, so Foo.static might mean something

That's not the reason to disqualify the word - in practice, it won't hurt anyone. Most people believe static is a keyword anyway. And if used in the form static(A), you can be quite certain no one has a global method called static. (It might be even possible to reserve the keyword static retroactively).

Still, it's not clear what static(A).runtimeType or static(static(A)) will evaluate to. Some synthetic types like _Static$A and _Static respectively?

@lrhn
Copy link
Member

lrhn commented Dec 10, 2024

Class A It would have to say class A static implements Object.static, which would give it nothing (as I now understand @eernstg's proposal) because Object won't have any static implements clause, so Object.static is just Type.

@ghost
Copy link

ghost commented Dec 10, 2024

If A is a regular class, we can always say class B implements A, and implement all methods of A in B.
Similarly, I guess the class can say class B static implements A.static and implement static methods of A in B (or else the compiler will complain about unimplemented methods).
So, if we declare class B static implements Object.static we will have to implement no-arg constructor, hash, hashAll and hashAllUnordered.
or else the compiler will complain. (Just a guess)

Q: Can class say class B /* no "static"! */ implements A.static? And what will it even mean? (Probably not, b/c "this" type mistmatch).

@ghost
Copy link

ghost commented Dec 11, 2024

Kotlin's companion object model is worth looking into, it won't take long: https://kotlinlang.org/docs/object-declarations.html#companion-objects

Main difference is that the companion object has a name and a type, and - most importantly - Kotlin has a word for it, which is (predictably) "companion object". It would be very difficult to even talk about this concept without the word, so I will use it below.

The idea is that you can extract a companion object from the object, e.g., (using dart's syntax), FooCompanion companion = foo.Companion (the name is by default capitalized, but you can assign any name). This immediately brings the companion object to a familiar territory, thus radically simplifying understanding.

I don't know if there's a simple way to express the same concept in dart without introducing the term "companion object". Does anyone have issues with this wording?

Interestingly, you very rarely need to extract the companion object explicitly, but the very possibility of doing so explains a lot: it's a real object; consequently, it has a type; this type can extend or implement other types - the whole mental model revolves around the term "companion object".

@eernstg
Copy link
Member Author

eernstg commented Dec 11, 2024

@Wdestroier wrote:

Could the following ... be syntactic sugar for this proposal's syntax:

abstract class Json {
  static fromJson(Map<String, dynamic> json);
  Map<String, dynamic> toJson();
}

class User implements Json { ... }

Desugared:

abstract class Json$1 {
  Map<String, dynamic> toJson();
}

abstract class Json$2 {
  fromJson(Map<String, dynamic> json);
}

class User implements Json$1 static implements Json$2 { ... }

I agree with @mateusfccp that it is going to break the world if a clause like implements Json implies not just the instance member constraints that we have today, but also a similar set of constraints on the static members (and, we should remember, constructors!).

In other words, it's crucial for this proposal that static implements has no effect on the subtypes of the declaration that has this clause, each class starts from scratch with respect to the static interface.

abstract class StaticInterface1 { int get foo; }
abstract class StaticInterface2 { void bar(); }

class B static implements StaticInterface1 {
  final int i;
  B(this.i);
  static int get foo => 10;
}

class C extends B static implements StaticInterface2 {
  C(super.i);
  static void bar() {}
}

This also implies that there is no reason to assume that a type variable X has a specific static interface just because we know that X is a subtype of Y, and Y has that static interface.

void f<X extends Y, Y static extends StaticInterface1>() {
  Y.foo; // OK.
  X.foo; // Compile-time error, no such member.
}

@mateusfccp
Copy link
Contributor

I must say I am in love with this proposal.

It solves many "problems" at once (although it may introduce more? let's see how the discussion goes), and it's a very interesting concept.

I agree that it's not as straightforward to understand it, but once you understand, it makes a lot of sense.

I also understand the appeal in having the dynamic and static interfaces bundled together, as @Wdestroier suggests, so if we could come with a non-breaking and viable syntax for doing it (while still providing the base mechanism), I think it would be valuable.

@eernstg
Copy link
Member Author

eernstg commented Dec 11, 2024

@tatumizer wrote:

This can't be! Today String is Object is tautologically true, but under the proposed reforms, the predicate will acquire a new meaning: "String implements static interface of Object", which is not true: static interface of Object includes the method hash and a couple of others, but these methods are not inherited by String.

Today String is Object evaluates to true because String evaluates to an instance of type Type when it is used as an expression, and Type is a subtype of Object. There is nothing in this proposal that changes this behavior.

If you want to test whether the reified type object for String implements a given type you can test this using the same unchanged features: String is SomeStaticInterface. This wouldn't be very useful, because you can just use the current syntax to call String.aStaticMethod() if String has that static method, but it would be possible:

void main() {
  var ReifiedString = String; // Obtain the reified type object for the type `String`.
  if (ReifiedString is Object) { // Same operation as `String is Object`.
    // Here we know that `ReifiedString` is an `Object`, but
    // we knew that already, so we can't do anything extra.
    ReifiedString.toString(); // Can do, not exciting. ;-)
  }
}

Testing that ReifiedString is Object doesn't imply that the type String that ReifiedString reifies has any particular static members or constructors, including hash, hashAll etc.

It's the instance members of the tested type A which must be available as instance members of the reified type object for B when we declare that class B ... static implements A {...}

This means that if A has an int get foo member then the reified type object for B also has an int get foo, and this is guaranteed to be implemented as a forwarder to a static member B.foo.

The reason why I'm emphasizing that it's all about instance members is that we already have the entire machinery for instance members: Late binding, correct overriding, the works.

The static members and the constructors are just used to derive a corresponding instance member that forwards to them, and all the rest occurs in normal object land, with normal OO semantics.

@ghost
Copy link

ghost commented Dec 11, 2024

@eernstg : I understand each of these points, but they don't self-assemble in my head to result in a sense of understanding of the whole. The problem is terminological in nature. See my previous post about Kotlin.
(The very fact that you have to explain it reinforces the impression that the wording is not perfect. My guess is that we are missing an intermediate concept of "companion object" or similar).

@eernstg
Copy link
Member Author

eernstg commented Dec 11, 2024

About Kotlin's companion object, I already mentioned it here. The main point is that you cannot obtain the companion object based on a type argument, you have to denote the concrete class.

This reduces the expressive power considerably, because you can pass the type along as a type parameter and you can pass the companion object on as a regular value object, and then you can use the type as a type and the companion object as a way to access static members and constructors of that type. This is tedious because you have to pass the type and the object along your entire call chain whenever you want to use both.

You could also pass the type alone, but then you can't get hold of the companion object.

Or you could pass the companion object alone, but then you can't use the type (which is highly relevant, e.g., if you're calling constructors).

I spelled out how it is possible to write a companion object manually in Dart in this comment.

With this proposal, we can obtain the "companion object" (that is, the reified type object) for any given type (type variable or otherwise) by evaluating that type as an expression at any time we want, and we can test whether it satisfies a given interface such that we can call the static members / constructors of that type safely.

@eernstg
Copy link
Member Author

eernstg commented Dec 11, 2024

@tatumizer wrote:

My guess is that we are missing an intermediate concept of "companion object" or similar

The role of the companion object in Kotlin and Scala is played by the reified type object in this proposal. We could of course also introduce an indirection and just add a SomeInterface get companion getter to the reified type object's interface, but I don't think that's going to help in any way, it's just an extra step for no reason. Does that make more sense?

@ghost
Copy link

ghost commented Dec 11, 2024

@eernstg : that's where I have to disagree. It's an extra step for cognitive reason, which is a hell of a reason. :-)
But it's not only that! If you start reformulating the proposal in terms of companion object, many problems will resolve themselves automatically. Maybe a companion object is just a mental crutch, but I suspect it's more than that - or else we can get bogged down in hair-splitting about the meanings of words and their superpositions. Please give it a thought.

(Main property of a companion object, apart of its very existence, is that it has a clear type, which is visibly distinct from the type of the object itself, and it's a part of an (explicitly) different hierarchy. The difference is in explicitness).

@eernstg
Copy link
Member Author

eernstg commented Dec 11, 2024

If we just use the phrase 'companion object' rather than 'reified type object', would that suffice? I don't think there's any real difference between the members of a companion object in Scala and Kotlin, and the forwarding instance members in this proposal, so the reified type object is the companion object.

There is a difference in that this proposal only populates the companion object with forwarding instance members when the target class has a static implements A clause, and then only with forwarders that implement A.

We could also choose to populate every reified type object with all static members and all constructors. However, I suspect that it's a better trade-off to avoid generating so many forwarding methods because (1) they will consume resources (time and space) at least during compilation, and perhaps the unused ones won't be fully tree-shaken, and (2) they aren't recognized by the static type system if the reified type object doesn't have any useful denotable supertypes (so all invocations would then have to be performed dynamically).

The difference is in explicitness

Good point!

It is indeed an element of implicitness that this proposal works in terms of a 1:1 connection between each targeted static member / constructor and a forwarding instance member of the reified type object. We don't have that kind of relationship anywhere else in the language. The connection is created by specifying exactly how we will take a static member declaration and derive the corresponding instance member signature, and similarly for each constructor.

I don't know how this could be made more explicit without asking developers to write a lot of code that would be computed deterministically anyway (which is basically the definition of redundancy).

For the type, the static implements A clause does specify explicitly which type (in addition to Type) the reified type object will have.

@ghost
Copy link

ghost commented Dec 11, 2024

If we just use the phrase 'companion object' rather than 'reified type object', would that suffice?

I don't know if that will suffice, but it would certainly be a step in the right direction. When I see the expression "reified type object", my head refuses to operate further - though I can guess what this means, I'm not sure the guess is correct, and the word "reified" especially scares me off (like, WTF: how this reified type object differs from just type object?).

I'll respond to the rest of your comments here later, need time to process :-).

@Wdestroier
Copy link

In other words, it's crucial for this proposal that static implements has no effect on the subtypes of the declaration that has this clause

True, it's important to be an opt-in feature.
Example: the abstract keyword means the person opted in.

abstract class MyClass {
  // No effect on subtypes.
  static String name() => 'MyClass';

  // Has effect on subtypes.
  abstract MyClass();

  // Has effect on subtypes.
  abstract static MyClass fromMap(Map<String, dynamic> json) =>
    MyConcreteClass(property: json['property']);
}

@ghost
Copy link

ghost commented Dec 11, 2024

@eernstg:

Here's an arrangement I could understand:

  1. every Type object receives an additional getter. let's call it "companionObject". That is, we can say String.companionObject and get a real Object. By naming convention, the type of this object is (strawman) StringCompanion.
print(String.companionObject is StringCompanion); // true
print(String.companionObject.runtimeType == StringCompanion); // true

This is achieved by adding one method to the Type class:

class Type {
  Object companionObject; // just an Object
  // etc...
}
  1. If we want to invoke the method of parametrized type, we have to declare the type like this
class Foo<T static implements SomeKnownInterface> {
   bar() {
     T.companionObject.methodFromKnownInterface(...);
   }
}
  1. In the class declaration, we have to add "static implements" (no "extends" or "with")
class Foo static implements FromJson<Foo> {
   // no changes to the existing syntax
  static Foo fromJson(String str) { ... }
}

The companion object will only include the methods from the static implements interface(s)

  1. If the class doesn't declare static implements, then the associated companionObject will remain empty.

The difference of this design and the original one is that, given a type parameter T, you can write if (T.companionObject is SomeKnownInterface) , but you cannot write if (T is SomeKnownInterface), because the latter doesn't make sense - it's always false, as it is today. Other differences follow from here.
(I apologize in advance for any possible misunderstanding of the current proposal)

WDYT?

(Possible alternative for "companionObject": "staticInterfaceObject" or something that contains the word "static")
(Another alternative: "staticProxyObject", or just staticProxy, and the name of its type is like StringStaticProxy)

@eernstg
Copy link
Member Author

eernstg commented Dec 13, 2024

Here's an arrangement ...

Very good! I think I can add a couple of comments here and there to explain why this is just "the same thing with one extra step" compared to my proposal. You may prefer to have that extra step for whatever reason, but I don't think it contributes any extra affordances.

Also, I'll try to demystify the notion of a 'reified type object'.

every Type object receives an additional getter. let's call it "companionObject".

Right, that's exactly what I meant by 'We could of course also introduce an indirection' here.

print(String.companionObject is StringCompanion); // true
print(String.companionObject.runtimeType == StringCompanion); // true

You'd have to use (String).companionObject because companionObject would be an instance getter on the result of evaluating String as an expression, which is what you get by using (String) as the receiver. In contrast, String.companionObject is an error unless companionObject is a static member of the class String or String.companionObject is a constructor (and that wouldn't be useful, because the whole point here is that we want to abstract away from the concrete type such that we can, for example, call constructors or static members of different classes/mixins/etc from the same call site). So companionObject must be an instance member of the value of evaluating String as an expression. So we'd have this:

print((String).companionObject is StringCompanion); // true
print((String).companionObject.runtimeType == StringCompanion); // true

But the value of evaluating a type literal like String as an expression is exactly what I call 'the reified type object' for the type String. It's a perfectly normal object (currently it only has the five Object members, and it overrides operator ==, so it's quite boring. However, we can use it for comparisons like obj.runtimeType == String or MyTypeVariable == String).

The core idea in this proposal (meaning "the proposal in the first post of this issue") is that this object should (1) have instance members forwarding to the static members and constructors of the type which is being reified, and it should (2) have a type that allows us to call those static members and constructors (indirectly via forwarders) in a type safe manner, without knowing statically that it is exactly the static members and constructors of String that we're calling. (If we know that we're operating on the companion object or reified type object of exactly String then we could just as well have used the normal static member invocation mechanism that we have today: String.staticMethod()).

Returning to StringCompanion, this is a type whose interface has instance members corresponding to the static members (and perhaps constructors?) of String. So we can use (String).companionObject.staticMethod() to call a static method staticMethod (let's just say that String.staticMethod exists and is a static method). It will do the same thing as String.staticMethod().

To compare, this proposal will do exactly the same thing in the following way (assuming that String has the clause static implements Interface where Interface is the interface that corresponds to the set of static members and constructors that String wants to support via its reified type object):

print(String is Interface); // true
print((String).runtimeType == Interface); // false

The second query is false because the run-time type is not exactly Interface, it is a subtype of Interface, and it is a subtype of Type. This is again not a problem (in other words, you don't need this to be true) because you can just call the static member or constructor using the syntax we have today if you know exactly what the run-time type of the companion object / reified type object is: You can simply do String.staticMethod() if you know it's String.

If we want to invoke the method of parametrized type, we have to declare the type like this

Here's how to do it in this proposal:

class Foo<T static extends SomeKnownInterface> {
   bar() {
     (T).methodFromKnownInterface(...);
   }
}

The proposal has an extra shortcut: If we encounter T.methodFromKnownInterface(...) where T is a type variable (such that it is definitely a compile-time error today) then this simply means (T).methodFromKnownInterface(...). So you can omit the parentheses around type variables, which makes this kind of invocation syntactically similar to the current syntax C.methodFromKnownInterface(...) where C denotes a class/mixin/etc declaration that actually declares a static member named methodFromKnownInterface.

In the class declaration, we have to add "static implements" (no "extends" or "with")

class Foo static implements FromJson<Foo> {
   // no changes to the existing syntax
  static Foo fromJson(String str) { ... }
}

The companion object will only include the methods from the static implements interface(s)

These statements are true for this proposal as well.

If the class doesn't declare static implements, then the associated companionObject will remain empty.

The corresponding statement for this proposal is that if the class doesn't declare static implements then the reified type object does not have any instance members that are forwarders to the static members or constructors of the corresponding class declaration. (The reified type object in this proposal will still have toString etc., like any other object, but that might well be true for your companion objects as wall. So they are very similar.)

The difference of this design and the original one is that, given a type parameter T, you can write if (T.companionObject is SomeKnownInterface), but you cannot write if (T is SomeKnownInterface), because the latter doesn't make sense - it's always false, as it is today. Other differences follow from here.

In your proposal you can write (T).companionObject is SomeKnownInterface, and in this proposal the exact same thing is written as T is SomeKnownInterface, which does make sense and will evaluate to true if and only if the class/mixin/etc. declaration that corresponds to the given value of T does have the clause static implements SomeKnownInterface (or, silly corner case: if Type <: SomeKnownInterface, e.g., if SomeKnownInterface is dynamic or Object).

I hope this illustrates that the two approaches correspond to each other very precisely, and the only difference is that the companionObject getter is invoked in your proposal in a number of situations, and you simply skip that step in my proposal.

@ghost
Copy link

ghost commented Dec 13, 2024

@eernstg:
my bad, I don't know why I wrote String.method whenever I meant (String).method - I perfectly understand the differences.

I think I can pinpoint the single place where our views diverge, and it's this:
To explain why "reified type object" (RTO) for one class has different methods than that of another, the types of these "reified type objects" have to be different. E.g. RTO for String may have type _StringTypeObject - it's specific to String. (currently, the runtime type of RTO is _Type; after the change, _StringTypeObject becomes a subclass of _Type).
But I'm not sure such reinterpretation is possible (for the reasons of backward compatibility). Or maybe it is?
(You need to be able to explain to the user why RTO for Foo contains not all static methods of Foo, but only those that are part of declared static interfaces. So if the class name explicitly says FooStaticInterfaceProxy, that would help).
Otherwise, I agree that it's the same idea.

@eernstg
Copy link
Member Author

eernstg commented Dec 13, 2024

Great, I think we're converging!

To explain why "reified type object" (RTO) for one class has different methods than that of another, the types of these "reified type objects" have to be different.

That's generally not a problem.

In my proposal, the RTO for a given class has a type which is a subtype of Type (such that current code doesn't break) and also a subtype of the specified static superinterfaces (introduced by static implements).

Currently, we already have a situation where the result returned from the built-in runtimeType getter has type _Type rather than Type, and the same is true for evaluation of a type literal as an expression (hence String is Type). This is just standard OO subsumption, and there's nothing special about the fact that we don't (officially) know the precise type of String used as an expression.

This implies that it isn't a breaking change to make those evaluations yield a result whose type isn't _Type, but types of the form _Type & MetaMembers_Of_String, or something along those lines. As long as the given object has type Type we're happy.

Next step, the static type of an expression that evaluates a type literal that denotes a class/mixin/etc. declaration can include the meta-member mixin. In other words, the static type of String as an expression can be _Type & MetaMembers_Of_String.

This implies that we can safely assign this RTO to a variable with the static implements type:

abstract class StaticFooable { void foo(); }

class A static implements StaticFooable {
  static void foo() => print('A.foo');
}

void main() {
  StaticFooable companion = A;
  companion.foo(); // Prints 'A.foo'.
}

However, we can not use an overriding declaration of runtimeType to improve on the type: It is true that the runtimeType of an instance of A will return an RTO whose run-time type is a subtype of StaticFooable, but the actual receiver type may be some subclass of A which might not static implement StaticFooable.

This means that we don't know anything more specific than Type when we obtain a RTO from a type variable, which is the reason why I'm using a dynamic test (var reifiedX = X; if (reifiedX is StaticFooable) reifiedX.foo();).

It might seem nice and natural if we could make the static interface co-vary with the interface of the base object itself (such that we could use SomeMoreSpecificType get runtimeType; to override runtimeType in every class that static implements SomeMoreSpecificType), but I do not think it's possible: There is no reason to assume that a subclass would have a static interface which is a subtype of the static interface of its superinterfaces. I think the instance member interface and the static interface are simply independent of each other.

However, note that it would certainly be possible for a type argument to use subsumption in static interface types:

abstract class StaticFooable { void foo(); }
abstract class StaticFooBarable implements StaticFooable { void bar(); }

class B extends A static implements StaticFooBarable {
  static void foo() => print('B.foo');
  static void bar() => print('B.bar');
}

void baz<X static extends StaticFooable>() {
  X.foo(); // OK
  X.bar(); // Compile-time error, no such member.
  (X as dynamic). bar(); // Succeeds when `X` is `B`.
}

void main() {
  baz<B>(); // OK, `StaticFooBarable <: StaticFooable` implies that `B is StaticFooable`.
}

@ghost
Copy link

ghost commented Dec 13, 2024

It might seem nice and natural if we could make the static interface co-vary with the interface of the base object itself (such that we could use SomeMoreSpecificType get runtimeType; to override runtimeType in every class that static implements SomeMoreSpecificType), but I do not think it's possible

Why do you need this SomeMoreSpecificType get runtimeType ?
It would be perfectly fine to leave it as Type get runtimeType. Probably, I'm missing something here.
When we invoke (Foo).runtimeType, we can get ANY type that extends or implements Type. It can be, say FooInterfaceObject, which is declared internally as class FooInterfaceObject extends Type. At least, for the user it should look like this. Under the hood, it can be implemented differently - no one cares how exactly.

Compare with this: we can declare some method as returning num, but the runtime type of the returned value can print double.
Are Type and num fundamentally different in this respect? Why?

To be sure if we are on the same page, please answer this question.
Suppose we have 2 objects obj1 and obj2.
We invoke obj1.runtimeType and get "Foo".
We invoke obj2.runtimeType and get "Foo".
Can we conclude that objects obj1 and obj2 implement exactly the same set of methods?

The static type of expression Foo can remain Type - same as the static type of String, or int. Maybe I said something contrary to that earlier - if so, it was a mistake.

(After re-reading your response, I am not even sure we disagree on anything important, And I do acknowledge the benefits of your proposal, assuming we agree that Foo.runtimeType is not just _Type, but a more specific FooInterfaceObject (or something) that extends Type and has Foo in its name. Then the idea will be very easy to understand, and the description will fit in a single page - in fact, it will be even easier to explain than the "companion" object in Kotlin)

@ghost
Copy link

ghost commented Dec 14, 2024

After considering it more, the idea of including in RTO only the references to methods from the declared static interfaces might be unnecessary. Whenever the tree-shaker decides to preserve class C, it will most likely have to preserve its noname constructor, too (otherwise, no one can create an instance). Apart from constructors, the classes rarely have static methods, and those that are potentially used could be identified by name (e.g. if someone says (T as dynamic).foo(), the tree-shaker can preserve all methods called foo in all classes potentially passed as T into the class in question.
The names of static methods in most cases are unique. So the restriction won't buy much in terms of size but may damage the logical integrity of a concept. It would be much better to maintain the illusion that all static methods are included in RTO.


Another point: I think static interfaces don't make much sense. I can't imagine the situation where a class expects type parameter to implement some static methods without also expecting the instances to implement some specific regular methods.
E.g., some class can compute polynomials over a ring (represented by an interface R), where R must provide static methods for zero and unity, and the instances must define operations of addition and multiplication.
To make this possible, we have to add support for the static abstract methods, e.g.

abstract class Ring<T> {
  abstract T operator+(T other);
  abstract T operator*(T other);
  // ...etc
  abstract static T zero; // SIC!
  abstract static T unity;  
}
class Integer implements Ring<Integer> {
  final int _n;
  Integer(this._n); 
  // ...implementation of operators, 
  static const zero = Integer(0);
  static const unity = Integer(1); 
}

@eernstg
Copy link
Member Author

eernstg commented Mar 10, 2025

Reordered response:

Spin-off operator

What can you DO with the type variables?

Do you mean 'what can you do with the reified type objects'? The type variables do not have any new affordances, but if you evaluate a type variable as an expression then you'll obtain a reified type object. As always, they have type Type (that is, some internal subtype thereof, but we shouldn't have to know that), and they are basically able to determine whether some other Type is a reification of the same type, as in t1 == t2, or not. That's all you can do with a Type today.

However, with this proposal you'll get an instance whose run-time type is a subtype of Type (as always), but also a subtype of S for each static implements S clause or static extends S clause in the declaration of the type. In both cases it's a compile-time error to have those clauses unless the target class allows for S to be fully implemented.

abstract class Fooable {
  void foo();
}

class A1 static implements Fooable { // OK.
  static void foo() {}
}

class A2 static implements Fooable { // Error!
  static void foo(int i) {} // Cannot be used to implement `Fooable.foo`.
}

E.g., can you say:

var t = Type.reify(String, Deserializable<String>);
var str=<t>['a'];

No, a reified type object can not be used as a type, they are not the same kind of thing.

Probably not. Maybe t is a regular object that implements a regular Deserializable<String> interface, and the only thing you can do with it is to call the methods defined in Deserializable<String> (that is, readFrom)?

Exactly (assuming that there is an extension somewhere that adds this capability to String).

This object has no relation to String RTO then.

Not true. We still have properties like assert(t == String && t != int). The RTO plays a role which is similar to the role played by a metaclass in systems where that concept has been around for at least 40 years (e.g., CLOS). This proposal is much less powerful than CLOS metaclasses, but we don't want to pay the price for the generality in Dart. We just want some simple things like deserialize or callWithTypeArguments.

It has some synthetic type. Not sure. But if this is the case,

Yes, we're implicitly inducing a class which is a subtype of Type and a subtype of Deserializable<String>, which implies that it must have a synthetic type. But we only need to know that this unknown type is a subtype of Type and a subtype of Deserializable<String> (and we might very well ignore the former if we're just focusing on deserialization).

then who stops us from doing the same with regular extensions? E.g. (making things up),

Extension methods are statically resolved. That is, it is known at compile time exactly which portion of code is being executed when we call an extension method.

The whole point with 'more capable type objects' is that they allow us to invoke static members and constructors without the need to know statically exactly which portion of code you will be running.

In other words "what stops us" is the simple fact that we're aiming to do the exact opposite of what extensions do.

But the following is very far from being regular extensions:

extension on List<int> implements Summable<int> {
   int sum() => this.fold(0, (accum, item) => accum + item);
}

and then somehow build the instance of Summable in runtime, e.g. Object.reify([1,2,3], Summable<int>);. Now you can pass this object around as an instance of Summable - even pass to the places that don't see your extension.

Trying to understand what that would mean, it looks like we're declaring an extension that adds an extension instance method sum to List<int>. So we can call it on any receiver that has static type List<int> (or a subtype, e.g., LinkedList<Never>). Nothing new here, no need for implements Summable<int>, so that'll be the next step.

Next, I'd assume that Summable is declared approximately like this:

abstract class Summable<T> {
  T sum();
}

So you're saying, IIUC, that we could introduce a feature whereby the compiler/analyzer would implicitly induce a class declaration which is a subtype of Summable<int> and which will implement sum by calling the sum method of this extension.

We'll need a receiver (the object denoted by this in the extension body), so we'd have something like the following:

// Implicitly induced, based on the extension shown above.
class SummableInt_for_ListInt {
  List<int> list;
  SummableInt_for_ListInt(this.list);
  int sum() => list.sum();
}

But this means that Object.reify([1,2,3], Summable<int>) would be equivalent to SummableInt_for_ListInt([1, 2, 3]). I have no doubt this idea could be spelled out in detail, and that it might be useful.

However, I don't think there is much of a relationship with this issue, or with the static implements clauses in extensions that I've introduced as part of the proposed feature. In particular:

it builds a new object, hidinig the original object so that it can't be retrieved by casting an interface back to the original type. (Maybe it can be retrieved by other means - not sure).

When we're reifying a type there is no original object (a type is not an object). We're just using a mechanism that provides support for obtaining an object (a reified type object) from a given type. We've had such a mechanism since the very beginning of the language Dart.

What I'm proposing here is that we should generalize this mechanism such that (1) the reified object which is obtained directly from the type itself (that is, the one which is based on the declaration of that type) can provide access to static members and/or constructors of that declaration, and (2) extensions can support additional ways to reify the type to RTOs that implement/extend specific types.

The former is the basic mechanism which can be used whenever you "own" all the types that you wish to use in a particular way. The latter allows us to add support for whichever feature you want to specific types that you can't edit. (For example, we can make List<T> support deserialization for any T even though we can't edit the class List).

var s = String ^^ Deserializable; // instead of Type.reify(...)
var l = [1, 2, 3] ^^ Summable; // instead of Object.reify(...).

You'd need to specify a very special semantics for ^^ in order to allow String ^^ Deserializable to do things that are similar to Type.reify<String, Deserializable<String>>(). For example, if ^^ is an operator then String will already have been reified before we can get started doing the job that Type.reify does, which is then impossible (because there's no way to obtain the type String from the RTO of String which is based on the declaration of String). So it's not just a new operator.

Next, with [1, 2, 3] ^^ Summable (not Summable<int>?), we definitely want to evaluate [1, 2, 3] as an object. So that's again a very different ^^ than the one we would use with String ^^ Deserializable.

I think I'll stick to the syntax I've proposed.

@eernstg
Copy link
Member Author

eernstg commented Mar 10, 2025

consider the following (made-up) declaration:

extension on String static implements Deserializable {
static String readFrom(Reader r) => r.readString();
}

What's wrong with that? Why do you need "static extends" and yet another class from which it extends?

I'll assume it means something like the following:

// I'm assuming the following declaration of `Deserializable`:
abstract class Deserializable<Self> {
  Self readFrom(Reader r);
}

extension on String {
  static implements Deserializable<String>;
  static String readFrom(Reader r) => r.readString();
}

In terms of the proposal in this issue, this is a compile-time error: String is not able to static implements Deserializable<String> because String does not declare a static member (or constructor) whose name is readFrom (respectively String.readFrom).

The static member readFrom in the extension cannot be called because it is never called inside the body of the extension, and it is not in scope anywhere outside the body of the extension. So it's dead code.

However, if we assume that we've added static extensions to the language then the declaration of readFrom in the extension would actually make it possible to call String.readFrom(...), as if we had added that declaration to String itself.

With that, it should be a reasonable small generalization of the mechanism that implicitly induces a class that it can create an implementation that forwards invocations of readFrom to that extension-based static method.

So if we assume that static extensions are added to the language then it's reasonable to say that there is nothing wrong.

If we're only assuming that the feature proposed in this issue is added then we need static extends and the extra class, because we need the RTO to inherit the implementation of readFrom.

@eernstg
Copy link
Member Author

eernstg commented Mar 10, 2025

Can we define what Type.reify means exactly?
Is the following correct?

Type.reify(String, Deserializable<String>) returns the instance of Type object, which is either created dynamically, or - if the reified instance was created earlier - gets retrieved and returned. The returned type statically implements Deserializable<String>.

Type.reify(String, Deserializable<String>) will return an RTO for String whose run-time type is a subtype of Deserializable<String> (and also a subtype of Type, of course), or null.

In the search for a way to create a suitable RTO, the declaration of String is considered first. If String declares that it static implements Deserializable<String> (or static extends it) then we create the RTO of the declaration and return that.

Otherwise, we consider all extensions in scope whose on-type can be matched by String (so it could be on String, but it could also be on Pattern), where the extension contains one or more static implements or static extends members.

Among those static implements/extends members, select the ones whose operand type is a subtype of the requested type (so we keep static implements Deserializable<String> but drop static implements Deserializable<Pattern> and static implements SomeOtherType<What, Ever>). During this step, we don't care whether it's static implements or static extends, we just know that the implicitly induced class is a subtype of Deserializable<String>.

If this yields exactly one static implements/extends clause then we create an instance of the implicitly induced class associated with that member, and return that as the result of the invocation of Type.reify.

If it doesn't yield any, or if it yields several such static implements/extends clauses then the invocation of Type.reify fails. When there are no results I'd prefer to return null. When there are multiple results I'd prefer to raise an exception at run time.

However, note that when the 2nd type argument passed to Type.reify is a compile-time constant type then it is always possible to check at compile time that there is at most one static implements/extends extension member whose operand type is a subtype of the requested type. We can decide this because both the operand type of each static implements/extends member and the requested type are compile-time constant types.

However, we can also allow the 2nd type argument to Type.reify to be a type variable or a type that contains type variables. This is more dangerous because it allows the invocation to fail with an "ambiguous RTO" error. This implies (to me) that using a type which is not a compile-time constant as the second type argument to Type.reify is a bit like dynamic invocations: We can allow it, but we'll surely warn against it (using some kind of strict-type-reify option).

Like every Type object in dart, it internally contains the reference to a base class and a number of references to all implemented interfaces.

No, Type objects actually don't have to have a representation of any information about subtype relationships (for example, with Type t1, t2;, we can't do t1 < t2). They may or may not have a representation of the underlying type. For instance, with non-generic classes, the corresponding RTO for a given class C might just contain an int value which is a class ID. This would suffice to enable an implementation of operator ==. So we can't assume that a Type as such contains much.

But when we call Type.reify(String, Deserializable<String>), the system does... what?

It finds the relevant extension and the relevant static extends _DeserializableForString; member. Then it creates an instance of the implicitly induced class (that actually is a subtype of Deserializable<String>), because that's how this class was generated.

creates a new RTO which is a copy of the existing String RTO with the added reference to Deserializable

No, that wouldn't work. We can't make a class implement-or-extend Deserializable<String> just by saying so. In particular, the actual class that we're creating an instance of must have an implementation of String readFrom(Reader r). That kind of thing doesn't pop into existence just because we need it. However, we have written an implementation that works in _DeserializableForString, and the use of static extends ensures that the implicitly induced class inherits that implementation.

what is the relationship between a standard String RTO and this newly created "String++" RTO?

They both satisfy is Type. The standard String RTO is basically just that. But the RTO that we got from the extension will also satisfy is Deserializable<String>. They are == to each other (because there is no need to disrupt the behavior of == for RTOs).

we discussed the idea of injecting references to interfaces into RTOs in the context of union types, and you were against the idea. Maybe you came up with the concept of Type.reify later, and changed your opinion? Or what?

I'm not 100% sure I can recall what that was about.

However, I think they are completely different things:

  • For union types I don't remember having said anything at all about RTOs, but I remember having said that it's a serious problem with union types if they make type inference hugely less performant. So I'd prefer to keep the subtype relationships between union types and to/from union types and other types simple. In particular, I think they should be declared explicitly.
  • For Type.reify, it's specifically about being able to request an RTO for a given type which is not provided by the corresponding class declaration itself, it's provided "post-hoc" by an extension. I don't think this has anything to do with the injection of subtype relationships, it's more like this: "We have a type X and we want an RTO with type MyType, and then we can look at the declaration of the value of X (which always wins if it does have the requested type) as well as all matching extensions (which are given 2nd priority, and must not be ambiguous). Each of those RTOs is a subtype of Type and a subtype of whatever it is declared to be, but otherwise they have no relationship to each other."

@eernstg
Copy link
Member Author

eernstg commented Mar 10, 2025

OK, @tatumizer, that was a long rant. Hope it helps! ;-)

@ghost
Copy link

ghost commented Mar 10, 2025

Thanks! It a progress :-)
At least, my guess about the behavior of Type.reify was close to the intended meaning.

Let's stick to your syntax.

  • I thought, saying Type.reify(String, Deserialiable) is enough because the type parameter <String> can be inferred from the only extension which adds a static implementation of Deserializable<String> to String. It wouldn't be an error to write the whole thing explicitly Type.reify(String, Deserialiable<String>) - it's just redundant (having another extension adding Deserializable<Pattern> for a String is a very remote possibility - at worst, compiler can require disambiguation).

  • I was pretty sure static extensions is a done deal b/c I saw your recent update on the proposal of static extensions. Without it, the matters become quite convoluted, as you can see from my questions.

  • Why did I think the object returned by Type.reify(String, Deserialiable) has nothing to do with String? Because:

  1. The static type of the value returned by Type.reify has to be Deserializable<String> - a normal interface. There's no way to express the static type in the form "Type that extends Type<String> and also implements (non-statically!) Deserializable<String>". (After re-reading this sentense, I am not sure I can understand it myself). Dart doesn't even support the Type<String> notation as other languages do. The lack of such notation makes it difficult even to discuss the subject. Can we introduce the Type<String> - at least informally, within this discussion- so that we won't accidentally confuse the type String with the variables of type String while talking about the subject in "prose"?

  2. Consider var t = Type.reify(String, Deserialiable<String>); How can we find out that t has anything to do with Type<String>? We cannot test if (t is String) - it won't work. We cannot say if (t is Type<String>) - the notation is not supported. We cannot say var t1 = Type.reify(t, AnotherInterface) - even if we have a static extension that adds static implements AnotherInterface to String. If we pass the t to some function expecting a parameter of type Deserializable, it won't be able to ever guess what the origin of this value was. So what's the point?

  • Your guess was correct; it's just a wrapper around the List variable, but I assumed it would be a private field. Indeed, it works via delegation:
class SummableInt_for_ListInt {
  List<int> _list;
  SummableInt_for_ListInt(List<int> list): _list=list;
  int sum() => _list.sum();
}

EDIT: Upon reflection, this doesn't make much sense. To be revisited later.


Let's forget about union types for now. There is a connection to union types, but it's a connection of a different kind - let's talk about it later.

No, that wouldn't work. We can't make a class implement-or-extend Deserializable just by saying so. In particular, the actual class that we're creating an instance of must have an implementation of String readFrom(Reader r)

I see. My mental model was wrong: I thought type object directly contains references to base class and interfaces, but (I'm guessing) it contains a class id, and there's a table somewhere that maps class id of every class to class ids of the base class and class ids of the interfaces. Logically, it's the same, but with a twist. And I understand now why you have to generate a class that extends Type<String>. (BTW, did you notice how the notion of Type<String> simplifies the discussion? Would it be a breaking change if dart adds support for it?)


I've been working on the etude of my own. Write a function f(Type t){...} that, being called with a parameter List<List<List<int>>>, prints:

List<List<List>>
List<List>
List
int

It is supposed to work with the Russian doll of nested Lists, recursively calling itself, until it reaches something that is not a List.

I have no idea how to write it.
I think we need recursive functions like that to be able to implement deserialization generically (via Type.reify and friends).


Belatedly responding to your earlier comment:

Well, if the point you're making is "code duplication is not a problem when the code is generated" then I think developers working on large projects would tend to disagree: Code size matters, too.

I don't think it matters much. The implementation of deserialization for any LIst<List<List<A>>> is basically a 3-liner for each of the nested lists (there are 3 of them), and if a project contains 100 classes, the total amount of code generated for them will dwarf this extra expense. But, if you forget to annotate A with @Deserializable, the compiler will show a static error on the generated line A.fromJson, which is easy to understand. In contrast, with the recursive runtime deserializer, the error will show up only in runtime, with an undecipherable message. Not a showstopper, of course (the concept may have aesthetic value that transcends the physical world, but it will require more work 😄 1), but the same argument appplies to any static vs dynamic tradeoff. Anyway, it's not very important in the discussion of a concept itself.

Footnotes

  1. Recommended read on the subject. Also, the youtube channel

@eernstg
Copy link
Member Author

eernstg commented Mar 11, 2025

I thought, saying Type.reify(String, Deserialiable) is enough

In principle, we can specify any arbitrary semantics we want for every term which is syntactically recognizable as such. For example, we can define that A.name(_, _) is not an invocation of a static member of A named name, it's a special form that has a completely different semantics. I already rely on this technique for Type.reify and the compiler must generate specialized code for this at each call site. This kind of technique is a major case in the bucket of things that we call "magic".

However, I'd like to keep the "magic" at the lowest possible level. One reason for this is that I want to avoid introducing large amounts of special casing in the semantics of the language: That's harder to understand and internalize as "natural", and it gives rise to more special casing code in the analyzer and compilers.

In particular, it's crucial that Type.reify<String, Deserializable<String>>() receives actual type arguments, not regular value arguments. For example, this means that we don't need any kind of "magic" in order to specify that the return type of Type.reify is the second type argument. If it is called as Type.reify(String, Deserializable) then we would need to specify something like "The second value argument has type Type, and the return type is the type which was reified to obtain that object". We can't do that, already the following shows that it won't ever work:

class MyType implements Type {} // Sure, we can do that!

In other words, there is no guarantee that every object of type Type is a reified type object. Passing a type as a value argument is an irreversible loss of information and capabilities if you need to use it as a type, you just can't go back from the RTO to the type.

So we definitely don't want to call Type.reify like Type.reify(String, Deserializable), it must receive the types as type arguments, as in Type.reify<String, Deserializable<String>>().

Next, we can't just specify Deserializable and omit String because there's no reason to assume that String will always and only support an RTO of type Deserializable<String>. The declaration of Deserializable doesn't require the readFrom method to return an instance of the type which is reified by the RTO. If we want String to support Deserializable<String> as well as Deserializable<List<Widget>> then we can go ahead and write the code which is needed for doing that. So we definitely need to support specifying the requested type in full detail.

I was pretty sure static extensions is a done deal

I think that's true, but we don't promise anything before we're basically ready to release a feature. It's is always possible that something causes the evolution of the language to take a turn in an unexpected direction.

Why did I think the object returned by Type.reify(String, Deserialiable) has nothing to do with String?

It's an RTO for String and it has the property that it is == String. I don't think that's "nothing".

The static type of the value returned by Type.reify has to be Deserializable<String>.

My proposal says that it is S? where S is the 2nd type argument of the invocation of Type.reify. It is not easy to change this to a type which is a subtype of Type and also a subtype of S, but if you do is Type on it then you will get true.

The static type of an evaluation of a type literal as an expression can be more precise and include both supertypes. This means that the implicitly induced class corresponding to a static implements/extends A clause on a class declaration can be the inferred type of a variable (var myRTO = SomeType;). This is needed in order to avoid breakage (with this declaration of myRTO, existing code may rely on the fact that myRTO can be assigned to a variable of type Type). But myRTO also needs to have a type which is a subtype of A, in order to enable type checked invocations of members of A (without doing if (myRTO is A) ...).

There's no way to express the static type in the form "Type that extends Type<String> and also implements (non-statically!) Deserializable<String>".

// The following class is generated by the compiler/analyzer.
class MyReifiedType extends Type implements Deserializable<String> {
  String readFrom(Reader r) {...}
}

What was the problem?

About Type<T>: We don't have this feature, but please vote for #2090 if you wish to help moving it forward.

For now, we can just continue to talk about String as a type vs. the RTO which is the result of evaluating the type literal String as an expression (and with support for extension members of the form static implements/extends: the RTOs).

Consider var t = Type.reify(String, Deserialiable<String>); How can we find out that t has anything to do with Type<String>? We cannot test if (t is String)

You can test String == t which is true if and only if t is an RTO for String. (Testing t == String is not safe, because t, if you don't know where it came from, could be an instance of a class whose operator == just returns true no matter what).

We cannot say var t1 = Type.reify(t, AnotherInterface)

Rigth, you cannot say that because Type.reify accepts two type arguments and no value arguments. It's Type.reify<TheTypeToReify, TheDesiredInterface>(), and it's simply a compile-time error to pass a Type where a type is required, as in Type.reify<t, ...>(). Same thing as <23>[], which is also an error because 23 denotes an object, not a type.

So what's the point?

Well, I'm thinking "What's the problem?" ;-)

I've been working on the etude of my own. Write a function f(Type t){...} that, being called with a parameter List<List<List<int>>>, prints:

List<List<List>>
List<List>
List
int

If f is invoked with an RTO for the type List<List<List<int>>> that implements CallWithTypeParameters then you can do this:

void f(Type rto) {
  print(rto);
  if (rto is! CallWithTypeParameters) {
    // Could be a generic type without support, but we just assume we're done.
    return;
  }
  if (rto.numberOfTypeParameters != 1) throw IllegalArgumentException("...");
  final nextRto = rto.callWithTypeParameter(
    1,
    <X>() => Type.reify<X, CallWithTypeParameters>())
  );
  if (nextRto == null) throw IllegalArgumentException("...");
  f(nextRto);
}

You can't do it in current Dart, but if you enhance the capabilities of RTOs as in this proposal then it is possible.

About static type checking and deserializing a List<List<List<A>>>:

But, if you forget to annotate A with @Deserializable, the compiler will show a static error on the generated line A.fromJson

That's a good point.

With the feature which is proposed here, we're talking about including static implements Deserializable<A> in the declaration of A, not about remembering to add @deserializable.

(By the way, how would you manage to add @deserializable to List? )

In the situation where we can equip every class that participates in a particular activity with the required behaviors (e.g., enabling deserialization with every class which is part of a deserialization step), then there is no need to have less complete type checking: Each class C<X extends B1, Y extends B2> that participates will then have static implements Deserializable<C<X, Y>>, and it is a compile-time error if the compiler/analyzer isn't able to generate the associated RTO class. Next, every type parameter X which is used for deserialization should have a bound: X static extends Deserializable<X>. This ensures that X.readFrom(myReader) is completely type safe, just like the current treatment of type parameters (that is, things don't get unsafe just because we are working on types that are not known at compile time).

So if we're in a position where we can add support for Deserializable<...> to every class that participates in deserialization then there is no issue, it's just type safe from the outset. This might actually be possible with special exceptional cases like Deserializable because it is expected to be very widely used.

However, in the case where we wish to include support for some classes that we can't edit (e.g., that's quite likely to be true for List, Map, String, and int, especially if we don't have exactly one canonical way to do deserialization), we need to allow working on types that combine those widely used but "post-hoc" supported classes with the classes that have deserialization as a "built-in" feature (that is, the ones that have static implements Deserializable<...>).

In this case we can't require X static extends Deserializable<X> for all the relevant type variables, and it isn't going to be easy to specify a kind of bound that allows for the mixture I mentioned, and nothing more.

I do think, though, that the added expressive power of the type-abstracting approach based on 'more capable type objects' (that is, avoiding a lot of code duplication, and simply handling more cases) will be a very meaningful choice in many cases where we must handle some classes "on the side" because we can't directly edit them. If we can just opt in all the classes that are participating then there is no trade-off, as I mentioned, it's just as type safe as the rest of Dart.

@ghost
Copy link

ghost commented Mar 11, 2025

What was the problem?

You can declare any problem a non-problem by providing an (imaginary) non-solution. :-)

How does the RTO in question becomes equal to String (== Type<String>) if it's a proper subtype of Type<String>? To determine subtype relationship, we need to use is, but how? Note that the value t returned by Type.reify should (according to your design) satishy both t is Type<String> and t is Deserializable<String> (not t is Type<Deserializable<String>>. So, what is the static type of t then? You didn't say. The example of the class you provided is not an answer:

class MyReifiedType extends Type implements Deserializable<String> {
  String readFrom(Reader r) {...}
}

Where does it say it's a subtype of Type<String>? Maybe it's a subtype of some Type<Foo>?. And what about the name MyReifiedType? I was under the illusion that the class gets generated by Type.reify, so it has to have an autogenerated name. What name?
E.g. String$ExtensionFor$Deserializable$String$UsingExtension$E. Do you mean this is a static type of t? A user-visible one?
Which cannot be exported or imported anywhere? So is it a private name?
(I'm just scratching the surface :-)).

EDIT: rather than creating a new class like MyReifiedType, you could just inject implements Desrerializable<String> (together with the method implementing it) into the existing String RTO.
This is a routine device in javascript, python and other languages, I simply forgot its name. ChatGPT helped me to refresh my memory: it's called "monkey patching".


By the way, how would you manage to add @deserializable to List?

I don't have to. For a code generator, List<E> is assumed to be deserializable by definition., assuming E is also deserializable. Same goes for a Map<String, E>. But user-defined classes like A, B, Foo, Person have to be marked with @deserializable explicitly.
(Writing a basic generator for serializer/deserializer is an easy excercise)


Here's my version of the etude:

f(Type T) {
  print(T);
  if (T case Type<List<var X>>) f(X);
}
// instance version
g<T>(T value) {
  print('type=$T value=$value');
  if (T case Type<List<var X>>) g(value.first as X);
}
main() {
  f(List<List<List<int>>>);
  g([[[1]]]);
}

Here's the connection to union types: we discussed the serialization of a recursive Serializable type expressed as a union type. It would be highly surprising if for deserialization we would have to use a totally different mechnism. Note that with a naive generator, these two are perfectly symmetric. Same symmetry would be expected here. Maybe this observation can give us a clue?


This begs the question: what is the minimal set of features that allows us to implement seralization/deserialization using a generator, with no redundant code?
My claim is that we need just 2 features:

  • parameterized "type" variables Type<T> (like Type<String>, Type<List<List<int>>>
  • existential open - that is, the ability to write if (T case Type<List<var X>>) ..;

All other features are unnecessary for that purpose - specifically:

  • union types (they are good for other purposes, but not this one)
  • extensions of any kind
  • interfaces of any kind
  • Type.reify and friends

Agree? :-)

@eernstg
Copy link
Member Author

eernstg commented Mar 14, 2025

How does the RTO in question becomes equal to String (== Type<String>) if it's a proper subtype of Type<String>?

Type doesn't take any type arguments so Type<String> doesn't make sense. I mentioned that I'd be very happy to see Type modified such that it takes a type argument T such that every reified type rto obtained by reifying the type T would satisfy rto is Type<T>. However, even if we get that then there's still no problem.

class A {
  final int i;
  A(this.i);
  bool operator ==(Object other) => other is A && i == other.i;
  int get hashCode => i.hashCode;
}

class B extends A {
  B(super.i);
}

void main() {
  print(B(15) == A(15)); // 'true', no problem.
}

A reified type object as an object whose run-time type is a subtype of Type. There is no problem whatsoever in defining an operator == such that some objects can be equal in spite of the fact that they don't have precisely the same run-time type.

In this example, the instance variable i corresponds to the type that the Type is a reification of. We can't directly access a type in Dart as a run time entity (it's not an object, and everything that we can obtain as the result of an expression evaluation must be an object), but reified types are created by the runtime and hence they can manage entities that are outside the set of entities which can be the value of an instance variable in regular Dart code.

To determine subtype relationship, we need to use is, but how?

We have reified type objects that are reifying various types, but they don't have to have a type that has any direct relationship to the type that they reify.

For example, it's perfectly possible for an implementation to reify all types as instances of a specific class (say, _Type in 'dart:core'). This means that the type of the RTO for num and for int and for String and for List<Widget> is _Type in all cases. In particular, the run-time type of the RTO for String is a subtype of the run-time type for the RTO of int and vice versa, because they are all of type _Type. They just happen to have different values for the representation of the type that they reify.

With this feature, they would not be instances of _Type, they would be instances of some subtypes of _Type. This means that, for example, there is no reason to assume that the run-time type of the RTO for Object has any relation to the run-time type for the RTO for String, in the case where the RTO for Object has some static interface that String doesn't have.

Note that the value t returned by Type.reify should (according to your design) satishy both t is Type<String> and t is Deserializable<String> (not t is Type<Deserializable<String>>.

No, the RTO t returned by Type.reify<String, Deserializable<String>>() satisfies t is Type and t is Deserializable<String>. There is nothing new or magic in having more than one superinterface.

So, what is the static type of t then?

It is Deserializable<String>?. It would be possible to give guaranteed success an absolute priority and then remove expressive power from Type.reify in various ways until it can be Deserializable<String>, but I prefer to at least keep it in mind that we can handle a large universe of situations gracefully if we allow for the search for a suitable RTO to use the information which is actually available at run time, in return for possibly not always succeeding.

It is still very important, of course, to allow developers who wish to stick to a more restricted usage to determine at compile time that the lookup is guaranteed to succeed.

The example of the class you provided is not an answer: ..

class MyReifiedType extends Type implements Deserializable<String> {
  String readFrom(Reader r) {...}
}

Where does it say it's a subtype of Type<String>? Maybe it's a subtype of some Type<Foo>?. And what about the name MyReifiedType? I was under the illusion that the class gets generated by Type.reify, so it has to have an autogenerated name. What name?

It isn't a subtype of Type<String> because that's not a thing (yet)! It's a subtype of Type. Anyway, I can't see how it would create any problems if we had Type<T> and it would be a subtype of Type<String> as well as a subtype of Deserializable<String>. There's no scarcity of classes that have multiple superinterfaces with type arguments.

The name MyReifiedType would be a generated name, so you can't denote it. But Type.reify<R, I> has a return type of I?, and that's all you know when it is the return value of an invocation of Type.reify. If you intend to use it as an instance of Type you can test if (t case Type rto) ... rto ....

It would be nice to find a way to preserve the Type part of the type of the result returned by Type.reify, but we'd need something like an intersection type to do that. But I don't think this is a show-stopper, it's more like a wart.

Another idea which has been mentioned would be that the RTO for a type T should be denoted by T.static or something like that. However, that doesn't combine well with the need to cover some types post-hoc (that is, to give an existing type a capability even though you can't edit the declaration of that type). Also, it certainly wouldn't work as X.static where X is a type parameter.

you could just inject implements Desrerializable<String> (together with the method implementing it) into the existing String RTO.

That wouldn't work when the capability is added by an extension: Two extensions could add support for two different interfaces with a conflicting member name (same name, incompatible signatures, or, even worse, compatible signatures and different purpose). That's the reason why I'm proposing that two extensions can contribute to the capabilities of the RTOs of the same type, but they will do it by means of completely separate and independent RTOs (based on completely separate and independent compiler-generated classes).

Here's my version of the etude:

You're assuming that we have an existential open operation. One of the really powerful things about this proposal is that we can create support for a reasonable replacement for an existential open operation (callWithTypeParameters).

The important thing to be aware of here is that the existential open mechanism would be a substantially bigger effort. Also, an existential open mechanism won't help us if the goal is to be able to invoke some static members and/or constructors without being forced to write monomorphic code (where all types are resolved at compile time): An existential open mechanism can only do something if you're working with a generic type, but one (very basic) scenario for the invocation of static members involves non-generic classes. There's no way you can use an existential open operation to help you abstract over an invocation of two static members MyClass.foo() vs. MyOtherClass.foo() if it is expressed as X.foo() where X is a type variable whose value can be MyClass or MyOtherClass.

You might think we could obtain the RTO for X, and that would then have type Type<MyClass> or Type<MyOtherClass>, but all you can do with the existential open is to gain access to that type argument — which is again just a type variable whose value may be MyClass or MyOtherClass.

Yes, I know, this is only type safe if we have X static extends I where I has a member with signature T foo() for some T, but that's part of the deal: If you want safe and abstract invocation of static members then we need to provide some constraints (like static extents I) such that we can check at compile time that this foo() method is actually guaranteed to exist for that RTO.

@ghost
Copy link

ghost commented Mar 14, 2025

(Continuing a correspondence chess game. Here's my next move)

Before going forward, let's check the position on the board:

var t = Type.reify(String, Deserializable<String>);
// static type of t is "Deserializable<String>?", period! Not anything "more specific".
if (t != null) {
   t == String; // true
   t is Type; // true
   t is Deserializable; // true
   t is Deserializable<String>; // true
}

Here's a copy of your earlier program

void f(Type rto) {
  print(rto);
  if (rto is! CallWithTypeParameters) {
    return;
  }
  if (rto.numberOfTypeParameters != 1) throw IllegalArgumentException("...");
  final nextRto = rto.callWithTypeParameter(
    1,
    <X>() => Type.reify<X, CallWithTypeParameters>())
  );
  if (nextRto == null) throw IllegalArgumentException("...");
  f(nextRto);
}

This program reminds me of Zones. (No need to say more :-)).
Also, I don't understand where X comes from. Probably, some extra move was not recorded.
Can you please explain what's going on here? :-)
How does it help solve the original problem of printing nested type of the form List<List<List<NotAList>>>>?.
How can the function determine whether rto has a type argument of the form List<Something>?)


Here's another version that works without parametrized Types or existential open:

void f(Type rto) {
  print(rto);
  if (Type.isSubtypeOf(rto, List)) {
    f(Type.typeArguments(rto)[0]);
  }
}

Another (better?) syntax for the same idea would be

void f(Type rto) {
  print(rto);
  if (Type(rto).isSubtypeOf(List)) {
    f(Type(rto).typeArguments[0]);
  }
}

This resembles the syntax we use for disambiguation while working with named extensions like E(obj).myextensionMethod().
(The names isSubtypeOf and typeArguments are consistent with Mirrors API.)
This is another "minimal set" of functions that enables writing "generic deserialization without redundancy".
To enable calling fromJson for a user-defined class A (marked as @JsonDeserialiable), the generator can associate a function with each class using Expando or something: deserializers[A]=A.fromJson;, so for this particular feature (deserialization) you don't need interfaces.


As soon as we introduce Type.isSubtypeOf and Type.typeArguments, we obtain a syntactic template where Type.reify fits rather nicely. You don't have to override == b/c isSubtypeOf is enough. Type.reify can be interpreted as a softer version of "monkey-patching" where the original rto is left intact. But I'm not sure whether this feature can be applied to type like List<int>, List<List<int> - in other words, whether it's possible to implement something like

extension on List<int> static implements Bar {
  static void helloFromBar(); // the method from Bar interface
}
Bar bar = Type.reify(List<int>, Bar);
bar.helloFromBar();
if (Type.isSubtypeOf(bar, List<int>) {
  Type origType = Type.cast(bar, List<int>);
}

I'm afraid the feature could rather confusing and not very useful.

@eernstg
Copy link
Member Author

eernstg commented Mar 17, 2025

@tatumizer wrote:

Not anything "more specific".

Right, the return type of Type.reify<A, B>() is B?. But the static type of a type literal evaluated as an expression will have to be a subtype of Type (to avoid breakage) as well as a subtype of whatever is promised via static implements/extends clauses or type variable bounds. This can be done by letting the static type of the type literal expression be the compiler-generated class that has all those types as superinterfaces.

void main() {
  var t = Type.reify(String, Deserializable<String>);
  if (t != null) {
    // All succeeding.
    assert(t == String);
    assert(t is Type);
    assert(t is Deserializable); // Which means `Deserializable<dynamic>`.
    assert(t is Deserializable<String>);
  }
}

Here's a copy of your earlier program ...

Here's a version of the program that actually runs. It emulates the behavior of Type.reify for the special cases that are being used.

I had to fix a couple of bugs, but the overall structure works just fine: First, promotion of rto won't happen when the type of rto is Type, so I declared a local variable localRto whose declared type makes it promotable to CallWithTypeParameters. Next, we must handle the case where the type argument doesn't have support for CallWithTypeParameters. In this case the invocation of Type.reify<X, CallWithTypeParameters>() returns null, and then we have to invoke callWithTypeParameter one more time requesting nothing more than a Type.

Here is the result:

abstract class CallWithTypeParameters {
  int get numberOfTypeParameters;
  R callWithTypeParameter<R>(int index, R Function<X>() callback);
}

class RtoForList<E> implements Type, CallWithTypeParameters {
  int get numberOfTypeParameters => 1;
  R callWithTypeParameter<R>(int index, R Function<X>() callback) {
    if (index != 1) throw ArgumentError("Expected index 1");
    return callback<E>();
  }

  String toString() => 'RTO for List<$E>';
}

class NotList {}

Request? Type_reify<Reify, Request>() {
  if (Request != CallWithTypeParameters) {
    throw ArgumentError("Unsupported type");
  }
  return switch (Reify) {
        const (List<List<List<NotList>>>) => RtoForList<List<List<NotList>>>(),
        const (List<List<NotList>>) => RtoForList<List<NotList>>(),
        const (List<NotList>) => RtoForList<NotList>(),
        _ => null,
      }
      as Request?;
}

void f(Type rto) {
  Object localRto = rto; // Enable promotion.
  print(localRto);
  if (localRto is! CallWithTypeParameters) {
    return;
  }
  if (localRto.numberOfTypeParameters != 1) {
    throw ArgumentError("Expected 1 type parameter");
  }
  var nextRto =
      localRto.callWithTypeParameter(
            1,
            <X>() => Type_reify<X, CallWithTypeParameters>(),
          )
          as Type?;
  if (nextRto == null) {
    // No support for `CallWithTypeParameters`, just get the plain RTO.
    nextRto = localRto.callWithTypeParameter(1, <X>() => X)!;
  }
  f(nextRto);
}

void main() {
  f(RtoForList<List<List<NotList>>>());
}

The crucial part is, as usual, the creation of a reified type object based on a given type (in this case: a type argument which is passed to Type.reify). So if the type to reify is MyClass<T1 .. Tk> then we create an instance of RtoForMyClass<T1 .. Tk> and return that. In this case we're returning an RtoForList<E> for some E.

The emulation in Type_reify needs to handle each specific case separately because we don't have this feature today, but all it does it to perform that mapping for a finite set of special cases that happen to be enough to run the program.

Also, I don't understand where X comes from.

X occurs twice. The first occurrence is a type parameter declaration of the function literal which is the second argument to callWithTypeParameter, and the second one is a usage where it is passed to Type.reify as the first actual type argument.

How does it help solve the original problem of printing nested type of the form List<List<List<NotAList>>>>?.

Run it and see!

The first invocation passes an RTO for List<List<List<NotAList>>>> which has type CallWithTypeParameters (as well as Type). The invocation of rto.callWithTypeParameter will invoke the function literal with List<List<NotAList>>> as the actual type argument which is bound to X, and this means that the invocation of Type.reify will return an RTO for List<List<NotAList>>> which has static type CallWithTypeParameters?.

If the result is null then the type argument doesn't support CallWithTypeParameters, so we compute the plain RTO in another invocation of callWithTypeParameter, and then call f recursively.

I'm sure this could be written more elegantly by working on a type parameter rather than on an RTO, but even with the RTO and with some type arguments that don't support CallWithTypeParameters it is still possible.

How can the function determine whether rto has a type argument of the form List<Something>?

The reification (including Type.reify) is able to create a reified type object for the given type. When the given type is of the form MyClass<T1 .. Tk>, the resulting RTO is an instance of RtoForMyClass<T1 .. Tk>, and RtoForMyClass can have various capabilities—in this case: CallWithTypeParameters. This allows us to explore the actual type argument that you call Something.

Here's another version that works without parametrized Types or existential open:

Type.typeArguments(rto)[0] and Type(rto).typeArguments[0] are slightly similar to an existential open, but much weaker. In particular, you can never use this kind of mechanism to get access to a type, you only get access to an RTO. That's hardly worthwhile (unless, of course, you introduce a smart feature like 'more capable Type objects' ;-).

This is another "minimal set" of functions that enables writing "generic deserialization without redundancy".

Then please use it to re-do this example. I find it highly unlikely that this would be possible. Having an RTO for List<int> doesn't allow you to create a List<int>.

@ghost
Copy link

ghost commented Mar 18, 2025

Thanks for the example. I'm running it, inserting prints here and there, and have already come up with a couple of theories about its behavior. One of the conjectures is that the compiler is trying to solve the system of inequalities over X, R, and other letters. This process is ongoing, and the results will be reported at the end of the experiment. (BTW, how was I supposed to know all that? Could you provide a link to something not too academic?)

Anyway, we have a parallel game on another board, let's switch to that for now.
Here's the current position:

void f(Type rto) {
  print(rto);
  if (Type(rto).isSubtypeOf(List)) {
    f(Type(rto).typeArguments[0]);
  }
}

I don't understand your criticism of the mechanism for being "weaker" than ex. open. You remember, I floated the idea of planting an ex. open into dart, but you were not excited about it, referring to some (unspecified) difficulties of implementation. Now, sure, we have to substitute a (possibly) weaker version to address that concern. BTW, how much weaker it is - not clear, it might be the same for all practical purposes. If the problem is "Having an RTO for List doesn't allow you to create a List" then it's solvable trivially: Type(rto).instantiate() 1. This mechanism is straightforward and doesn't impose the prerequisite of PhD degree on the user. It doesn't even require any substantially new features in the language: the runtime has all the information about the types available - it's just a matter of exposing this information to the user. As a result, we get a subset of operations normally requiring reflection, but without reflection, and for free. What not to like here? :-)

(Even without "instantiate", the generator can generate a map "type -> new instance", for all collection types encountered in the program, which is enough to implement generic deserialization).

Footnotes

  1. it returns null if the operation is not supported for the type. Can be enabled for all collections - in a way similar to CallWithParameters, but much easier to understand. If you need to pass type args, you can do it: Type(rto).instantiate([String, int]).

@eernstg
Copy link
Member Author

eernstg commented Mar 18, 2025

One of the conjectures is that the compiler is trying to solve the system of inequalities over X, R, and other letters.

(Note that I haven't used those letters as anything other than the names of completely normal type parameters. No special compiler algorithms needed, but, of course, the standard ones like type inference can be applied according to the normal rules. But we can return to that later. ;-)

(BTW, how was I supposed to know all that? Could you provide a link to something not too academic?)

I think the missing part is that I'm proposing a mechanism which has one core behavior that we can't recreate in current Dart: The ability to create an instance of a compiler-generated (but completely normal, no-magic-needed) class ReifiedTypeForC<T1 .. Tk> in response to an action on a type C<T1 .. Tk>. The action is (1) evaluate a type literal whose value is C<T1 .. Tk> as an expression (for instance, we might evaluate a type variable X whose value is C<T1 .. Tk>); or (2) it is Type.reify<S, ...>() where S is a type whose value is C<T1 .. Tk>.

That's it. The rest is normal Dart semantics.

However, we need some syntax to specify how the compiler can generate that class. The clauses static implements S and static extends S on the declaration of C specifies some constraints on the class, and the compiler will then know how to generate ReifiedTypeForC based on C and S. The only thing the compiler can do is to add method implementations that are missing (again: according to completely normal rules about concrete classes and abstract member declarations and overriding), and the only implementation it can generate is a forwarding declaration that will invoke a static member or a constructor declared by C.

Next, we need some syntax to specify that a given type variable has a value that satisfies static implements/extends S, which is declared as X static extends S. This means that we can safely invoke X.foo() where foo is a member of the interface of S (which might again invoke a static member or constructor of C, or, in case of class C static extends S ..., an inherited implementation of a member of S).

It is not a design which is known from other languages, but it does have conceptual connections to the notion of metaclasses (mostly known from dynamic languages like Smalltalk or CLOS), so I can't point to any papers where this idea has been further unfolded. Instead, the design and explanation work is what I've been doing here. ;-)

I don't understand your criticism of the mechanism for being "weaker" than ex. open.

Consider an arbitrary method:

class List<E> {
  Iterable<T> map<T>(T toElement(E e)) {...}
  ...
}

If RTOs are just as useful as types then we can go ahead and change this method such that it receives an RTO rather than a type argument:

class List {
  final Type e; // RTO of former `E`.
  // Suitable constructors where `e` is initialized.

  Iterable<Object?> map(Type t /*RTO of former T*/, Object? toElement(Object? e)) {...}
  ...
}

We can't use e and t in the ways that we used E and T. The reason for this is that an RTO is hugely weaker than a type. I don't understand how you can even consider the idea that you'd be able to do the same kind of things with RTOs as the things you can do with types.

I floated the idea of planting an ex. open into dart, but you were not excited about it,

I've argued in favor of that idea for many years. However, it is such a massive effort that it isn't going to happen unless it get's promoted heavily (in particular: it's explicitly, consistently, eagerly, massively requested by the community). Even with massive support it's going to be a big effort because it introduces a new kind of types.

(For example, it didn't take long for that kind of typing to be eliminated from the 'patterns' feature, because it was beyond reach with the given schedule. And patterns wasn't a small thing.)

'More capable type objects' is a tiny, lightweight thing in comparison. It doesn't give us exactly the same, but it does allow us to get the actual behavior associated with an existential open, and then we will have to insert a couple of type casts here and there because it isn't a full 'existential open' feature. I think that's a useful trade-off.

Type(rto).instantiate(). This mechanism is straightforward

It looks like an invocation of a constructor of Type, but rto looks like it would be an instance of Type already. So perhaps it's something that we'd call a 'copy constructor' in some languages?

Anyway, we now have an instance of type Type, and it might be an RTO (nothing prevents class MyType extends Type {}). Let's assume that it is an RTO for the type C<T1 .. Tk> for some class C and some types T1 .. Tk.

So we can now call instantiate on an instance of Type. Presumably it would return an object whose run-time type is C<T1 .. Tk>. However, if instantiate is an instance method on Type then it would conflict with all other instance methods named instantiate on Type; in particular, we'd need to use different names for different signatures (e.g., we need a new name for every constructor that has a different set of named formal parameters, or just differently typed formal parameters).

How would you type check this kind of object construction statically? If all these methods are members of the interface of Type then it's going to get really crowded. On the other hand, if it's always invoked dynamically because we can't have an interface that includes "one member for every constructor signature" then it's going to be less safe than what I'm aiming for.

To me it looks like it's getting less and less straightforward the more I try to understand what it would do.

the generator can generate a map "type -> new instance", for all collection types encountered in the program, which is enough to implement generic deserialization).

As I mentioned, that won't suffice if you write a generic solution where the same code can handle more than one type.

@ghost
Copy link

ghost commented Mar 18, 2025

(Note that I haven't used those letters as anything other than the names of completely normal type parameters. No special compiler algorithms needed, but, of course,

I've already figured that out. When <X>() => X is matched against R Function<X>() callback, the X is the first "expression" has nothing to do with the X in the second - you could as well write it as <XYZ>() => XYZ. It's an existential type variable saying "There must exists a type XYZ that, being substituted covariantly into the second expression as X, and contravariantly as R, works in both roles". I didn't know such a trick existed. (Hope I got it right). (I may still have confused "co-" with "contra-", but that's OK)

However, if instantiate is an instance method on Type then it would conflict with all other instance methods named instantiate on Type

No, it won't conflict with anything, due to special syntax Type(t). Ok, let's rename it (temporarily) to TypeExt, just to avoid confusion.
The expression TypeExt(t) works in the same way as the disambiguation like E(obj).extensionMethod() while explicitly calling the method from the extension E, which won't conflict with anything from the object itself.

How would you type check this kind of object construction statically?

But your method Type.reify is not a saint either. :-)
The whole thing based on Type.reify is dynamically checked, it can cause a throw.

To me it looks like it's getting less and less straightforward the more I try to understand what it would do.

You are biased! :-)

I understand your point though: you are trying to advance the theory of generics, and I find the intention highly laudable.
I can certainly sympathize with that. But if your final product looks like Zones... I don't know. Maybe you've found a stepping stone towards a real breakthrough, but in the end, everyone will appreciate you producing a real gem, something to behold. :-)

Maybe you can play on the second board more positively? Explore other things you can do with TypeExt(rto)... How about that? :-)

@eernstg
Copy link
Member Author

eernstg commented Mar 19, 2025

When <X>() => X is matched against R Function<X>() callback, the X is the first "expression" has nothing to do with the X in the second - you could as well write it as <XYZ>() => XYZ.

Right, the X in <X> is a type parameter declaration and it introduces X into a scope that includes the second occurrence. Hence, the second X is a reference to the first one. Same thing at the value parameter level: (x) => x. Again, nothing changes if you rename the (value) parameter: (xyz) => xyz, and callers never need to know the name of a positional parameter (which includes all type parameters).

It's an existential type variable saying "There must exists a type XYZ that, being substituted covariantly into the second expression as X, and contravariantly as R, works in both roles".

It's a parameter (more specifically: a type parameter), and it is provided at each call site. It is quite uncommon to say that it is 'existential', and it is certainly not more existential than a value parameter: If you consider the function literal (x) => x then the parameter x is existential in the sense that "whenever this function is invoked, there will exist an object which is bound to x". But it isn't very helpful to use a fancy word like 'existential' when we just want to communicate that a particular identifier is a parameter (one occurrence will be the declaring one and all others in the associated scope are references).

due to special syntax Type(t) ... rename ... to TypeExt

Was it ever defined? It looks like a constructor invocation. Does it denote an object? Which one? What is the static type of that object? (Presumably, the members we can invoke will be the ones that belong to that static type.)

The expression TypeExt(t) works in the same way as the disambiguation like E(obj).extensionMethod()

This sounds like Type(t).instantiate() invokes the member SomeType instantiate() on t, using Type as an extension of sorts.

Extension method invocation is statically resolved, which means that it is inherently unable to perform any kind of dispatch at run time. In other words, we can only use this kind of semantics in a situation where we could also invoke the given method statically (so if t reifies a class C then we could just as well do C.instantiate()).

This contradicts the most fundamental purpose of the proposal in this issue: It's all about calling static members and/or constructors without knowing the exact type of the class that declares them. In other words, there's no way it could be relevant to talk about statically resolved invocations here.

Could there be some other way to understand what you said such that the invocation is dispatched (that is, late-bound)?

If it is dispatched (using anything like the normal OO method invocation semantics) then there must be an underlying object (and it isn't the RTO denoted by t?), and that underlying object has a member called instantiate, and that's what we are calling when we do Type(t).instantiate() or TypeExt(t).instantiate(). If we wish to use this mechanism to invoke constructors of the type which is reified by t then there will be other cases where we wish to invoke instantiate(aName: anArgument), and others again like instantiate(anotherName: anotherArgument, aName: aDifferentlyTypedArgument) and so on ad infinitum.

You can do it if Type(t) has type dynamic. I don't see any other way, because of the massive name clash (there's no finite limit on the number of signatures we might encounter using the name instantiate, or any other name for that matter).

The whole thing based on Type.reify is dynamically checked, it can cause a throw.

Type.reify<T, S>() has return type S?, which is fully statically safe.

It is true that it may be able to throw in the case where multiple extensions can deliver an RTO of the desired type (we could also just return null), but there's a huge difference between the type safety of an operation which is invoked on a receiver of type dynamic, and an operation which is statically typed, but may throw.

We can also choose to restrict the expressive power of Type.reify such that it is guaranteed at compile time that there will never be an ambiguity. For instance, if we require S to be a ground type (not a type variable) then we can decide at compile time exactly how many extension provided static implements/extends U members have a U which is a subtype of S, and then we can just report a compile time error if it is zero or it is more than one. That's just a design choice. In contrast, I can't see how Type(t).instantiate() can be any safer than dynamic.

Maybe you can play on the second board more positively?

I'm trying! However, doing type things with RTOs is just such a huge impediment that we don't have to impose on ourselves, and I honestly can't see how it could work. ;-)

@ghost
Copy link

ghost commented Mar 19, 2025

Consider

extension E on String {
  get length=>100;
}
main() {
  print(E('abc').length); // prints 100
}

This is the syntax of explicitly calling the method from the extension. (Please try it out). One may ask: what is the type (static, runtime, whatever) of the expression E{'abc'). The answer: the question doesn't make sense! var x = E('abc'); will be flagged as an error, so the question of type is irrelevant.

Now let's assume dart introduces a built-in extension named TypeExt, or Types, or something (it could be even Type but might cause confusion), like

extension Types on Type {
   Object? instantiate()=>... something
}

This resolves the issue of what static type is and what it is not.
This extension comes from the compiler, we cannot write it ourselves.

var t=List<int>;
Object? list=Types(t).instantiate();

Sure, its static type of is Object?, it has to be eventually cast to something concrete to be useful. But this happens all the time when we use reflection. Basically, Types gives us access to a miniscule part of a reflection mechanism.
Reflection existed in dart from day 1, but... you know the history. It is not supported not because it's unsafe, but because it's very expensive. But here, we get this functionality for free, with no extra overhead normally associated with the reflection.
(We already have a fragment of reflection-like functionality in Function.apply(), which is unsafe, but no one complains. We have dynamic, which is unsafe from top to bottom. You certainly know more examples).
Eventually the result of instantiate will be cast to a proper type.

class A {
  List<List<int>> list;
  static A fromJson(StringBuffer s) {
    // generated code!
     var a=A();
     a.list = deserialize(List<List<int>>, s) as List<List<int>>;
     // "deserialize" uses Types(t).instantiate recursively under the hoold
  }
}

Does it make sense?
(I will write about (non-?)existential X later, to avoid piling up too many things :-)


A separate question:
if the issue of supporting parametrized Type<...> (i.e. String == Type<String>) is related to backwards compatibility only, then maybe we can try to define another thing like Rto<T> such that `

assert(String == Rto<String>);
assert(Rto<String> is Type);

I haven't thought it though. (Need a better name for Rto anyway)

@eernstg
Copy link
Member Author

eernstg commented Mar 19, 2025

It is true that E('abc').length is an explicitly resolved extension method invocation, and also that E('abc') does not have a type (as I wrote here).

We could indeed have an extension (let's use the name TypeExt for that) and it could have some members whose implementation is provided by the system (because it needs to do "magic" things that we can't write as Dart code).

extension TypeExt on Type {
   Object? instantiate() => /*magic*/;
}

Let's say that the "magic" code allows us to invoke a constructor whose name is the name of the class and whose parameter list allows for invocations with no arguments.

However, honestly, this is a very rudimentary feature.

First, as you mention, every object created in this way has static type Object?, and any other typing must be established by is and/or as. There is no connection between the type of the new object and the typing properties of the RTO which is used to perform the extension method invocation.

Next, it only works with classes that have such a constructor (with that kind of name, and with a parameter list that allows us to pass no arguments).

Consider the proposal in this issue:

abstract class NoArgInstantiable<Self> {
  Self call();
}

abstract class A {
  int get aThing => 0;
}

class B1 implements A static implements NoArgInstantiable<B1> {
  B1([int weCanAcceptOptionalArguments = 1]) {...}
  int get b1Thing => 1;
}

class B2 implements A static implements NoArgInstantiable<B2> {
  B2({String alsoNamedOnes = "Hello!"}) {...}
  int get b2Thing => 2;
}

X foo<X static extends NoArgInstantiable<X>>() => X();
  
void main() {
  // All statically typed.
  var b1 = foo<B1>();
  var b2 = foo<B2>();
  var i = b1.b1Thing + b2.b2Thing;
  A a = someCondition ? b1 : b2;
  i += a.aThing;
}

This is not the feature, this is just one example showing what it can do. In particular, we have created a static interface that allows us to express the constraint that a class must have a constructor with the name of the class that accepts the empty argument list (it can have optional parameters, it just has to allow for the invocation passing no args). We could also specify that it must accept one positional argument of a specific type (that we can parameterize as a type parameter on the static interface, if need be), or any other shape of formal parameter list. That is, we can just write the code which is needed in order to support any signature whatsoever, and also to support a constructor whose name is C.name where C is the name of the class.

There are no name clashes because all those signatures will be expressed in separate classes similar to NoArgInstantiable.

As you can see, X() has the type X because X has the static interface NoArgInstantiable<X>, so there's no Object? or dynamic typing about the body of foo. Next, the type of the new object is expressed both precisely and in a flexible manner using X static extends NoArgsInstantiable<X>. For example, it works fine with B1 and B2 even though those classes are unrelated (and the common supertype A does not support the same kind of construction because it's abstract).

This illustrates that the typing of late-bound object creation is just as strict as other kinds of Dart code.

Here is how I think it would be done using TypeExt:

abstract class A {
  int get aThing => 1;
}

class B1 implements A { // Can't declare that it supports `instantiate()`.
  B1([int weCanAcceptOptionalArguments = 1]) {...}
}

class B2 implements A { // Can't declare that it supports `instantiate()`.
  B2({String alsoNamedOnes = "Hello!"}) {...}
}

X foo<X>() => TypeExt(X).instantiate() as X;
  
void main() {
  var b1 = foo<B1>(); // Type `Object?`.
  var b2 = foo<B2>(); // Ditto.
  var i = (b1 as B1).b1Thing + (b2 as B2).b2Thing; 
  A a = (someCondition ? b1 : b2) as A;
  i += a.aThing;
}

I can't see how we can avoid dropping the static typing on the floor basically at every step in the code (and then we have to restore the type using as SomeType). Nothing stops us from getting in wrong and doing (b2 as B1).b1Thing or (someCondition ? b1 : b2) as SomeOtherType, it just fails at run time.

this happens all the time when we use reflection

This feature is not reflection, it's a much smaller, simpler, and safer thing. The only piece of magic I'm proposing is the ability to create an instance of ReifiedTypeForC<T1 .. Tk> based on the type C<T1 .. Tk>. The rest is just ordinary Dart code.

Even the implicitly generated classes could be written by hand. If we do that, though, then we'd need some other way to tell the compiler that this is the class (ReifiedTypeForC) that you must create an instance of when we reify that type (C), and we must enforce that C and ReifiedTypeForC take exactly the same type parameters with exactly the same bounds. That's the job done by static extends and static implements.

But here, we get this functionality for free, with no extra overhead normally associated with the reflection

I agree very much on the idea that it should be possible to introduce a mechanism which is a tiny corner of a reflective system which has no run-time cost for code that doesn't use this feature. It's just that I also want expressive power, and I want static typing wherever possible.

@ghost
Copy link

ghost commented Mar 19, 2025

I need time to process that.

Meanwhile, I'm thinking of one fascinating possibility :-)
The extension TypeExt or whatever its name is, is a language construct. It doesn't necessarily have to follow strict dart rules (the user cannot define such an extension anyway b/c magic).
Suppose TypeExt(const Type).instantiate() returns the result of static type Type? 1, and if called with non-const type, returns the result of Object? type (because there's nothing else meaningful to return anyway).
Now TypeExt(List<int>>).instantiate() returns a variable of a static type List<int>. Does it solve the problem?
Or we are just sweeping the real problem under the carpet?


A theory question. I don't understand how compiler handles generics in dart. In the languages where primitive types are not Objects (e.g. zig, julia, C++ etc) each invocation of a generic function may behind the scenes create a different version of the function, or struct, or something (it can be memoized to avoid repetition if called with the same type parameter from several places). What happens in dart - I have no idea (only conjectures). Could you point to some post where this is explained in non-academic language? ChatGpt was of no help: it characterized zig/C++ approach as "monomorphization", and noted that the languages with reified types (like dart) don't need that, but it doesn't tell me anything about how it actually works. What happens with all these Xs and Ts? How are they represented during compilation phase? They are clearly comptime constants, but on each invocation, they are different constants. 😕

Footnotes

  1. I mean, the variable of the same Type as passed as a parameter as a constant Type literal. The idea that the parameter is const, but the returned value is not, is not expressible in a normal dart syntax.

@ghost
Copy link

ghost commented Mar 20, 2025

Let's recall our initial exchange about the generator of deserializer for List<List<...<int>>>. I said my generator creates a dedicated function for each kind of a list: e.g. deserializer for List<List<List<int>>> invokes (in a loop) a deserializer for List<List<int>>, etc.
All these elementary deserializers are statically safe.
Your argument was that we generate too much code, and a better way could be found by implementing some generic recursive function.
Upon thinking more about it, here's my conjecture.
Indeed, it would be possible to implement such a function so that for every explicit invocation in the user's program, it will be statically safe. In other words, when the user invokes it as var list=deserializeList<List<List<int>>, passing type argument List<List<int>> (exact syntax doesn't matter), then it would be statically safe to do so. But inside the implementation of deserializeList, the recursive calls to deserializeList of a lower dimension won't be statically safe. I think that's OK for practical purposes, but we are discussing a general topic, so we have to get to the bottom of why it's not statically safe.

The argument is rather hazy, and I can't prove it as a theorem or something, but my conjecture is that while attempting to make these turtles (all the way down) statically safe, we would have to implement something equivalent to monomorpization - that is, generating a number of instances of deserializeList, each with specific parameter, which, in turn, would be equivalent (in terms of the amount of code, and also functionally) to the original setup where I generated these turtles by a generator.

I admit that I don't know exactly what I'm talking about, but hope that it will somehow make more sense to you than it does to me :-)

@eernstg
Copy link
Member Author

eernstg commented Mar 21, 2025

The extension TypeExt or whatever its name is, is a language construct. It doesn't necessarily have to follow strict dart rules

That's true! This is exactly the loophole that I'm using with Type.reify: An invocation of Type.reify is compiled into code that searches all extensions in scope (if they satisfy some additional criteria), and that wouldn't be possible if Type.reify had been an actual function.

One consequence of this is that it is not possible to obtain a function object that behaves in the same way as Type.reify (you can't tear it off). I don't think that's a problem because Type.reify is actually not a function, it's a language construct.

Suppose TypeExt(const Type).instantiate() returns the result of static type Type?, and if called with non-const type, returns the result of Object? type (because there's nothing else meaningful to return anyway).

That would be possible, surely, but the fact that the late-bound instance creation expression has such a general type is the main reason why I strongly prefer a mechanism that allows us to get a much more specific type.

In the example that I showed here, X() is statically known to have type X because this is known to be a type that satisfies static implements NoArgInstantiable<X>. It follows (strictly and soundly!) that if foo is called with an actual type argument whose value is B1 then it returns an instance of type B1 as well, and similarly for B2. And it is known statically that B1 and B2 are appropriate type arguments for an invocation of foo because they declare that they static implements NoArgInstantiable<T> where T is B1 respectively B2.

It is also known statically that A does not have this property, which means that foo<A>() is rejected as a compile-time error.

Now TypeExt(List<int>>).instantiate() returns a variable of a static type List<int>. Does it solve the problem?

No, there's no point in using late-bound invocations of static members or constructors if you know the exact type, you'd just do <int>[] (in general: you would just call the constructor exactly as we've always done).

The only situation where this mechanism is useful at all is the case where the type isn't known at compile time: It is a type variable, or a type that contains type variables, and you want to call a constructor of that type (or a static member of the underlying class/enum/mixin).


About generics in Dart: They are reified, which means that there is information which is encoded in the run-time state of the program that represents the value of each actual type argument (of each instance of a generic class, of each invocation of a generic function, etc).

This is just like value parameters and instance variables: If you investigate the run-time representation of a Point(3, 4) then it will have some amount of memory that holds the information that the first coordinate is 3, and you're calling foo(3, 4) then the activation record for this invocation will have a representation of 3 as the first actual argument.

In that sense, formal type parameters and actual type arguments are exactly the same kind of pair as formal (value) parameters and actual (value) arguments, except that (1) value arguments are objects and type arguments are types (which are not objects and cannot be the value of an expression in Dart), and (2) value arguments can only be omitted in the case where they are declared to be optional, but type arguments can generally be omitted, and they will then be provided by type inference (OK, the tools can say "I couldn't find a suitable type argument", but the basic approach is that type arguments can always be omitted, and then we get a compile-time error in those unusual situations where it isn't possible to compute them).

This explains why <int>[] can describe its run-time type (print(<int>[].runtimeType) as 'List' rather than just 'List' (which is all you'd be able to know in, say, Java, because Java erases the values of actual type arguments rather than reifying them). This is possible because there is a location in the representation of the list that stores information about the type argument, so we can just go there and see what it is.

A language that uses monomorphization will include constraints to ensure that it can be decided at compile-time exactly which actual type arguments are used at every location that applies a generic entity to a list of actual type arguments. If you don't have that kind of restriction on the expressive power of the type system then you can't monomorphize the program.

However, when it is guaranteed that programs can be monomorphized, it's possible to eliminate all type arguments in a program by "macro expanding" every usage of a generic declaration. This basically means that we're inlining the actual type arguments into each instantiation of a generic class (for example, we'd create one copy of List where the type E is replaced by int everywhere, and another copy of List where it is String, and so on). This can give rise to a size explosion, of course, and there has been a lot of work on sharing generated code when it turned out to be exactly the same code with certain pairs of type arguments.

They are clearly comptime constants, but on each invocation, they are different constants.

That's not true!

class A<X> {
  A();
  A<A<X>> get next => A<A<X>>();
}

main() {
  var i = 20;
  A a = A<Null>();
  while (--i > 0) a = a.next;
  print(a.runtimeType);
}

This shows that there's no finite limit on the number of different types we can have at run-time. This also implies that it is undecidable in general to compute the value of a Dart term that denotes a type at compile-time.

@ghost
Copy link

ghost commented Mar 21, 2025

I understand:

  • why Type<T> is complicated.
  • why .typeArguments is no good
  • why pattern matching if (t case List<var X>) is untenable.
  • why TypeExt(t) is ugly

But I think there's an option that could satisfy the most sophisticated taste: pattern matching based on (quasi-)static types.
Here's a strawman:

f<X>() {
  print(X);
  if (X typecase List<final Y>) f<Y>();
}

Important: typecase is not applicable to runtime types (the operands are "constant type expressions" - that is, either a literal type, or some type expression involving type arguments in tail positions).

BTW, it doesn't make Type.reify unnecessary. But it certainly makes CallWithOneParameter (the ugliest part) unnecessary.
WDYT?


If this works, then consider something like

X build<X, Y>(Y y) {
  if (X typecase List<final Z>) { 
    return <X>[build<Z, Y>(y)];
  } else if (X typecase Set<final Z>) {
    return <X>{build<Z, Y>(y)};
  } else if (Y typecase X) {
    return y as X;
  }
  throw ArgumentError("Inconsistent call `build<$X, $Y>(_)");
}

Here's a nicer form based on "switch without a scrutinee"

X build<X, Y>(Y y) {
  return switch {
    X typecase List<final Z> => <X>[build<Z, Y>(y)].
    X typecase Set<final Z> => <X>{build<Z, Y>(y)},
    Y typecase X => y as X,
    _ => throw ArgumentError(...),
  }
}

Generalizing based on static interfaces:

X build<X, Y>(Y y) {
  return switch {
     X typecase CollectionKind<final Z>) => X<Z>.empty()..add(build<Z, Y>(y)),
     Y typecase X => y as X,
     _ => throw ArgumentError(...),  
  }
}

where (improvising...)

abstract class CollectionKind<C extends Collection<C>> {
   Collection<C> empty();
}
abstract class Collection<E> {
   void add(E elem);
}

(We assume List<E> is retrofitted with static implements CollectionKind<E> implements Collection<E> and provides a constructor empty() and add method. Same for Set<E> and others of collection kind.)

The problem is that there's nothing in the predicate X typecase CollectionKind<final Z> that tells us that X implements CollectionKind statically - without this assumption, the invocation of X<Z>.empty() won't be valid. I am not sure how to fix that (there's a number of ways, need to choose).


I had a lively conversation with chatGPT about the issue. All variants it suggested (copied from other languages) are pure abstract nonsense. The approach based on static interfaces is much cleaner and more intuitive, but we have to find the best formalization.

@eernstg
Copy link
Member Author

eernstg commented Mar 26, 2025

Here's a strawman:

f<X>() {
  print(X);
  if (X typecase List<final Y>) f<Y>();
}

Important: typecase is not applicable to runtime types (the operands are "constant type expressions" - that is, either a literal type, or some type expression involving type arguments in tail positions).

But the operands in your example are not constant type expressions: X is a type variable, which is not constant. Basically, every type which is not constant is non-constant because of one or more type variables, so they are the essence of non-constancy.

I can't see how this could be anything less than an existential open operation, and that's a significantly bigger thing than the proposal I've made here. It is also a quite different thing, because an existential open operation won't help us call static members and constructors.

I do agree, though, that an existential open operation (using any syntax we might want for that) can be used to implement the build example.

The point I wanted to make when I mentioned the build example was actually the opposite: "Look, we can do it with this simple mechanism, we don't need a big and expensive thing like existential open to do it!" 😀

@ghost
Copy link

ghost commented Mar 26, 2025

They are not constant type expressions, but they are not runtime types either. I tried to find the formal name for this kind of thing in the spec, but it's not there.
You can easily model this kind of pattern matching if you use string representation of types like X above.
How can it be a "big and expensive thing" if I can write it easily based on strings? I can find a string representation of Y satisfying
X typecase List<final Y> 1, but I can't go from string representation to RTO as easily. But the compiler can!

The reason why this won't work with runtime types is that the true runtime types can have internal names like (making it up) _$InternalList, and the compiler can't promote anything based on types like that. Not to mention that when you create an instance of such type, the runtime type can be overridden and, in principle, may not match this internal type. At least, not exactly match, which might be confusing. 2

If the compiler doesn't implement this kind of matcher, I can't easily see how you can implement a generic deserializer for lists that can handle List<int>, List<List<A>, etc. all by one generic function. (The deserializer has been the main motivator for the feature.).

because an existential open operation won't help us call static members and constructors.

I don't understand that. I gave an example of build method in my comment.

Footnotes

  1. if X is List<List<int>, I can convert it to string "List<List<int>" and match it against a string "List<final Y>" using a regexp or my own matcher, from which I get a string representation of Y as "List<int>". The matcher needs access to the inheritance tree for all classes. This tree can be simulated manually or generated by a macro.

  2. in your latest write-up, there's a section where the entire logic is based on runtime types. This might be a problematic idea.

@eernstg
Copy link
Member Author

eernstg commented Mar 27, 2025

but they are not runtime types either

They are not obtained as the run-time type of an object, but they are definitely "run-time types" in the sense that (1) we can't compute such types at compile time, that's in general an undecidable question, and (2) they have a run-time representation, which means that we can actually check what they are when a program execution reaches that part of the code.

How can it be a "big and expensive thing" if I can write it easily based on strings?

How is your string based existential open going to handle the situation where more than one class in your program has the name C, and you encounter a List<C>?

How will it handle the situation where it encounters an Int8List as the value of e in e typecase Iterable<final X>? It would need to traverse the superinterface hierarchy above Int8List in order to discover that it implements List<int> which implements Iterable<int>. I think you'd need to create a database which is basically describing the structure of the program (in particular: all subtype relationships) in order to even get started handling this task.

How would it handle the bounds of the type variable? For example, with e typecase Iterable<final X> we might know that the static type of e is List<S> for an S which is declared as S extends Comparable<S>. The subsequent static analysis should then be able to handle X according to this knowledge (for instance, if x has type X then we can do x.compareTo(x) but x.compareTo('Hello') should be a compile time error).

And so on. It's not reasonable to claim that you can easily do these things based on Type.toString().

because an existential open operation won't help us call static members and constructors.

I don't understand that. I gave an example of build method in my comment.

There are several versions of build. The last one relies on 'more capable type objects' (renamed as 'metaobjects' in the PR), so that one is not relevant if your point is "we don't need metaobjects".

The first two versions do not use metaobjects, but they aren't invoking any static members or constructors using late binding, so they cannot possibly be relevant when the claim is "the build example does help us calling static members and constructors" (OK, I should have said "using late binding" at that point, but that's a given because that was the new thing in this proposal from the very beginning, and then later I discovered that it could also be used to deconstruct types, as in CallWithTypeParameters).

@ghost
Copy link

ghost commented Mar 27, 2025

I think you'd need to create a database which is basically describing the structure of the program (in particular: all subtype relationships) in order to even get started handling this task

Of course you need this database! Is it a big deal? The late Mirrors used to have this information available. It's not a huge database.

On the issue of two C from 2 different libraries - dart knows how to distinguish them today1, otherwise nothing would ever work!
It's just while simulating the matcher with Strings, I can't do it - but I never said the real version should be based on Strings.

The problems you mentioned are solvable. I even think they are easily solvable, but I don't want to make provocative statements. :-)

The last one relies on 'more capable type objects' (renamed as 'metaobjects' in the PR), so that one is not relevant if your point is "we don't need metaobjects".

Initially, I misunderstood the role of metaobjects. But my views evolved based on new information received during the discussion (which is the whole point of having any debate, to begin with). You need mataobjects, but they can be created behind the scenes without exposing anything like Type.reify to the user.

I have a clear mental picture of both the problem and the solution by now (maybe I'm missing some minor nuances, but that's OK) - so I consider the problem solved. Whether this solution will, or will not, be implemented, is orthogonal to the problem itself. If not, the problem can be reclassified as "recreational puzzle", which is fine with me. :-)

Footnotes

  1. in most languages, the problem never occurs b/c each class has a fully qualified name. In dart, there's no notion of FQN, but the runtime still tracks the information about who came from where and prints it in some error messages.

@ghost
Copy link

ghost commented Mar 27, 2025

I want to address this separately b/c the issue is fascinating in its own right

They are not obtained as the run-time type of an object, but they are definitely "run-time types" in the sense that (1) we can't compute such types at compile time, that's in general an undecidable question, and (2) they have a run-time representation, which means that we can actually check what they are when a program execution reaches that part of the code.

I'd call these types quasi-static. They can be computed in compile time (with some restrictions). The compiler can see the invocation of f<List<List<int>> and, by looking into the code of f, can figure out what patterns it's matched against, and who potentially calls whom recursively, and with which parameters. Sure, if the function f continues wrapping A<A<...<X>>, the process can diverge -BUT the compiler will detect its own inability to handle the case (by limiting the total number of iterations) and flag the invocation statically. Now the issue is what to do in these complicated cases. Two variants: 1) nothing - reject the code 2) allow overriding, thus enabling the thing to be relegated to runtime, with no guarantees of anything (neither convergence nor performance).

Some languages take variant 1 - and that's it. There's a hard limit on the complexity of type expressions. This is a pragmatic solution 1
You can still work around the limitations by tweaking the source, but that's on you. See a fascinating illustration. Also this

The major issue here is this: does dart want to handle the problem properly or not. The emphasis here is on "want". If the answer is "no" then a good argument can be made in favor of doing nothing at all (it's even less work after all! :))

Footnotes

  1. one interesting idea would be to introduce the measure of type expression complexity M(expr) and impose the restriction that M(expr[n+1]) < M(expr[n]) where n denotes the iteration #n. This will prevent issues that can lead to a "3x+1 Collatz conjecture" scenario.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Proposed language feature that solves one or more problems meta-classes
Projects
None yet
Development

No branches or pull requests

9 participants