generic functions throw TypeError once generic information is lost

[ahmednfwela]

the following code:

Object? _string(String value) {
  return value;
}

Object? _int(int value) {
  return value;
}

class GenericClass<T> {
  final Object? Function(T converter) converter;
  final T exampleValue;

  const GenericClass({
    required this.converter,
    required this.exampleValue,
  });
}

const stringVariant =
    GenericClass<String>(converter: _string, exampleValue: "hello");
const intVariant = GenericClass<int>(converter: _int, exampleValue: 10);

void main() {
  final genericClassList = <GenericClass>[
    stringVariant,
    intVariant,
  ];
  for (var element in genericClassList) {
    // this line throws type '(String) => Object?' is not a subtype of type '(dynamic) => Object?'
    final converter = element.converter;
    final result = converter(element.exampleValue);
    assert(result == element.exampleValue);
  }
}

throws the following error:

Unhandled exception:
type '(String) => Object?' is not a subtype of type '(dynamic) => Object?'

what's actually funny is if I run in debug mode, and type the same statement in debug console, it outputs the correct value:

then when going to the next line it throws:

• Windows 11
• Dart version and tooling diagnostic info (dart info)

If providing this information as part of reporting a bug, please review the information
below to ensure it only contains things you're comfortable posting publicly.

#### General info

- Dart 3.4.3 (stable) (Tue Jun 4 19:51:39 2024 +0000) on "windows_x64"
- on windows / "Windows 10 Pro" 10.0 (Build 22631)
- locale is en-AE

#### Project info

- sdk constraint: '^3.4.3'
- dependencies: 
- dev_dependencies: lints, test

#### Process info

| Memory | CPU | Elapsed time | Command line |
| -----: | --: | -----------: | ------------ |
| 111 MB |  -- |              | dart.exe     |
| 523 MB |  -- |              | dart.exe     |
| 106 MB |  -- |              | dart.exe     |
| 154 MB |  -- |              | dart.exe     |
| 102 MB |  -- |              | dart.exe     |

[dart-github-bot]

Summary: The code throws a TypeError when a generic function is assigned to a variable with a lost generic type. The error occurs because the compiler cannot infer the correct type for the function when the generic information is lost, leading to a type mismatch.

[ahmednfwela]

note that reintroducing the generic type again will make it work:

for (var element in genericClassList) {
  Object? result;
  if (element is GenericClass<String>) {
    result = element.converter(element.exampleValue);
  } else if (element is GenericClass<int>) {
    result = element.converter(element.exampleValue);
  }
  assert(result == element.exampleValue);
}

and completely removing type information will also make it work:

for (var element in genericClassList) {
  final result = (element as dynamic).converter(element.exampleValue);    
  assert(result == element.exampleValue);
}

[eernstg]

@ahmednfwela, I'm afraid you are using a dangerous combination of features. I created #59050 in order to request a diagnostic message when it occurs, such that developers will know that they shouldn't do this, or at least that they should be very careful when doing it. You can vote for that issue. ;-)

The core point is that GenericClass<T> has an instance variable whose type has a contravariant occurrence of the type parameter T:

class GenericClass<T> {
  final Object? Function(T converter) converter;
  GenericClass(this.converter);
}

void main() {
  GenericClass<num> g = GenericClass<int>((_) => 0);
  g.converter; // Throws, you don't even have to call it.
}

It throws because the static type of g.converter is Object? Function(num), but the run-time type is Object? Function(int), and the latter is not a subtype of the former (because function types are contravariant in their parameter types).

So the crucial point is that g is dangerous if and only if it has a static type which is GenericClass<T> and a dynamic type which is some subtype of GenericClass<S> where S differs from T (that is, necessarily, S is a subtype of T, in the example: S == int and T == num).

The reason why it helps to promote element to GenericClass<String> is that this aligns the static type and the run-time type (they are both GenericClass<String>).

The reason why it helps to use (element as dynamic).converter is that this eliminates the conflict between the static and the dynamic type (the static type is now dynamic, both for the receiver (element as dynamic) and for the getter invocation (element as dynamic).converter). So you don't expect anything specific, and we don't check that you get that specific kind of thing, and no exception is raised.

The simplest fix is to avoid using "contravariant members" at all. That is, don't declare an instance variable (final or not, that doesn't matter) whose type is a function type where a type variable of the class occurs as a parameter type of the function type.

There is another way to make the construct statically type safe, but it is somewhat involved: What you really need is to make the class invariant in that type argument.

We may add statically checked variance to Dart (see dart-lang/language#524), in which case we can do this:

// Needs `--enable-experiment=variance`

class GenericClass<inout T> {...}
...

If you do that then you will get a compile-time error at the initialization of genericClassList, which is the place where the covariance is introduced (which is the typing relationship that causes the exception to happen later on).

So this means that you'll get a heads-up when the problem is created, and you may be able to eliminate the problem at that point.

[ahmednfwela]

@eernstg thanks for the quick reply and explanation, I didn't know such a bug existed in dart for so long

can you also please explain why changing the structure of the class to use inheritance instead of passing variables makes it work ?

abstract class GenericClass<T> {
  const GenericClass({required this.exampleValue});

  Object? converter(T value);
  final T exampleValue;
}

class StringVariant extends GenericClass<String> {
  const StringVariant({required super.exampleValue});

  @override
  Object? converter(String value) {
    return value;
  }
}

class IntVariant extends GenericClass<int> {
  const IntVariant({required super.exampleValue});

  @override
  Object? converter(int value) {
    return value;
  }
}

void main() {
  final genericClassList = <GenericClass>[
    StringVariant(exampleValue: 'hello'),
    IntVariant(exampleValue: 10),
  ];
  for (var element in genericClassList) {
    element.converter(element.exampleValue);
  }
}

[ahmednfwela]

another workaround I discovered by doing inheritance:

abstract class GenericClass<T> {
  Object? converter(T converter);
  const GenericClass();
}

class GenericClassVariant<T> extends GenericClass<T> {
  const GenericClassVariant(this.converterFunction);

  final Object? Function(T value) converterFunction;

  @override
  Object? converter(T converter) {
    return converterFunction(converter);
  }
}

void main() {
  GenericClass<num> g = GenericClassVariant<int>((_) => 0);
  g.converter(5); // Works!
}

[eernstg]

Yes, when a type parameter occurs contravariantly it is in general much safer to use an instance method where some parameter has a type which is a type parameter of the class than it is to have an instance variable whose type is a function type with the same parameter type.

This is because method tear-offs get their type adjusted when torn off, and also because you may not need to tear off the method in the first place.

class A<X> {
  void f(X x) {}
}

class B<X> {
  void Function(X) f;
  B(this.f);
}

void main() {
  A<num> a = A<int>();
  B<num> b = B<int>((_) {});

  a.f(1); // OK, just calls `a.f` with arguments.
  b.f(1); // Throws because we evaluate `b.f`, with a type check (and then we'd call it, except that we don't get that far).
}

You can even tear off the instance method (as in a.f) safely, but you can't evaluate the instance variable without incurring the type check (as in b.f, that throws). The reason why this is possible is that the instance method doesn't exist as a separate function object at the time where it is torn off, so we can provide an object whose run-time type satisfies the expectation (a.f is void Function(num) is true). We can't do that with the instance variable because it is not acceptable to return a different object than the value of b.f (otherwise we could have returned a wrapper function like (num n) => b.f(n as int)).

We will need to check the type of the argument at some point, but with the instance method we can do it late, and with the instance variable we must do it already when someone gets hold of the function object.

The approach that uses converterFunction is another known workaround, and a good one (which was mentioned here, for example). The reason why this works is that you are calling converterFunction in a context where it has static type Object? Function(T) where T denotes the actual type argument. This means that there is no covariance, because this is actually a way to denote the run-time value of this type argument (the body of a class has access to the real value, clients only know something which may be a supertype).

[ahmednfwela]

thanks for the explanation, this is the most generic workaround I have found for this:

abstract class _FunctionWrapper<TOut, TIn> {
  const _FunctionWrapper();
  TOut call(TIn value);
}

class FunctionWrapper<TOut, TIn> extends _FunctionWrapper<TOut, TIn> {
  final TOut Function(TIn value) _function;
  const FunctionWrapper(this._function);

  @override
  TOut call(TIn value) {
    return _function(value);
  }
}

I am not sure if extension types can be used here to reduce the memory overhead of creating an object per function, but if you know a way to make it possible that would be great

Edit: found it:

extension type const FunctionWrapper<TOut, TIn>(Function fn) {
  TOut call(TIn value) {
    return fn(value) as TOut;
  }
}

however this accepts any function of any type, which might not be preferrable.

[eernstg]

Here is a rather general approach that does two things: (1) It emulates invariance for GenericClass, and (2) it introduces a non-generic superclass, GenericClassBase, such that you can have lists containing GenericClass instances with different type arguments (otherwise that would have to be a List<Object> because GenericClass<S> and GenericClass<T> are simply unrelated types unless S == T).

Object? _string(String value) {
  return value;
}

Object? _int(int value) {
  return value;
}

// Add a superclass to abstract away the type argument.

abstract class GenericClassBase {
  R callWithTypeArgument<R>(R Function<S>(GenericClass<S> self) callback);
}

// Emulate invariance for `GenericClass`.

typedef Inv<X> = X Function(X);
typedef GenericClass<T> = _GenericClass<T, Inv<T>>;

class _GenericClass<T, Invariance extends Inv<T>> implements GenericClassBase {
  final Object? Function(T converter) converter;
  final T exampleValue;

  const _GenericClass({
    required this.converter,
    required this.exampleValue,
  });

  R callWithTypeArgument<R>(R Function<S>(GenericClass<S> self) callback) =>
      callback<T>(this);
}

const stringVariant =
    GenericClass<String>(converter: _string, exampleValue: "hello");
const intVariant = GenericClass<int>(converter: _int, exampleValue: 10);

void main() {
  final genericClassList = <GenericClassBase>[
    stringVariant,
    intVariant,
  ];
  for (var element in genericClassList) {
    element.callWithTypeArgument(<T>(typedElement) {
      final converter = typedElement.converter;
      final result = converter(typedElement.exampleValue);
      assert(result == typedElement.exampleValue);
      print('Note that we have access to the real T: $T');
    });
  }
}

[eernstg]

I think we can close this issue because it's all about topics that have been raised elsewhere.

Do vote for #59050, though, such that we can finally get a compile-time heads up when someone uses one of those "contravariant members"! It's never going to happen if only 4 people in the universe cares about it. ;-)

[eernstg]

Ah, there was one more question that I overlooked: You can't use an extension type because they are using the same subtype checks on functions as any other part of the language (so you can't "hide a soundness violation" inside an extension type).

It is possible to use a wrapper class like FunctionWrapper, but that's just recreating the same issues with an extra layer on top, because FunctionWrapper<Tin, Tout> essentially provides access to a function under a type which is covariant in Tin, which is not sound. So you'd need to enforce invariance in Tin for all accesses to FunctionWrapper in order to make that class safe, and that's just extra work, no extra safety.

class FunctionWrapper<TOut, TIn> {
  final TOut Function(TIn value) _function;
  const FunctionWrapper(this._function);
  TOut call(TIn value) => _function(value);
}

void main() {
  FunctionWrapper<void, num> w = FunctionWrapper<void, int>((int i) {});
  w.call(1.5); // Throws.
}