[FLINK-37546] Use DefaultInstantiatorStrategy for FlinkScalaKryoInstantiator

[mxm]

KryoSerializer loads FlinkScalaKryoInstantiator via Reflection which configures Kryo to use StdInstantiatorStrategy. After FLINK-3154 removed the Twitter Chill library, various types are broken. See EsotericSoftware/kryo#1173

At first, it seemed that the Kryo update caused this but I can reproduce this also with Kryo 2.24.0 with the Chill library removed.

This change matches the default strategy when the Scala serializers are unavailable: https://github.com/apache/flink/blob/f7a53cede10fa25cb025d5554126a83773ba6bf9/flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/kryo/KryoSerializer.java#L496

[flinkbot]

CI report:

• 0b154bf Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[davidradl]

@mxm there are numerous table planner test failures in the CI tests.

[davidradl]

Please could you provide a unit test for this change.
Also the CI is failing - on the face of it due to this change.

[mxm]

Thanks for the review @davidradl!

I briefly looked into adding a test but it proofed a bit difficult due to zero tests for the Scala serializer extensions. The test would have to go into the Scala module or the e2e test module. It isn't possible to test this from flink-core where KryoSerializer resides because the code is dynamically loaded from the table api scala module.

I'm OoO currently but I'll take a look next week.

[michaelparkin]

Hi @mxm , @davidradl - is there any update on this PR?

We're experiencing the same problem and it would be great to have this in flink 2.0.1. Thanks.

[mxm]

Hey @michaelparkin! Could you share more details about your use case? When do you run into the serialization issue?

[neoneo008]

Hi @mxm @davidradl , any update on the PR.

We are facing similar issue with Mongodb connectors where we get null pointer since the constructor is not called by kyro. PR which has failure in build logs. apache/flink-connector-mongodb#57

Happy to help on this PR if anything is pending.

[mxm]

I think we need to fix some of the custom Scala serializers after this change. The current serializer configuration relies on the default strategy to be StdInstantiatorStrategy, which we changed here. I'm going to rebase and revisit this.