[FLINK-38733] Add new SplitterEnumerator on JdbcSource

[eskabetxe]

This pull request refactors the JDBC source split enumeration logic in the Flink connector, replacing the previous SqlTemplateSplitEnumerator and related classes with a new abstraction called SplitterEnumerator. The builder and enumerator classes are updated to use this new abstraction, simplifying split management and improving extensibility. The changes also update lineage handling and testing methods to use the new splitter abstraction.

Split Enumeration Refactoring

• Replaced SqlTemplateSplitEnumerator and JdbcSqlSplitEnumeratorBase with the new SplitterEnumerator abstraction across JdbcSource, JdbcSourceBuilder, and JdbcSourceEnumerator, updating constructors, fields, and method calls accordingly. [1] [2] [3] [4]
• Updated the builder to support setting a custom SplitterEnumerator, and added logic to select the appropriate splitter implementation based on provided parameters. [1] [2] [3]
• Refactored enumerator startup and split assignment logic to use the new splitter, improving asynchronous split enumeration and handling both bounded and unbounded modes.

Lineage and Testing Updates

• Changed lineage vertex extraction to use the new splitter abstraction for query information, removing direct calls to SqlTemplateSplitEnumerator. [1] [2]
• Updated visible-for-testing methods to expose the new splitter abstraction instead of the old provider.

Miscellaneous

• Updated equality checks and state snapshot logic to use the new splitter abstraction and its serializable state. [1] [2]
• Removed obsolete annotations for the old split enumerator methods in the architecture violation files.

[eskabetxe]

@ferenc-csaky can you have a look when have free time..

[ferenc-csaky]

Did not check the implementation details yet, added some interface-related comments.

[ferenc-csaky]

Since this is a public API, we cannot throw it away like this, we need to deprecate it first and keep the related config methods (deprecated) as well.

[eskabetxe]

I think about this, and I don't deprecated it because it is only used internally and cannot be passed in any way to source.. could have a wrong annotation?

[RocMarshal]

I have rechecked the reason why this annotation was added at the time:

• A：The org.apache.flink.connector.jdbc.core.datastream.source.JdbcSource is annotated as @PublicEvolving
  • But the constructor methed of JdbcSource visible scope is default without public keyword, although It's a bit odd.
• B：Therefore, not only the classes directly exposed in JdbcBuilder, but also the classes exposed in the constructors of the Jdbc class should be annotated as PublicEvolving, that is, JdbcSqlSplitEnumeratorBase is annotated with PublicEvolving is correct.

Since this is a public API, we cannot throw it away like this, we need to deprecate it first and keep the related config methods (deprecated) as well.

Therefore, overall, this suggestion requires our careful consideration.

[eskabetxe]

So what is the recommendation? Keep the class and have two constructors on JdbcSource one of them deprecated?

As you say is a little odd the use today, as constructor is only accessible by Builder and there is the only place we use it..
For example the current lineage expects an SqlTemplateSplitEnumerator not and JdbcSqlSplitEnumeratorBase..

I will try to rethink this..

[RocMarshal]

So what is the recommendation? Keep the class and have two constructors on JdbcSource one of them deprecated?

As you say is a little odd the use today, as constructor is only accessible by Builder and there is the only place we use it.. For example the current lineage expects an SqlTemplateSplitEnumerator not and JdbcSqlSplitEnumeratorBase..

I will try to rethink this..

@eskabetxe
Sorry for not being able to devise a more elegant way to handle this transition.

• I prefer that we should at least keep this class for a while, so that users who are already relying on it have time to migrate and adapt.

• If we remove this class and its related components directly, it will certainly result in a poor experience for our users. If it must be removed, it would be best to include some release notes or documentation explaining the reasons behind this necessary change.

In contrast, the impact of this change on the SQL side is negligible.

Hope you have already thought of a better way to move forward.

[eskabetxe]

@RocMarshal @ferenc-csaky
Can you check again, I came with a new approach to keep this class during the transition..