Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-33929][jdbc-connector] Support JDBC String field read Fragment read #87

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

zhilinli123
Copy link

Copy link

boring-cyborg bot commented Dec 28, 2023

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

@zhilinli123
Copy link
Author

PTAL: @MartijnVisser thanks !

@MartijnVisser
Copy link
Contributor

@zhilinli123 I still don't understand the Jira ticket, so I can't really review it. Perhaps @snuyanzin or @eskabetxe understand the goal of this PR.

@zhilinli123
Copy link
Author

@zhilinli123 I still don't understand the Jira ticket, so I can't really review it. Perhaps @snuyanzin or @eskabetxe understand the goal of this PR.

hi~ @MartijnVisser The current jdbc slicing support type is limited, currently does not support for string type fields for slicing read, many table design using UUID as the table key without self-increment primary key, resulting in the use of flink reading is no way to slice read, the current function is to support the current shortage

@eskabetxe
Copy link
Member

@zhilinli123 I still don't understand the Jira ticket, so I can't really review it. Perhaps @snuyanzin or @eskabetxe understand the goal of this PR.

For what I can assume, @zhilinli123 are trying to use scan.partition on a table that is hash partitioned by a string field..

My understanding of scan.partition is that only divides the query on multiples query based on params of scan.partition but is not using partitioning of table, it only adds "field BETWEEN ? AND ?" in query, and for that it only allow numeric, date, or timestamp column..

@zhilinli123
Copy link
Author

zhilinli123 commented Jan 16, 2024

@zhilinli123 I still don't understand the Jira ticket, so I can't really review it. Perhaps @snuyanzin or @eskabetxe understand the goal of this PR.

For what I can assume, @zhilinli123 are trying to use scan.partition on a table that is hash partitioned by a string field..

My understanding of scan.partition is that only divides the query on multiples query based on params of scan.partition but is not using partitioning of table, it only adds "field BETWEEN ? AND ?" in query, and for that it only allow numeric, date, or timestamp column..

@eskabetxe You're right, but slicing string hash md5 doesn't need to be done BETWEEN? AND ? It's much more efficient to check this shard directly

# Conflicts:
#	flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/table/JdbcDynamicTableFactory.java
@zhilinli123
Copy link
Author

If you have time, could you review it for me? Express gratitude @MartijnVisser @eskabetxe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants