-
Notifications
You must be signed in to change notification settings - Fork 340
[doc]: Doc fix for CLI usage #3215
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
| --service-account (Only for GCS) The service account to use when connecting to GCS | ||
| --property A key/value pair such as: tag=value. Multiple can be provided by specifying this option more than once | ||
| --catalog-connection-type The type of external catalog in [iceberg-rest, hadoop, hive]. | ||
| --catalog-connection-type The type of external catalog in [ICEBERG, HADOOP, HIVE]. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did we intentionally change it to lower case? cc @HonahX @CodingBangboo
Reference: https://github.com/apache/polaris/pull/2825/files, #2761 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think it should be the lower cased version, since they are the actual value of those enum type:
polaris/client/python/apache_polaris/cli/constants.py
Lines 51 to 58 in be3c88b
| class CatalogConnectionType(Enum): | |
| """ | |
| Represents a ConnectionType for an EXTERNAL catalog -- see ConnectionConfigInfo in the spec | |
| """ | |
| HADOOP = "hadoop" | |
| ICEBERG = "iceberg-rest" | |
| HIVE = "hive" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
K. let me fix the tool doc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so it doesn't appear it matters:
Argument(Arguments.CATALOG_CONNECTION_TYPE, str,
Hints.Catalogs.External.CATALOG_CONNECTION_TYPE, lower=True,
choices=[ct.value for ct in CatalogConnectionType]),
as we are applying lower=True anyway. Also, in other places, we are refers to upper case ones as well:
# Allows both REST and HIVE connection type
polaris.features."SUPPORTED_CATALOG_CONNECTION_TYPES"=["ICEBERG_REST","HIVE"]
Also, as all other arguments with enum are pass in as upper case, should we keep them as upper case then. For example:
CATALOG_AUTHENTICATION_TYPE = (
"The type of authentication in [OAUTH, BEARER, SIGV4, IMPLICIT]"
)
...
Argument(Arguments.CATALOG_AUTHENTICATION_TYPE, str,
Hints.Catalogs.External.CATALOG_AUTHENTICATION_TYPE, lower=True,
choices=[at.value for at in AuthenticationType]),
Another thing is we have it as ICEBERG but the actual value is iceberg-rest and in the java server side, we have it as ICEBERG_REST, should we keep them consistent?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm fine with [ICEBERG-REST, HADOOP, HIVE] in that case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for diving deep into this!
Upper case SGTM. As for the iceberg-rest to iceberg change that would be a breaking change for current users. It's worth to mention in the changelog if we modify it.
IMHO, ICEBERG-REST should be fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for review. As the current update is correct (the upper case ones). Should we proceed with this change first? I will create another PR for changing from ICEBERG_REST to ICEBERG-REST (have deprecation for ICEBERG_REST then we can remove it in the next release?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tested this against the main branch. Passing --catalog-connection-type ICEBERG currently fails with the following error:
yufei@Yufeis-MacBook-Pro-2 polaris % ./polaris --profile dev catalogs create --type external --catalog-connection-type ICEBERG --default-base-location file://temp/c1 --storage-type file --catalog-authentication-type BEARER --catalog-bearer-token my-token-xxx c1
usage: ...
polaris catalogs create: error: argument --catalog-connection-type: invalid choice: 'iceberg' (choose from 'hadoop', 'iceberg-rest', 'hive')
Given that we’ve already been using ICEBERG_REST on the server side, I think it makes sense to keep it as-is to avoid introducing a breaking change. Sorry for the confusion earlier, I didn’t mean to suggest changing it to ICEBERG-REST.
For the CLI, one option could be to standardize on uppercase enum-style values (e.g. ICEBERG_REST, HADOOP, HIVE) for consistency with other arguments.
What do you think? @MonkeyCanCode @HonahX
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it. Thanks for testing. Let me look into this tomorrow and fix this.
|
@flyrain mind take another look? Then I do the needed change for |
Synced usage from latest CLI.
Checklist
CHANGELOG.md(if needed)site/content/in-dev/unreleased(if needed)