Handle listTables for non-existing schema in IcebergNativeMetadata

[agrawalreetika]

Description

Handle listTables for non-existing schema in IcebergNativeMetadata

Motivation and Context

IcebergHiveMetadata returns an empty list instead of exception when the non-existing schema is queried. But currently in IcebergNativeMetadata

Here iceberg is configured as a Hadoop (Native) catalog of Iceberg.

presto> SELECT table_name FROM iceberg.information_schema.tables WHERE table_schema = 'non';
Query 20250222_203015_00002_jwvp3 failed: Namespace does not exist: non
org.apache.iceberg.exceptions.NoSuchNamespaceException: Namespace does not exist: non
        at org.apache.iceberg.hadoop.HadoopCatalog.listTables(HadoopCatalog.java:199)
        at com.facebook.presto.iceberg.IcebergNativeMetadata.listTables(IcebergNativeMetadata.java:179)
        at com.facebook.presto.spi.connector.classloader.ClassLoaderSafeConnectorMetadata.listTables(ClassLoaderSafeConnectorMetadata.java:266)
        at com.facebook.presto.metadata.MetadataManager.listTables(MetadataManager.java:551)
        at com.facebook.presto.connector.informationSchema.InformationSchemaMetadata.lambda$calculatePrefixesWithTableName$7(InformationSchemaMetadata.java:302)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271)
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at com.facebook.presto.connector.informationSchema.InformationSchemaMetadata.calculatePrefixesWithTableName(InformationSchemaMetadata.java:306)
        at com.facebook.presto.connector.informationSchema.InformationSchemaMetadata.getTableLayoutForConstraint(InformationSchemaMetadata.java:239)
        at com.facebook.presto.metadata.MetadataManager.getLayout(MetadataManager.java:381)
        at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout.pushPredicateIntoTableScan(PickTableLayout.java:308)
        at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout.pushPredicateIntoTableScan(PickTableLayout.java:253)
        at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout$PickTableLayoutForPredicate.apply(PickTableLayout.java:137)
        at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout$PickTableLayoutForPredicate.apply(PickTableLayout.java:102)
        at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.transform(IterativeOptimizer.java:237)
        at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreNode(IterativeOptimizer.java:189)
        at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:151)
        at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreChildren(IterativeOptimizer.java:281)
        at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:153)
        at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreChildren(IterativeOptimizer.java:281)
        at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:153)
        at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.optimize(IterativeOptimizer.java:138)
        at com.facebook.presto.sql.Optimizer.validateAndOptimizePlan(Optimizer.java:112)
        at com.facebook.presto.execution.SqlQueryExecution.lambda$doCreateLogicalPlanAndOptimize$5(SqlQueryExecution.java:586)
        at com.facebook.presto.common.RuntimeStats.recordWallAndCpuTime(RuntimeStats.java:158)
        at com.facebook.presto.execution.SqlQueryExecution.doCreateLogicalPlanAndOptimize(SqlQueryExecution.java:584)
        at com.facebook.presto.common.RuntimeStats.recordWallAndCpuTime(RuntimeStats.java:158)
        at com.facebook.presto.execution.SqlQueryExecution.createLogicalPlanAndOptimize(SqlQueryExecution.java:555)
        at com.facebook.presto.execution.SqlQueryExecution.start(SqlQueryExecution.java:483)
        at com.facebook.presto.$gen.Presto_null__testversion____20250222_201911_1.run(Unknown Source)
        at com.facebook.presto.execution.SqlQueryManager.createQuery(SqlQueryManager.java:319)
        at com.facebook.presto.dispatcher.LocalDispatchQuery.lambda$startExecution$8(LocalDispatchQuery.java:213)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

Impact

Maintain same behaviour in Iceberg Native catalog as Iceberg Hive catalog.

After Change -

presto> SELECT table_name FROM iceberg.information_schema.tables WHERE table_schema = 'non';
 table_name 
------------
(0 rows)

Test Plan

Test added

Contributor checklist

• Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
• PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
• Documented new properties (with its default value), SQL syntax, functions, or other functionality.
• If release notes are required, they follow the release notes guidelines.
• Adequate tests were added if applicable.
• CI passed.

Release Notes

== NO RELEASE NOTE ==

[hantangwangd]

Although it feels a bit weird, but after doing some tests I found that Iceberg lib's behavior towards listTables for non-existing schema is to return an empty list on the Hive catalog and throw exceptions on the Hadoop catalog.

So do you think it makes sense to keep consistency with Iceberg lib, and fix the problem you described in the logic of the method InformationSchemaMetadata.calculatePrefixesWithSchemaName(...)? As I understand, we could filter the schema specified by the schema column constraint in the results of metadata.listSchemaNames(...). In this way, we will get an empty set for non-existent schema. Or is there anything else we need to consider?

[agrawalreetika]

So do you think it makes sense to keep consistency with Iceberg lib, and fix the problem you described in the logic of the method InformationSchemaMetadata.calculatePrefixesWithSchemaName(...)? As I understand, we could filter the schema specified by the schema column constraint in the results of metadata.listSchemaNames(...). In this way, we will get an empty set for non-existent schema. Or is there anything else we need to consider?

Thanks for your review @hantangwangd as I checked since the non-existing schema is part of the predicate, its part of the domain and returning schema in here which is why metadata.listSchemaNames calls are not going from InformationSchemaMetadata.calculatePrefixesWithSchemaName(...) instead previous code block this is getting executed. Please let me know if I am missing something here

[hantangwangd]

its part of the domain and returning schema in here which is why metadata.listSchemaNames calls are not going from InformationSchemaMetadata.calculatePrefixesWithSchemaName(...) instead previous code block this is getting executed.

Bring for discussing, should we refactor the code like follows? Although this always executes listSchemaNames(...), it does handle the non-existent schemas issue in-place and do not leave the processing logic decision to specific connector. Since some connectors may think they should return an empty list, while others think they should throw an exception.

        Session session = ((FullConnectorSession) connectorSession).getSession();
        List<String> existingSchemas = metadata.listSchemaNames(session, catalogName);

        Optional<Set<String>> schemas = filterString(constraint, SCHEMA_COLUMN_HANDLE);
        if (schemas.isPresent()) {
            return schemas.get().stream()
                    .filter(existingSchemas::contains)
                    .map(schema -> new QualifiedTablePrefix(catalogName, schema))
                    .collect(toImmutableSet());
        }

        return existingSchemas.stream()
                .filter(schema -> !predicate.isPresent() || predicate.get().test(schemaAsFixedValues(schema)))
                .map(schema -> new QualifiedTablePrefix(catalogName, schema))
                .collect(toImmutableSet());

[agrawalreetika]

@jaystarshot Could you please also share your suggestion around handling non-existing schemas here?