Merge pull request #803 from pyathena-dev/feat/786-glue-metadata-fallback

Answer throttled metadata requests from Glue in the cursor

Author: laughingman7743

docs/sqlalchemy.md

@@ -102,7 +102,7 @@ A throttling or permission error from a table-metadata lookup never by itself es
 For failed metadata requests, the error response establishes absence only when it is a recognized `EntityNotFoundException`; an unrecognized error is never guessed to mean a missing table.
 The `information_schema` queries described below can still establish absence after a failed request, from the query's result rather than from the error.
 Column reflection and `has_table()` do not retry a table-metadata request that `information_schema` can answer; they read `information_schema.columns` instead, executed without query result reuse, and log a warning.
-That covers a throttled request in any catalog.
+That covers a throttled request in any catalog that the cursor's Glue fallback, described below, does not answer.
 It also covers a `MetadataException` carrying no recognized Glue error envelope, but only outside `AwsDataCatalog`: a federated catalog reports a missing table in its connector's own words, so absence is decided by the query against that catalog rather than by an unrecognized message.
 In `AwsDataCatalog` an unrecognized `MetadataException` still propagates, because Glue does state missing tables and permission failures in a recognized envelope, and `information_schema` filters by Lake Formation instead of failing, so reading it there would report a table the caller cannot see as absent.
 Other error codes listed in the connection's `RetryConfig.exceptions` are still retried on that path, except those; list the wrapped Glue codes instead of `MetadataException`.
@@ -114,6 +114,13 @@ Table comments and table options still come from the metadata API with the confi
 When that request fails, they raise `NoSuchTableError` for a recognized `EntityNotFoundException` and propagate any other error, except that for an unrecognized `MetadataException` outside `AwsDataCatalog` they query `information_schema.columns` and raise `NoSuchTableError` if it has no row for the table.
 The query is skipped when column reflection in the same Inspector has already read the table's columns from `information_schema`.
 The dialect runs its own queries — this fallback and `get_view_definition()` — through the API cursor, whatever `cursor_class` or `unload` setting the connection carries, because it parses those result rows itself.
+
+In `AwsDataCatalog` and S3 Tables catalogs, the cursor answers a throttled metadata request from the AWS Glue Data Catalog, as described in {ref}`usage-table-metadata`.
+Reflection of columns, `has_table()`, table comments, table options, and table, view, and schema names uses it, so these need the Glue permissions listed there.
+A table that Glue reports as missing raises `NoSuchTableError`.
+When the Glue request fails, the metadata request runs again with the configured retries, and the paths above apply to its result.
+Set `glue_metadata_fallback=false` in the connection URL to turn the Glue fallback off.
+
 Athena applies its metadata API rate limits per account, and they are not listed in Service Quotas.
 PyAthena's API retries use exponential backoff with uniform jitter; `RetryConfig` documents the default attempt count and waits.
 PyAthena recognizes Glue error codes in Athena's `MetadataException` service-error envelope and applies `RetryConfig.exceptions` to those codes.

docs/testing.md

@@ -51,6 +51,7 @@ Do not commit credentials or include them in test output shared on a pull reques
 
 The tests need an S3 bucket and prefix for staging and data, an Athena SQL workgroup with an S3 query-result location, and an Athena Spark workgroup with a suitable execution role for Spark tests.
 The local test identity needs the corresponding Athena, S3, and Glue permissions; additional features require their own service permissions.
+The Glue metadata fallback tests call `glue:GetTable`, `glue:GetTables`, and `glue:GetDatabases` directly, including against the S3 Tables catalog; the template below grants them.
 The [test infrastructure template](https://github.com/pyathena-dev/PyAthena/blob/master/cloudformation/github_actions_oidc.yaml) describes the resources and permissions used by project CI.
 Its GitHub OIDC role is not a local credential setup: contributors must configure their own test identity.
 

docs/usage.md

@@ -661,6 +661,33 @@ heuristic behavior may see string values where it previously saw integers or flo
 To restore typed conversion, pass `result_set_type_hints` with the appropriate type
 signatures for the affected columns.
 
+(usage-table-metadata)=
+
+## Table and database metadata
+
+`get_table_metadata()`, `list_table_metadata()`, and `list_databases()` call the Athena metadata API.
+Athena applies its metadata API rate limits per account, and they are not listed in Service Quotas.
+In `AwsDataCatalog` and S3 Tables catalogs (`s3tablescatalog/<table-bucket>`), a throttled request is answered from the AWS Glue Data Catalog instead, and a warning is logged.
+The request goes to Glue on the first throttled response, without waiting for the retry policy.
+Glue throttling that Athena reports inside a `MetadataException` counts as throttled.
+The fallback calls these Glue APIs with the connection's credentials:
+
+| Cursor method | Glue API | IAM action |
+|---|---|---|
+| `get_table_metadata()` | `GetTable` | `glue:GetTable` |
+| `list_table_metadata()` | `GetTables` | `glue:GetTables` |
+| `list_databases()` | `GetDatabases` | `glue:GetDatabases` |
+
+Glue's report that the table does not exist, for `get_table_metadata()`, raises `OperationalError`, as Athena's does.
+If the Glue request fails for any other reason, for example for lack of permission or because the Glue endpoint cannot be reached, a second warning is logged and the Athena request runs again with the retry policy.
+A request that cannot connect to Glue, such as from a network with an Athena VPC endpoint but no route to Glue, or that finds no Glue endpoint for the connection's region and endpoint options, also turns the fallback off for the rest of that connection; a request that cannot connect first waits out the botocore connect timeout and retries of the connection's `config`.
+For `list_table_metadata()` and `list_databases()`, Glue's report that the database or catalog does not exist also returns to the Athena request.
+Requests in other catalogs use the Athena API with the retry policy only.
+
+The connection builds one Glue client on first use from its session, region, and botocore `config`, but not its `endpoint_url`.
+Pass `glue_metadata_fallback=False` to `connect()` to turn the fallback off.
+The Glue request does not carry the connection's workgroup; turn the fallback off where access depends on the workgroup, such as a workgroup enabled for IAM Identity Center.
+
 ## Environment variables
 
 Support [Boto3 environment variables](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#using-environment-variables).