Skip to content

Add support for Glue 5.0 #170

@nwheeler81

Description

@nwheeler81

AWS Glue is a serverless, scalable data integration service that makes it simple to discover, prepare, move, and integrate data from multiple sources. AWS Glue 5.0 upgrades the engines to Apache Spark 3.5.2, Python 3.11, and Java 17, with new performance and security improvements. Glue 5.0 updates open table format support to Apache Hudi 0.15.0, Apache Iceberg 1.6.1, and Delta Lake 3.2.0 so you can solve advanced use cases around performance, cost, governance, and privacy in your data lakes. AWS Glue 5.0 adds Spark native fine grained access control with AWS Lake Formation so you can apply table, column, row, and cell level permissions on Amazon S3 data lakes. Finally, Glue 5.0 adds support for Sagemaker Lakehouse to unify all your data across Amazon S3 data lakes and Amazon Redshift data warehouses.

Java 17 updates
[Scala] Upgrade AWS SDK calls from v1 to v2

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions