Update dependency deltalake to v1

[renovate]

This PR contains the following updates:

Package	Change	Age	Confidence
deltalake	<1.0.0,>=0.16.0 → <1.6.0,>=1.5.0

---

Warning

Some dependencies could not be looked up. Check the Dependency Dashboard for more information.

---

Release Notes

delta-io/delta-rs (deltalake)

v1.0.2

Full Changelog

⚠️ There is a known performance regression when opening very wide tables (50+ columns) that have hundreds of thousands of transactions. The fix is pending a new delta-kernel-rs release.

Implemented enhancements:

• Python: Automatically convert Pandas null types to valid Delta Lake types in write_deltalake() #​3691
• Update HDFS object store to 0.15 #​3680
• create a v2 uuid checkpoint regression test #​3666
• Feature: update python table vacuum to add keep_versions parameter #​3634
• TypeError in DeltaTable.to_pyarrow_dataset when using non-string partition filter values (e.g., int) #​3597
• Make "cloud" feature optional #​3589
• convert_to_deltalake cannot convert parquet dataset if it has millisecond-precision timestamps #​3535
• Musl wheels #​3399

Fixed bugs:

• Automatically register the AWS, Azure, GCS, HDFS, LakeFS, and Unity storage handlers when the corresponding feature is enabled so DeltaOps::try_from_uri no longer errors with unknown schemes such as gs://.
• Significant performance regression when opening S3 table on next branch #​3667
• Concurrent overwrite doesn't fail conflict checking #​3622
• source distributions missing in v1.1.1 #​3621
• Missing linux distro for v1.1.1 #​3620
• azurite tets failing in main #​3612
• Generic S3 Error on _last_checkpoint on ARM64 AWS Lambda with write_deltalake #​3602
• write_deltalake merge with list and large_list #​3595
• Python DeltaTable does not support writes in multiple threads (again?) #​3594
• Checkpoint creation fails on Azure in >=1.0.0 with "Azure does not support suffix range requests" #​3593
• Partition value strings containing reserved ASCII and non-ASCII are double-encoded. #​3577
• Deltalake version 1.0.2 errors with Azure Storage after appending many times #​3567
• Python deltalake 1.0.2 is not compatible with polars.Array #​3566
• Checkpoint schema breaking change between 0.25.5 and 1.0.2 #​3527

Closed issues:

• Array/list not encoded with partition filters #​3648

Merged pull requests:

• fix: reintroduce the 100 commit checkpoint interval #​3708 (rtyler)
• fix: enabling correctly pulling partition values out of column mapped tables #​3706 (rtyler)
• fix(format): fix formatting in Python for conversion file #​3705 (fvaleye)
• chore: remove unused dependencies #​3698 (rtyler)
• fix(pandas): implement-automatic-conversion-for-pandas-null-types #​3695 (fvaleye)
• chore: update hdfs object store to 0.15 #​3681 (Kimahriman)
• feat!: use kernel predicates on file streams #​3669 (roeap)
• chore: bump python #​3664 (ion-elgreco)
• fix: use RFC3896 percent encoding with delta protocol correctness #​3661 (ion-elgreco)
• feat!: kernel log replay #​3660 (roeap)
• ci: run integration tests against next branches #​3658 (roeap)
• fix: handle checking partition filters in array/list when converting … #​3657 (smeyerre)
• fix: aws special paths encoding #​3656 (roeap)
• feat: support converting parquet with non-microsecond timestamps to d… #​3654 (smeyerre)
• chore: use pytest-xdist for speeding up python tests #​3642 (rtyler)
• chore: remove deprecated use of kernel's Table #​3639 (rtyler)
• feat: add keep_versions parameter to vacuum command for python #​3635 (corwinjoy)
• chore: bump version for release #​3633 (rtyler)
• fix: avoid parsing generationExpressions as JSON #​3632 (rtyler)
• feat: build musl wheels upon release #​3631 (rtyler)
• fix: make the docs link checking more useful/less faily #​3630 (rtyler)
• fix: coerce polars.Array into a suitable Arrow list type #​3623 (rtyler)
• fix: ensure openssl-sys doesn't creep into the dependency via the kernel default engine #​3619 (rtyler)
• fix: allow writing to DeltaTable objects across Python threads #​3618 (rtyler)
• docs: fix broken daft links (how daft) #​3617 (rtyler)
• fix: ensure new checkpoints can be written after old checkpoints #​3616 (rtyler)
• fix: switch the url schemes for Azure integration tests #​3614 (rtyler)
• fix: allow non-string primitive types for partition filters when converting to pyarrow dataset #​3613 (smeyerre)
• refactor: make match_partitions and new_metadata public #​3605 (zeevm)
• fix: fix typo to fix CI typo check #​3604 (alamb)
• chore: update to DataFusion 49.0.0 #​3603 (alamb)
• chore: minor API changes after integration testing #​3598 (rtyler)
• fix: scan time was always 0 for merge metrics #​3596 (rtyler)
• refactor: make "cloud" feature in object_store optional #​3590 (zeevm)
• chore: generate a more recentish updated changelog #​3588 (rtyler)
• fix: creating new DeltaTable with invalid table name path no longer creates empty directory #​3504 (smeyerre)

v0.25.4

Full Changelog

Implemented enhancements:

• Make "cloud" feature optional #​3397
• Delta Column mapping feature #​3358
• Allow choosing kernel built-in engine #​3334
• Don't enable "cloud" feature on object_store by default #​3331
• Cannot use latest delta-rs with pyarrow 19.0.1 #​3323
• Support for Parquet Bloom Filters at RowGroup Level #​3322

Fixed bugs:

• Timezone information in schema dropped and converted to UTC when writing PyArrow table #​3402
• Got "Generic DeltaTable error: type_coercion" when updating the deltatable #​3400
• Removing pinned chrono version #​3391
• CDF changes fully loaded into memory #​3388
• Unable to use write_deltalake on nullable columns with mode = "overwrite" #​3387
• uv add deltalake fails without pinning to 0.25.4 #​3364
• Unable to use Arrow UUID type with DeltaLake and schema().to_pyarrow(), Python Exception raised. #​3363
• Writer Incompatibility Issue Between Delta Lake Protocol Version and Rust Writer #​3356
• Restore not readable in Synapse/Databricks #​3354
• Restore is not rolling back schema changes #​3352
• Schema Mode merge with subfields 'struct fields don't match' #​3350
• Error resolving renamed partition columns #​3348
• DeltaError on attempting merge operation involving map field #​3340
• MaxCommitAttempts error due to stale snapshot used during OPTIMIZE #​3337
• _internal.DeltaError: Failed to parse parquet: Parquet error: Z-order failed while scanning data #​3327
• Error writing delta table with Null columns #​3316
• Error in writing panda data frame with pyarrow engine to DeltaTable #​3315
• DeltaTable.is_deltatable sometimes return false for valid deltatable #​3314
• Runtime panic in new streaming writer: Cannot start a runtime from within a runtime #​3271
• Build failure due to conflicting strum versions in deltalake-core dependencies #​3267

Closed issues:

• How to read delta table in azure databricks unity catalog #​3276
• Feature request: provide capability for pass in timestamp value when dump delta log #​3258

v0.25.0

Full Changelog

Implemented enhancements:

• Configurable column encoding for parquet checkpoint files to address Fabric limitation #​3212

Fixed bugs:

• Writing from Windows host to Ubuntu WSL virtual machine doesn't succeed #​3307
• DeltaTable(path) errors out for unknown data types #​3305
• Commit Properties / Custom Metadata not loaded #​3304
• peek_next_commit() panics on invalid json #​3297
• cargo build fail on macOS 15.3.1 #​3295
• Error when writing polars dataframe with enum or categorical columns #​3284
• Schema evolution causing table ID to be regenerated, breaks Spark streaming jobs #​3274
• is_deltatable creates the path if it doesn't exist #​3259
• is_deltatable throwing S3 error in 0.25.1 on linux aarch64 build #​3241
• Datafusion error: External error: Failed to get a credential from UnityCatalog client configuration. #​3236
• deltalake 0.24.0 cannot compile with E0308 error #​3235
• 0.25.0 can't be pip installed on linux #​3234
• Azure blob - trying to fetch _delta_log for large table exception: Failed to parse parquet: External: Generic MicrosoftAzure error: error decoding response body #​3232
• Trying to open a DeltaTable at a non-existent path creates the path #​3228
• don't write deletion vector entry in the log #​3211

Closed issues:

• Memory leak on 0.25.x #​3306

Merged pull requests:

• fix: serialize empty deletionVector in add actions as absent #​3309 (rtyler)
• fix: prevent panics when peek_next_commit() encounters invalid data #​3308 (rtyler)
• fix(pandas): retain pyarrow decimal datatype in to_pandas() by adding types_mapper to prevent precision loss #​3296 (Abhishek1005)
• chore: bump python version for release #​3291 (rtyler)
• feat: remove optimize operations when building without Apache Datafusion #​3290 (rtyler)
• chore: upgrade the kernel version and bump our majorish versions too #​3289 (rtyler)
• fix: timestamp truncation in stats parsed #​3288 (ion-elgreco)
• refactor: drop pyarrow support, restructure python modules #​3285 (ion-elgreco)
• fix: compare before after schema before creating action #​3282 (ion-elgreco)
• refactor: simplify expressions #​3281 (ion-elgreco)
• chore: remove unused dev dependency on home #​3277 (alamb)
• fix(rust): on write, have a schema evolution maintain metadata table id #​3275 (liamphmurphy)
• fix: chrono 0.4.40 causes disambiguation syntax when building arrow deps #​3272 (Nordalf)
• refactor: replaced asterisk with constraint name in get_constraints #​3270 (Nordalf)
• fix: commitlint config #​3268 (roeap)
• chore: use builder API to create FileScanConfig #​3266 (alamb)
• chore: update tests to use Column::new and other expr_fn functions #​3265 (alamb)
• chore: fix clippy warnings on main #​3264 (alamb)
• fix: aarch64 ssl handshakes #​3263 (ion-elgreco)
• chore: fix some compiler warnings #​3262 (alamb)
• chore: upgrade to DataFusion 46.0.0 #​3261 (alamb)
• fix: enable missing cloud features for uc crate #​3260 (omkar-foss)
• docs: update CONTRIBUTING.md and Makefile to account for switch to uv #​3257 (adamreeve)
• refactor: async writer + multi-part #​3255 (ion-elgreco)
• fix: refresh snapshot after vacuuming logs #​3252 (ion-elgreco)
• chore: more manual inline format args #​3251 (nyurik)
• chore: update readme for Generated columns #​3247 (ion-elgreco)
• feat: set clientVersion for python binding #​3242 (ion-elgreco)
• chore: bump the patch for a new release of python #​3240 (rtyler)
• chore: use rustls #​3239 (ion-elgreco)
• fix: use tableprovider scan in z-order #​3238 (ion-elgreco)
• chore: bump python 0.25 #​3233 (ion-elgreco)
• chore: switch to cache@​v4 #​3230 (rtyler)
• feat: streamed execution, writer refactor, simplified generated columns and schema evolution #​3229 (ion-elgreco)
• fix(typo): s/implemtnation/implementation/ #​3227 (akesling)
• feat: cdf tableprovider with predicate pushdown support #​3220 (ion-elgreco)
• fix(rust, python): treat FSCK files_removed as strings #​3219 (liamphmurphy)
• fix: load cdf latest version #​3218 (ion-elgreco)
• feat: make Add:get_stats public #​3216 (jkylling)
• feat: configurable column encoding for parquet checkpoint files #​3214 (dmunch)
• chore: use flags for apple arm64 #​3213 (ion-elgreco)
• fix: only use stats for required cols #​3210 (ion-elgreco)
• fix: adhere write stats configuration #​3209 (ion-elgreco)

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.