is delta read supported

[djouallah]

is this supported in chdb ?

https://clickhouse.com/docs/sql-reference/table-functions/deltalake

[wudidapaopao]

Hi @djouallah, deltaLake table function was supported in chDB version 3.3.0. However, in the latest chDB 3.4.0 release, due to an internal upgrade of the ClickHouse version it uses, support for deltaLake has been temporarily affected. For example, in the latest version of ClickHouse, deltaLake table function is no longer supported on macOS, and a new Rust-based library has been introduced for connecting to Delta Lake.

As a result, deltaLake table function is currently unavailable in chDB 3.4.0. We're actively working on resolving this issue and aim to re-enable support for deltaLake table function on Linux platforms in the upcoming 3.4.1 release.

[djouallah]

@wudidapaopao now I am getting

RuntimeError: Code: 46. DB::Exception: Unknown table function deltaLakeLocal. Maybe you meant: ['deltaLakeAzure','deltaLakeS3']. (UNKNOWN_FUNCTION)

[wudidapaopao]

Hi @djouallah, deltaLakeLocal is supported in ClickHouse 25.6. Currently, chDB is using ClickHouse version 25.5, and we plan to upgrade chDB to version 25.6 in the near future.

[Gilbert09]

Hey @wudidapaopao, it looks like chdb still doesnt support deltaLake() in any version after 3.3.0 - although this PR suggests differently #343 ?

I'm getting RuntimeError: Code: 46. DB::Exception: Unknown table function deltaLake. (UNKNOWN_FUNCTION) on 3.4.1 and 3.5.0 - is there another open issue somewhere for this?

[wudidapaopao]

Hi @Gilbert09, on which platform do you use chdb, is it linux-x86?

[Gilbert09]

Hi @Gilbert09, on which platform do you use chdb, is it linux-x86?

@wudidapaopao We're using arm64 on AWS EKS

[wudidapaopao]

@Gilbert09 Currently, the deltaLake function is supported on Linux x86, but not on Linux ARM. The reason is that on the ARM platform, the deltaLake function of the latest version of ClickHouse currently relies on Neon instructions. For compatibility purposes, chdb does not assume that the running platform must support Neon instructions. Therefore, the latest version of chdb does not support the deltaLake function on ARM at present.
We will evaluate whether to support the deltaLake function on the Linux ARM platform as well.

[wudidapaopao]

Hi @Gilbert09, chdb will soon reintroduce support for deltaLake table functions on Linux ARM.
#374

[Gilbert09]

Hi @Gilbert09, chdb will soon reintroduce support for deltaLake table functions on Linux ARM. #374

Great news, thank you very much