Add Lance connector

[luohao]

Description

Initial Lance connector implementation. It adds 2 modules:

• trino-lance-file: java implementation of the Lance file format(reader for latest version 2.1 in this PR). Also included round trip tests to validate the Lance file reader with JNI writer provided by lancedb.
• trino-lance: java implementation of Lance table format. It's a basic connector implementation with read only capability. Added basic connector tests with TPCH data to cover e2e tests.

Additional context and related issues

Benchmark results show that this implementation is up to 5x faster than other implementation using the JNI lib. The following results are collected from a Macbook Pro with Apple M2 Max CPU. Some notes about the result:

• I have implemented vectorized reader for the fastlane encoding(https://github.com/luohao/fastlanes-java) for integers. The performance is on-par with the rust implementation. I believe the saving mostly come from eliminating all unnecessary overhead, including JNI overhead, memory copy and Arrow to Trino Block conversion.
• The readVarchar is worse than JNI version because the my current FSST decoder doesn't have any optimizations(e.g., vectorization, loop unrolling, and reduced memory copy). I have a TODO for this.

Benchmark                              Mode  Cnt    Score    Error  Units
BenchmarkColumnReaders.readBigInt      avgt   60    7.700 ±  0.196  ns/op
BenchmarkColumnReaders.readBigIntJNI   avgt   60    9.684 ±  0.251  ns/op
BenchmarkColumnReaders.readList        avgt   60  147.698 ±  1.692  ns/op
BenchmarkColumnReaders.readListJNI     avgt   60  396.676 ± 15.791  ns/op
BenchmarkColumnReaders.readStruct      avgt   60   21.180 ±  0.345  ns/op
BenchmarkColumnReaders.readStructJNI   avgt   60  110.885 ±  0.581  ns/op
BenchmarkColumnReaders.readVarchar     avgt   60  131.577 ±  1.683  ns/op
BenchmarkColumnReaders.readVarcharJNI  avgt   60   37.094 ±  0.228  ns/op

More details about the design and implementation of Lance file/table format can be found at

• VLDB paper: https://github.com/lancedb/lance-research/blob/main/file_2_1/paper/paper.pdf
• rust implementation: https://github.com/lancedb/lance

This is my first PR to kick off the Lance connector work. My immediate TODO list(aka Roadmap) already includes:

• Lance File Format
  • Enhance Reader
    • Support FixedSizeList
    • Support FullZip encoding
  • Implement Writer
• Lance Table Format
  • Support multiple data files per fragment
  • Support delete files
  • Support index files
  • Support other catalogs

And more to be added...

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)

[sourcery-ai]

Sorry @luohao, your pull request is larger than the review limit of 150000 diff characters

[Praveen2112]

Any specific reason for this TempFile.class ? Files.createTempFile we can use them to create the files on demand right ?

[luohao]

I am following the pattern in Orc and Parquet(e.g., https://github.com/trinodb/trino/blob/master/lib/trino-orc/src/test/java/io/trino/orc/TempFile.java, https://github.com/trinodb/trino/blob/master/plugin/trino-hive/src/test/java/io/trino/plugin/hive/parquet/ParquetTester.java#L556).

I think using Files.createTempFile works too.

[ebyhr]

We should test a SELECT statement.

[luohao]

The connector is read only right now, adding SELECT would need additional stuff to set up the data.

Do you have an example that prepares data for a read only connector?

[ebyhr]

We add static resource files or use the datasource's library in such a case.

[luohao]

I tried to add files to minio object store via copying files to minio container's /data/test-bucket/ directory but doesn't seem to work.

Do you know any example of copying directories to object store?

[ebyhr]

(Answered in Slack DM)