Tracking issues of Developer Experiences

[Xuanwo]

Summary

As the community has complained, Databend is reputed to be slow to compile and has a complex architecture.

We should definitely focus on these areas and identify opportunities for improvement.

Known Problems

• Slow to compile
• Complex architecture

Tasks

TBD, we will divide tasks after the known problems have been investigated.

[Xuanwo]

For slow to compile

The main bottleneck appears to be the databend-query crate itself, which contains all pipelines, functions, and interpreters.

[Xuanwo]

For complex architecture

Databend is relatively simple, consisting only of databend-meta and databend-query. It surprises me to hear that the Databend architecture is considered complex. Perhaps there is something unclear in our documentation?

[Xuanwo]

Some updates:

• Databend previously used z3, which required a considerable amount of time to compile; now we have removed it.
• Databend had to disable incremental compilation due to a bug in nightly Rust, but we have now re-enabled it.

[sundy-li]

cargo build timing

https://pastila.nl/?00011e2f/4f5985bd43b9dd82050a1ae1861e9aa3.html#mSmn4LdOMK8oze5e2wP1oA==

Let's remove some useless crates:

1. | aws-sdk-glue v1.53.0 | 60.7s | 15.5s (25%) | default, rt-tokio, rustls
2. | aws-sdk-s3 v1.43.0 | 35.1s | 7.5s (21%) | d