OSPP Summer 2022 Discussion

[sundy-li]

You can ask any other questions with new comments.

General

Q1: where is the url site of Summer-ospp projects of databend

https://summer-ospp.ac.cn/#/org/orgdetail/646b9834-3923-4e74-b98b-90afec341705/proid226460200

Adaptive String Hash Table

Q1: How to distinguish one-byte key and two-bytes key in S0 Table

If any key ends with \0, we can fall back to use Hash_Table_L. For other keys without enough bytes, we can fill \0 into fixed size keys.

Q2: What's the usage of KeysRef ? Seems it's pretty unsafe in rust.

It's unsafe, but we ensure it is safe manually. We can take KeysRef as a slice of a large heap object (such as a continuous memory space by memory arena pool)

SQLancer

Q1: The basic implementation of SQLancer is to support PQS ?

Yes, it's better to have NoREC & TLP

[sundy-li]

Here are some suggestions if you want to apply Adaptive String Hash Table

1. Understand SAHA paper but not limited to it. BTW, SAHA has been applied into ClickHouse, you may refer to here. You can also find some ideas to improve hash tables through other papers, eg: photo vectorized Hash Table 4.4 .

2. Implementing SAHA in databend is basic requirement. But it does not mean that all the ideas in SAHA must be implemented, if you prove that some features are trivial or useless, you can ignore them.

3. If you want to optimize performance improvement, be sure to have the proper way to measure it in previous.

4. Competition may be fierce as there is only one student place for each topic, so we love some cool & advanced ideas which we did not though of.

[aucker]

Thanks for the suggestions, they are really helpful

[hanyisong]

可以申请一个服务器来测试看看速度吗？最好是已经部署完环境的，不太确定是我的程序问题还是虚拟机的影响。（速度只有5-6 queries/s）
我另外测试了虚拟机的mysql速度（400-500 queries/s）与我宿主机速度比1：4

Can I apply for a server to test the speed? It is best to have already deployed the environment, and I am not sure whether it is my program problem or the impact of the virtual machine. (speed is only 5-6 queries/s)
I also tested the mysql speed of the virtual machine (400-500 queries/s) with a ratio of 1:4 to the speed of my host machine

[sundy-li]

我们没有对外提供服务器，是不是你用的是debug 模式编译的版本？

[hanyisong]

我从release上下载的压缩包

[hanyisong]

8.0.26-v0.7.159-nightly-a7f2c58-simd(rust-1.64.0-nightly-2022-08-10T16:29:03.797735015Z)

[sundy-li]

有没有复现的SQL 和 测试脚本，我在本地开个虚拟机试试看

[hanyisong]

@sundy-li 很抱歉没有及时回复，测试工具我已经提交到datafuse-extras/sqlancer#1