Skip to content

feat: Reuse intermediate materialization results #686

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat: Reuse intermediate materialization results #686

wants to merge 4 commits into from

Conversation

krlmlr
Copy link
Member

@krlmlr krlmlr commented Apr 10, 2025 •
edited
Loading

Closes #442.

@krlmlr krlmlr enabled auto-merge (squash) April 10, 2025 05:42
@krlmlr krlmlr disabled auto-merge April 10, 2025 05:43
@krlmlr
Copy link
Member Author

krlmlr commented Apr 10, 2025

Let's see what the benchmark results are here.

@github-actions github-actions bot mentioned this pull request Apr 10, 2025
Closed
@github-actions GitHub Actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 21a09b3 is merged into main:

  • ❗🐌001_tpch_01: 24.7ms -> 26.1ms [+0.73%, +9.95%]
  • ❗🐌001_tpch_02: 69.7ms -> 82.5ms [+17.44%, +19.3%]
  • ❗🐌001_tpch_03: 35.7ms -> 40.9ms [+12.21%, +17.07%]
  • ❗🐌001_tpch_04: 23.6ms -> 25.8ms [+6.04%, +12.97%]
  • ❗🐌001_tpch_05: 62.8ms -> 70.2ms [+9.21%, +14.45%]
  • ❗🐌001_tpch_06: 11.9ms -> 12.7ms [+3.88%, +8.16%]
  • ❗🐌001_tpch_07: 75.5ms -> 85.9ms [+12.76%, +14.64%]
  • ❗🐌001_tpch_08: 96.5ms -> 113ms [+16.15%, +17.67%]
  • ❗🐌001_tpch_09: 72.6ms -> 82.8ms [+12.13%, +15.98%]
  • ❗🐌001_tpch_10: 49.7ms -> 56.5ms [+12.63%, +14.88%]
  • ❗🐌001_tpch_11: 34.5ms -> 41.2ms [+17.35%, +21.76%]
  • ❗🐌001_tpch_12: 22.5ms -> 23.8ms [+3.2%, +8.46%]
  • ❗🐌001_tpch_13: 18.4ms -> 20.4ms [+10.01%, +11.64%]
  • ❗🐌001_tpch_14: 17.6ms -> 18.4ms [+3.29%, +5.11%]
  • ❗🐌001_tpch_15: 32.1ms -> 36.7ms [+13.4%, +15.34%]
  • ❗🐌001_tpch_16: 31.3ms -> 35.4ms [+10.48%, +15.61%]
  • ❗🐌001_tpch_17: 27.9ms -> 31.8ms [+12.28%, +15.71%]
  • ❗🐌001_tpch_18: 23.1ms -> 25.6ms [+8.64%, +12.48%]
  • ❗🐌001_tpch_19: 46.3ms -> 48ms [+1.59%, +5.75%]
  • ❗🐌001_tpch_20: 47.1ms -> 58.8ms [+23.3%, +26.55%]
  • ❗🐌001_tpch_21: 84ms -> 99.5ms [+17%, +19.77%]
  • ❗🐌001_tpch_22: 48.9ms -> 55.8ms [+12.21%, +15.83%]
  • ✔️010_tpch_01: 90.3ms -> 89.2ms [-6.37%, +3.98%]
  • ❗🐌010_tpch_02: 123ms -> 137ms [+8.52%, +14.57%]
  • ❗🐌010_tpch_03: 62.5ms -> 69.7ms [+5.57%, +17.5%]
  • ❗🐌010_tpch_04: 46ms -> 48.3ms [+2.93%, +7.43%]
  • ❗🐌010_tpch_05: 139ms -> 147ms [+5.24%, +6.83%]
  • ✔️010_tpch_06: 34.2ms -> 31.9ms [-17.12%, +3.94%]
  • ❗🐌010_tpch_07: 149ms -> 162ms [+5.91%, +12.28%]
  • ❗🐌010_tpch_08: 118ms -> 134ms [+12.47%, +14.11%]
  • ❗🐌010_tpch_09: 162ms -> 173ms [+6.01%, +7.42%]
  • ❗🐌010_tpch_10: 127ms -> 133ms [+3.86%, +5.87%]
  • ❗🐌010_tpch_11: 44.4ms -> 50.6ms [+11.77%, +15.86%]
  • ✔️010_tpch_12: 55ms -> 56ms [-0.02%, +3.7%]
  • ❗🐌010_tpch_13: 52ms -> 54ms [+1.21%, +6.25%]
  • ✔️010_tpch_14: 36.8ms -> 38.3ms [-1.24%, +9.65%]
  • ✔️010_tpch_15: 59.6ms -> 62.9ms [-0.45%, +11.36%]
  • ❗🐌010_tpch_16: 47.2ms -> 51.4ms [+4.3%, +13.31%]
  • ❗🐌010_tpch_17: 60.8ms -> 66ms [+4.51%, +12.53%]
  • ✔️010_tpch_18: 59.4ms -> 62.7ms [-0.06%, +10.95%]
  • ✔️010_tpch_19: 147ms -> 152ms [-0.51%, +8.02%]
  • ❗🐌010_tpch_20: 75.4ms -> 87.8ms [+10.58%, +22.33%]
  • ❗🐌010_tpch_21: 331ms -> 358ms [+4.2%, +11.98%]
  • ❗🐌010_tpch_22: 62.3ms -> 69.9ms [+9.49%, +14.96%]
  • ✔️100_tpch_01: 355ms -> 353ms [-22.05%, +21.16%]
  • ✔️100_tpch_02: 137ms -> 145ms [-6.36%, +17.59%]
  • ✔️100_tpch_03: 168ms -> 179ms [-2.64%, +15.4%]
  • ✔️100_tpch_04: 156ms -> 157ms [-10.19%, +11.9%]
  • ✔️100_tpch_05: 263ms -> 262ms [-12.47%, +11.38%]
  • ✔️100_tpch_06: 105ms -> 98.6ms [-21.95%, +10.09%]
  • ✔️100_tpch_07: 229ms -> 252ms [-0.17%, +19.85%]
  • ✔️100_tpch_08: 256ms -> 281ms [-5.16%, +24.26%]
  • ✔️100_tpch_09: 320ms -> 313ms [-13.21%, +9.1%]
  • ✔️100_tpch_10: 212ms -> 211ms [-6.66%, +5.23%]
  • ✔️100_tpch_11: 92.9ms -> 93.4ms [-30.21%, +31.38%]
  • ✔️100_tpch_12: 194ms -> 194ms [-14.2%, +14.61%]
  • ✔️100_tpch_13: 342ms -> 334ms [-15.03%, +9.93%]
  • ✔️100_tpch_14: 117ms -> 108ms [-20.31%, +4.91%]
  • ✔️100_tpch_15: 210ms -> 213ms [-3.87%, +6.94%]
  • ✔️100_tpch_16: 131ms -> 136ms [-2.08%, +8.8%]
  • ✔️100_tpch_17: 177ms -> 177ms [-6.2%, +5.86%]
  • ❗🐌100_tpch_18: 204ms -> 212ms [+0.39%, +7.12%]
  • ✔️100_tpch_19: 259ms -> 254ms [-14.62%, +10.66%]
  • ✔️100_tpch_20: 183ms -> 190ms [-17.55%, +25.16%]
  • ❗🐌100_tpch_21: 1.41s -> 1.51s [+3.66%, +11.16%]
  • ✔️100_tpch_22: 158ms -> 158ms [-2.69%, +2%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions GitHub Actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 27992b1 is merged into main:

  • ❗🐌001_tpch_01: 23.6ms -> 24.9ms [+1.11%, +10.15%]
  • ❗🐌001_tpch_02: 68.3ms -> 80.8ms [+17.22%, +19.2%]
  • ❗🐌001_tpch_03: 34.7ms -> 39.6ms [+12.67%, +15.22%]
  • ❗🐌001_tpch_04: 22.3ms -> 25.1ms [+11.77%, +13.26%]
  • ❗🐌001_tpch_05: 59.2ms -> 67ms [+12.3%, +13.95%]
  • ❗🐌001_tpch_06: 11.9ms -> 12.5ms [+2.44%, +7.05%]
  • ❗🐌001_tpch_07: 75.9ms -> 85.1ms [+10.12%, +14.25%]
  • ❗🐌001_tpch_08: 96.1ms -> 112ms [+15.35%, +17.47%]
  • ❗🐌001_tpch_09: 71.6ms -> 81.3ms [+12.9%, +14.17%]
  • ❗🐌001_tpch_10: 50.1ms -> 56.1ms [+10.37%, +13.29%]
  • ❗🐌001_tpch_11: 33.8ms -> 39.9ms [+17.56%, +18.79%]
  • ❗🐌001_tpch_12: 21.6ms -> 23.1ms [+5.66%, +8.02%]
  • ❗🐌001_tpch_13: 18.2ms -> 20ms [+8.62%, +11.07%]
  • ❗🐌001_tpch_14: 17.4ms -> 18ms [+2.42%, +4.33%]
  • ❗🐌001_tpch_15: 31.5ms -> 36ms [+12.59%, +16.02%]
  • ❗🐌001_tpch_16: 30.5ms -> 34.9ms [+13.1%, +15.47%]
  • ❗🐌001_tpch_17: 27.1ms -> 31.1ms [+13.42%, +15.86%]
  • ❗🐌001_tpch_18: 22.8ms -> 25.1ms [+8.44%, +12.2%]
  • ❗🐌001_tpch_19: 46.1ms -> 47.2ms [+1.31%, +3.59%]
  • ❗🐌001_tpch_20: 46.9ms -> 58.6ms [+24.08%, +25.82%]
  • ❗🐌001_tpch_21: 84.4ms -> 101ms [+17.91%, +21.77%]
  • ❗🐌001_tpch_22: 48.7ms -> 55.5ms [+12.35%, +15.31%]
  • ✔️010_tpch_01: 88.3ms -> 92.1ms [-1.22%, +9.98%]
  • ❗🐌010_tpch_02: 124ms -> 138ms [+9.26%, +13.69%]
  • ❗🐌010_tpch_03: 63.1ms -> 68.4ms [+7.24%, +9.43%]
  • ❗🐌010_tpch_04: 46.1ms -> 48.8ms [+1.47%, +9.95%]
  • ❗🐌010_tpch_05: 138ms -> 146ms [+4.65%, +6.94%]
  • ✔️010_tpch_06: 32.8ms -> 33.2ms [-6.21%, +8.67%]
  • ❗🐌010_tpch_07: 150ms -> 159ms [+4.71%, +7.57%]
  • ❗🐌010_tpch_08: 117ms -> 137ms [+12.89%, +20.08%]
  • ❗🐌010_tpch_09: 161ms -> 172ms [+5.19%, +7.37%]
  • ❗🐌010_tpch_10: 127ms -> 134ms [+4.74%, +6.44%]
  • ❗🐌010_tpch_11: 44.6ms -> 50.7ms [+10.86%, +16.3%]
  • ❗🐌010_tpch_12: 55.2ms -> 57.5ms [+0.4%, +7.9%]
  • ❗🐌010_tpch_13: 52.2ms -> 53.8ms [+1.33%, +4.63%]
  • ✔️010_tpch_14: 36.9ms -> 38.6ms [-1.21%, +10.42%]
  • ✔️010_tpch_15: 58.6ms -> 61.9ms [-1.21%, +12.25%]
  • ❗🐌010_tpch_16: 45.4ms -> 50.1ms [+7.71%, +13.04%]
  • ✔️010_tpch_17: 60.5ms -> 63.7ms [-0.65%, +11.11%]
  • ✔️010_tpch_18: 57.8ms -> 57.8ms [-5.38%, +5.64%]
  • ❗🐌010_tpch_19: 141ms -> 144ms [+1.28%, +2.64%]
  • ❗🐌010_tpch_20: 73ms -> 84.8ms [+12.76%, +19.55%]
  • ❗🐌010_tpch_21: 324ms -> 344ms [+3.29%, +8.84%]
  • ❗🐌010_tpch_22: 62.5ms -> 68.1ms [+6.09%, +11.74%]
  • ✔️100_tpch_01: 342ms -> 363ms [-15.87%, +27.91%]
  • ❗🐌100_tpch_02: 128ms -> 145ms [+2.79%, +23.66%]
  • ✔️100_tpch_03: 171ms -> 178ms [-4.63%, +12.85%]
  • ✔️100_tpch_04: 156ms -> 162ms [-9.34%, +17.04%]
  • ✔️100_tpch_05: 245ms -> 261ms [-3.64%, +16.83%]
  • ✔️100_tpch_06: 93.9ms -> 93.3ms [-6.5%, +5.15%]
  • ❗🐌100_tpch_07: 234ms -> 245ms [+3.28%, +6.2%]
  • ✔️100_tpch_08: 264ms -> 267ms [-5.55%, +8.12%]
  • ✔️100_tpch_09: 303ms -> 318ms [-4.2%, +14.12%]
  • ❗🐌100_tpch_10: 198ms -> 217ms [+1.03%, +18.98%]
  • ❗🐌100_tpch_11: 83.1ms -> 91.2ms [+3.66%, +15.73%]
  • 🚀100_tpch_12: 204ms -> 181ms [-16.24%, -5.99%]
  • ✔️100_tpch_13: 338ms -> 334ms [-8.86%, +6.66%]
  • ✔️100_tpch_14: 117ms -> 116ms [-14.32%, +13.34%]
  • ✔️100_tpch_15: 208ms -> 206ms [-7.5%, +5.14%]
  • ✔️100_tpch_16: 132ms -> 134ms [-0.48%, +3.7%]
  • ✔️100_tpch_17: 178ms -> 178ms [-8.05%, +7.79%]
  • ✔️100_tpch_18: 201ms -> 206ms [-6.17%, +10.35%]
  • ✔️100_tpch_19: 261ms -> 281ms [-3.97%, +19.39%]
  • ❗🐌100_tpch_20: 173ms -> 196ms [+1.12%, +25.77%]
  • ✔️100_tpch_21: 1.46s -> 1.47s [-2.24%, +4.71%]
  • ✔️100_tpch_22: 159ms -> 162ms [-12.64%, +16.59%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@krlmlr
Copy link
Member Author

krlmlr commented Apr 11, 2025

This probably should be opt-in. Could be an option because it doesn't affect behavior, only performance tradeoffs.

@github-actions GitHub Actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if eaa39b2 is merged into main:

  • ✔️001_tpch_01: 24.1ms -> 24.9ms [-0.79%, +7.46%]
  • ❗🐌001_tpch_02: 72ms -> 85ms [+16.25%, +20.05%]
  • ❗🐌001_tpch_03: 35.9ms -> 41.3ms [+13.65%, +16.39%]
  • ❗🐌001_tpch_04: 23ms -> 25.8ms [+10.88%, +13.19%]
  • ❗🐌001_tpch_05: 61.4ms -> 68.8ms [+10.64%, +13.33%]
  • ❗🐌001_tpch_06: 12.4ms -> 13ms [+0.69%, +7.8%]
  • ❗🐌001_tpch_07: 77.8ms -> 88.3ms [+12.56%, +14.64%]
  • ❗🐌001_tpch_08: 99.9ms -> 116ms [+14.88%, +17.73%]
  • ❗🐌001_tpch_09: 73.6ms -> 83ms [+11.78%, +13.7%]
  • ❗🐌001_tpch_10: 52.3ms -> 58.7ms [+9.71%, +14.64%]
  • ❗🐌001_tpch_11: 36.2ms -> 42.5ms [+15.73%, +19.03%]
  • ❗🐌001_tpch_12: 24ms -> 25.3ms [+3.21%, +8.14%]
  • ❗🐌001_tpch_13: 19.3ms -> 21.6ms [+9.38%, +13.81%]
  • ❗🐌001_tpch_14: 18.5ms -> 19ms [+0.43%, +5.03%]
  • ❗🐌001_tpch_15: 32.1ms -> 36.4ms [+11.91%, +14.62%]
  • ❗🐌001_tpch_16: 30.7ms -> 35.4ms [+13.49%, +17%]
  • ❗🐌001_tpch_17: 28.5ms -> 31.7ms [+6.03%, +16.47%]
  • ❗🐌001_tpch_18: 23.5ms -> 25.9ms [+8.36%, +12.37%]
  • ❗🐌001_tpch_19: 45.4ms -> 47.2ms [+2.85%, +5.23%]
  • ❗🐌001_tpch_20: 47.3ms -> 58.8ms [+23.81%, +25%]
  • ❗🐌001_tpch_21: 84.6ms -> 101ms [+17.72%, +21.34%]
  • ❗🐌001_tpch_22: 48.2ms -> 54.9ms [+12.87%, +14.69%]
  • ❗🐌010_tpch_01: 87.3ms -> 90.1ms [+0.22%, +6.24%]
  • ❗🐌010_tpch_02: 127ms -> 142ms [+7.6%, +15.89%]
  • ❗🐌010_tpch_03: 65.7ms -> 69.7ms [+4.25%, +8.07%]
  • ❗🐌010_tpch_04: 47.3ms -> 50.4ms [+3.62%, +9.34%]
  • ❗🐌010_tpch_05: 144ms -> 154ms [+5.28%, +8.9%]
  • ✔️010_tpch_06: 34.2ms -> 34.7ms [-5.5%, +8.13%]
  • ❗🐌010_tpch_07: 156ms -> 166ms [+4.37%, +7.99%]
  • ❗🐌010_tpch_08: 121ms -> 140ms [+12.67%, +17.8%]
  • ❗🐌010_tpch_09: 170ms -> 181ms [+4%, +8.39%]
  • ❗🐌010_tpch_10: 129ms -> 133ms [+0.4%, +5.25%]
  • ❗🐌010_tpch_11: 45.5ms -> 51.3ms [+10.05%, +15.81%]
  • ✔️010_tpch_12: 56.6ms -> 60ms [-6.16%, +17.89%]
  • ❗🐌010_tpch_13: 52.4ms -> 55.1ms [+2.15%, +8.06%]
  • ✔️010_tpch_14: 37.9ms -> 38.5ms [-0.26%, +3.52%]
  • ❗🐌010_tpch_15: 60.3ms -> 64.4ms [+0.25%, +13.38%]
  • ❗🐌010_tpch_16: 46.5ms -> 50ms [+4.64%, +10.67%]
  • ❗🐌010_tpch_17: 59.3ms -> 63.2ms [+4.59%, +8.37%]
  • ✔️010_tpch_18: 59.3ms -> 61.4ms [-3.83%, +10.87%]
  • ✔️010_tpch_19: 145ms -> 144ms [-3.05%, +1.7%]
  • ❗🐌010_tpch_20: 73.4ms -> 85ms [+13.72%, +17.83%]
  • ❗🐌010_tpch_21: 336ms -> 347ms [+0.43%, +6.34%]
  • ❗🐌010_tpch_22: 115ms -> 123ms [+5.47%, +8.63%]
  • ✔️100_tpch_01: 377ms -> 372ms [-5.29%, +2.37%]
  • ❗🐌100_tpch_02: 131ms -> 144ms [+8.64%, +11.88%]
  • ✔️100_tpch_03: 183ms -> 179ms [-8.72%, +4.82%]
  • ✔️100_tpch_04: 152ms -> 151ms [-12.73%, +11.46%]
  • ✔️100_tpch_05: 266ms -> 262ms [-18.5%, +15.6%]
  • ✔️100_tpch_06: 99.8ms -> 99.6ms [-10.92%, +10.51%]
  • ✔️100_tpch_07: 225ms -> 239ms [-2.11%, +13.85%]
  • ✔️100_tpch_08: 278ms -> 287ms [-5.2%, +12.12%]
  • ✔️100_tpch_09: 320ms -> 330ms [-11.47%, +17.85%]
  • ✔️100_tpch_10: 215ms -> 215ms [-9.84%, +9.93%]
  • ✔️100_tpch_11: 84.3ms -> 96.9ms [-7.97%, +37.78%]
  • ✔️100_tpch_12: 199ms -> 199ms [-3.77%, +3.76%]
  • ✔️100_tpch_13: 336ms -> 348ms [-5.55%, +12.38%]
  • ✔️100_tpch_14: 116ms -> 118ms [-15.31%, +17.45%]
  • ✔️100_tpch_15: 211ms -> 220ms [-9.24%, +17%]
  • ✔️100_tpch_16: 137ms -> 142ms [-1.31%, +9.54%]
  • ✔️100_tpch_17: 181ms -> 185ms [-9.11%, +13.41%]
  • ✔️100_tpch_18: 213ms -> 217ms [-12.4%, +16.85%]
  • ✔️100_tpch_19: 257ms -> 266ms [-7.8%, +14.64%]
  • ✔️100_tpch_20: 180ms -> 184ms [-3.11%, +8.37%]
  • ✔️100_tpch_21: 1.44s -> 1.46s [-6.3%, +8.52%]
  • ✔️100_tpch_22: 158ms -> 166ms [-7%, +16.81%]

Further explanation regarding interpretation and methodology can be found in the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Reuse already materialized data frames
1 participant
@krlmlr