Skip to content
This repository was archived by the owner on Jul 3, 2023. It is now read-only.

Web UI initial import #358

Merged
merged 30 commits into from
May 1, 2023
Merged

Web UI initial import #358

merged 30 commits into from
May 1, 2023

Conversation

gz
Copy link
Contributor

@gz gz commented Apr 9, 2023 •
edited
Loading

TODO before check-in:

  • Fix Editor Race
  • Simplify EntityTable Menu
  • Editable Connectors
  • Add CI
  • Error handling for builder
  • Put Instructions in README
  • Improve documentation

@codecov
Copy link

codecov bot commented Apr 10, 2023 •
edited
Loading

Codecov Report

Merging #358 (5cb1b0c) into main (dbe4663) will increase coverage by 0.15%.
The diff coverage is 70.09%.

❗ Current head 5cb1b0c differs from pull request most recent head a2bbcae. Consider uploading reports for the commit a2bbcae to get more accurate results

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #358      +/-   ##
==========================================
+ Coverage   72.99%   73.14%   +0.15%     
==========================================
  Files         237      237              
  Lines       49568    50146     +578     
==========================================
+ Hits        36180    36680     +500     
- Misses      13388    13466      +78
Impacted Files Coverage Δ
crates/dataflow-jit/src/codegen/intrinsics.rs 61.60% <0.00%> (-8.93%) ⬇️
crates/dataflow-jit/src/dataflow/mod.rs 49.88% <0.00%> (-10.08%) ⬇️
crates/dataflow-jit/src/ir/function/passes.rs 52.57% <ø> (ø)
crates/dataflow-jit/src/ir/ids.rs 61.29% <0.00%> (-2.05%) ⬇️
crates/dataflow-jit/src/ir/nodes/aggregate.rs 10.48% <ø> (-0.40%) ⬇️
crates/dataflow-jit/src/ir/nodes/constant.rs 47.74% <ø> (+1.25%) ⬆️
crates/dataflow-jit/src/ir/nodes/differentiate.rs 17.18% <ø> (+1.47%) ⬆️
crates/dataflow-jit/src/ir/nodes/filter_map.rs 41.62% <ø> (+0.32%) ⬆️
crates/dataflow-jit/src/ir/nodes/flat_map.rs 81.13% <ø> (-1.02%) ⬇️
crates/dataflow-jit/src/ir/nodes/index.rs 57.57% <ø> (-1.85%) ⬇️
... and 41 more

... and 3 files with indirect coverage changes

@github-actions
Copy link

github-actions bot commented Apr 10, 2023 •
edited
Loading

Benchmark results

Nexmark

  • 2 out of 21 queries have regressed ❗
  • Compared results from dbe4663 (main) with 5cb1b0c (PR)
    No benchmark results found for current main revision, compared against dbe4663
name main~26 [kOp/s] PR [kOp/s] Tput change [%] Assessment Peak RSS diff
q0 5262.63 4266.87 -19 🔻 20.1 MB
q1 5429.6 5457.16 1 ✔️ 12.1 MB
q2 5361.6 5358.9 0 ✔️ 4.4 MB
q3 5321.94 5455.31 3 ✔️ 47.9 MB
q4 4802.55 4453.2 -7 🔻 -380.6 MB
q5 5378.33 5372.31 0 ✔️ -380.6 MB
q6 4880.13 4850.02 -1 ✔️ -314.7 MB
q7 2559.31 4194.39 64 🌲 -6.1 GB
q8 5161.2 5088.75 -1 ✔️ -6.1 GB
q9 875.448 866.817 -1 ✔️ 482.7 MB
q12 4910.28 5046.03 3 ✔️ 482.7 MB
q13 3645.17 3560.34 -2 ✔️ 482.7 MB
q14 5481.43 5509.23 1 ✔️ 482.7 MB
q15 4895.11 5325.83 9 🌲 482.7 MB
q16 1045.16 1040.05 0 ✔️ 482.7 MB
q17 3134.56 3182.34 2 ✔️ -284.8 MB
q18 1440.13 1466.79 2 ✔️ 467.5 MB
q19 1435.15 1419.09 -1 ✔️ 467.5 MB
q20 1500.83 1537.87 2 ✔️ 467.5 MB
q21 5117.36 5220.91 2 ✔️ 467.5 MB
q22 5356.59 5398.67 1 ✔️ 467.5 MB

Galen

  • Compared results from dbe4663 (main) with 5cb1b0c (PR)
    No benchmark results found for current main revision, compared against dbe4663
name main~26 [s] PR [s] Runtime change [%] Assessment
galen 28.6952 28.6648 0 ✔️

LDBC

  • 1 out of 4 queries have regressed ❗
  • Compared results from dbe4663 (main) with 5cb1b0c (PR)
    No benchmark results found for current main revision, compared against dbe4663
algorithm dataset threads main~26 [kEVPS] PR [kEVPS] Tput change [%] Assessment Peak RSS diff
bfs graph500-22 1 1835.7 1775.24 -3 ✔️ -86.0 kB
bfs datagen-8_4-fb 6 8173.61 7658.89 -6 🔻 41.9 MB
pagerank graph500-22 1 681.655 692.045 2 ✔️ -118.8 kB
pagerank datagen-8_4-fb 6 1994.79 2030.99 2 ✔️ -150.4 MB

Nexmark (with Persistence)

  • Compared results from dbe4663 (main) with 5cb1b0c (PR)
    No benchmark results found for current main revision, compared against dbe4663
name main~26 [kOp/s] PR [kOp/s] Tput change [%] PR DRAM [kOp/s] DRAM diff [%] Assessment
q0 2399.23 2444.9 2 2399.47 2 ✔️
q1 1713.93 1709.85 0 1666.91 3 ✔️
q2 2407.67 2429.89 1 2336.92 4 ✔️
q3 2036.93 2073.46 2 2260.16 -8 ✔️
q4 360.256 362.387 1 1378.43 -74 ✔️
q5 2015.3 2030.62 1 2242.61 -9 ✔️
q6 332.421 339.702 2 1377.67 -75 ✔️
q7 639.584 665.907 4 1310.45 -49 ✔️
q8 2171.19 2283.95 5 2196.86 4 🌲
q9 79.0764 80.4758 2 385.606 -79 ✔️
q12 838.481 880.186 5 1790.59 -51 ✔️
q13 443.491 442.99 0 998.11 -56 ✔️
q14 1697.52 1682.13 -1 1614.97 4 ✔️
q15 198.112 198.22 0 1178.96 -83 ✔️
q16 25.5771 25.7467 1 285.279 -91 ✔️
q17 81.4521 81.5963 0 786.609 -90 ✔️
q18 124.763 125.325 0 784.663 -84 ✔️
q19 182.489 180.629 -1 649.054 -72 ✔️
q20 505.761 523.408 3 917.406 -43 ✔️
q21 1489.8 1435.29 -4 1484.57 -3 ✔️
q22 2102.89 2017.96 -4 2023.69 0 ✔️

@gz gz force-pushed the webui branch 3 times, most recently from 2f16e7d to 36c5fb5 Compare April 13, 2023 04:06
@gz gz temporarily deployed to github-pages April 13, 2023 06:28 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 13, 2023 06:31 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 13, 2023 08:43 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 13, 2023 08:56 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 13, 2023 08:59 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 14, 2023 08:41 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 14, 2023 08:45 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 14, 2023 08:55 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 14, 2023 08:57 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 14, 2023 09:32 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 15, 2023 00:47 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 15, 2023 01:04 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 15, 2023 01:07 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 15, 2023 01:46 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 15, 2023 01:49 — with GitHub Actions Inactive
@gz gz requested a review from mihaibudiu April 15, 2023 01:54
@gz gz marked this pull request as ready for review April 15, 2023 01:54
@gz gz temporarily deployed to github-pages April 15, 2023 01:56 — with GitHub Actions Inactive
@gz
Copy link
Contributor Author

gz commented Apr 15, 2023

@mbudiu-vmw sorry for the huge PR

I suggest to focus reviewing to most relevant folders with core-logic:

  • analytics
  • pages
  • streaming
  • connectors

@gz gz temporarily deployed to github-pages April 15, 2023 02:01 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 15, 2023 02:22 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 15, 2023 02:25 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 15, 2023 02:32 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages April 15, 2023 02:38 — with GitHub Actions Inactive
gz added 17 commits April 30, 2023 20:34
@gz
Refactoring of editor code.
151c827
@gz
Refactoring / Comments / Upate gitignore.
83e49a5
@gz
Document / refactor utility functions.
8bbf1b9
@gz
Edit Connector function.
9e27776
@gz
Add CI for website.
35a784b
@gz
Add generic connector config dialog.
fff533c
@gz
Comments for the builder.
1d8ccef 
Signed-off-by: Gerd Zellweger <[email protected]>
@gz
Dialog to complain about missing program schema.
688cc15
@gz
Have a default function to dispatch network requests.
00ae588
@gz
Add onError handler in queries where its missing.
07a88c2
@gz
Basic Error handling when SQL program tables/viewnames change.
6000d1b 
In this case we might not always be able connect all Connectors
them in the config anymore.
@gz
Delete unused file.
4c22d28
@gz
Avoid doing the import comments in all files.
4c7c3e1
@gz
Fix pipeline editor.
5d45d9e 
Fetching the connector data on-demand while adding a node
breaks the layout as we only know the size once it's too late
(e.g., position has been determined already).
We do know all the ConnectorDescr data when adding a new one
so we just generate it instead of fetching it on-demand.
@gz
Uodate line length.
2b1e8c1
@gz
No trees or people images.
7435d85
@gz
Refactor and document code.
7c0f957
@gz gz temporarily deployed to github-pages May 1, 2023 03:39 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages May 1, 2023 04:13 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages May 1, 2023 16:12 — with GitHub Actions Inactive
@gz gz temporarily deployed to github-pages May 1, 2023 16:18 — with GitHub Actions Inactive
@ryzhyk ryzhyk merged commit 34ea1f3 into main May 1, 2023
@ryzhyk ryzhyk deleted the webui branch May 1, 2023 17:23
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mihaibudiu mihaibudiu mihaibudiu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gz @mihaibudiu @ryzhyk