[QA] Vizro AI dashboard tests improvements #935

l0uden · 2024-12-24T12:57:09Z

Description

rewrote score calculation with numpy lib
added prompt text to report
added anthropic provider for an easy dashboard creation (it failed to build medium and complex dashboard)
added complex prompt to have score less than 1.0

Reference to potential complexity prompts improvements -> #935 (comment)

Notice

I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":
- I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
- I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorized to submit this contribution on behalf of the original creator(s) or their licensees.
- I certify that the use of this contribution as authorized by the Apache 2.0 license does not violate the intellectual property rights of anyone else.
- I have not referenced individuals, products or companies in any commits, directly or indirectly.
- I have not added data or restricted code in any commits, directly or indirectly.

…ests_improvements

for more information, see https://pre-commit.ci

…y/vizro into score_tests_improvements

petar-qb

I like these enhancements Alexey. There are a few comments, but other than that, it's all good

vizro-ai/tests/score/prompts.py

vizro-ai/tests/score/test_dashboard.py

lingyielia

Wow I really like this whole Score tests logic you created! 💯 I think it's ready to be checked while we touch the vizro-ai code, and can help track the performance more easily.

I just have some minor questions and suggestions. Overall it's really cool!

.github/workflows/test-score-vizro-ai.yml

vizro-ai/tests/score/prompts.py

…ests_improvements

for more information, see https://pre-commit.ci

…y/vizro into score_tests_improvements

l0uden · 2025-01-16T17:04:08Z

Huge thanks for all of your comments!
It is ready for review again

.github/workflows/test-score-vizro-ai.yml

vizro-ai/tests/score/prompts.py

vizro-ai/tests/score/test_dashboard.py

lingyielia

Great improvement! 🎉

maxschulz-COL

Did a light review, looks good! I think this is going in the right direction! Left a few small comments. ⭐ 💯

In general I think my main comments are:

Make adding tests as easy as possible (ideally just another prompt + expectation pair, nothing else)
Make adding and removing models etc as easy as possible (ideally just some test parametrization or so)
make the complexity of the dashboard a column in the report, and allow for individual names for newly added tests (such that we can have in the furture smth like, easy_1, easy_2,easy_abc, etc.

Other than that, I think it's exciting!

vizro-ai/hatch.toml

vizro-ai/tests/score/prompts.py

vizro-ai/tests/score/test_dashboard.py

vizro-ai/tests/score/prompts.py

…ests_improvements

…ests_improvements � Conflicts: � vizro-ai/hatch.toml � vizro-ai/pyproject.toml � vizro-ai/tests/e2e/data_classes.py � vizro-ai/tests/e2e/prompts.py � vizro-ai/tests/e2e/pytest.ini � vizro-ai/tests/e2e/test_dashboard.py

…ests_improvements

maxschulz-COL

⭐ Well done ⭐

I think this is much better, and looks much more usable. PLease address the outstanding comments still, but I think its almost there.

Could we add more models in the parametrization?

vizro-ai/tests/e2e/test_dashboard.py

…ests_improvements

l0uden · 2025-02-04T12:23:36Z

⭐ Well done ⭐

I think this is much better, and looks much more usable. PLease address the outstanding comments still, but I think its almost there.

Could we add more models in the parametrization?

Sure, added more

l0uden added 3 commits December 24, 2024 13:55

complex prompt, update report, improvements

98f6f4d

Merge branch 'main' of https://github.com/mckinsey/vizro into score_t…

2e71655

…ests_improvements

changelog

736254e

github-actions bot added the Vizro-AI 🤖 Issue/PR that addresses Vizro-AI package label Dec 24, 2024

pre-commit-ci bot and others added 4 commits December 24, 2024 12:57

[pre-commit.ci] auto fixes from pre-commit.com hooks

99d0fd7

for more information, see https://pre-commit.ci

add anthropic creds

34a7ccf

Merge branch 'score_tests_improvements' of https://github.com/mckinse…

d96a6e6

…y/vizro into score_tests_improvements

fix report aggregated

7139fb5

l0uden marked this pull request as ready for review December 27, 2024 11:58

l0uden requested review from Joseph-Perkins, antonymilne, huong-li-nguyen, maxschulz-COL and lingyielia as code owners December 27, 2024 11:58

petar-qb assigned l0uden Jan 7, 2025

petar-qb approved these changes Jan 7, 2025

View reviewed changes

lingyielia requested changes Jan 9, 2025

View reviewed changes

.github/workflows/test-score-vizro-ai.yml Outdated Show resolved Hide resolved

.github/workflows/test-score-vizro-ai.yml Outdated Show resolved Hide resolved

vizro-ai/tests/score/prompts.py Outdated Show resolved Hide resolved

l0uden and others added 8 commits January 16, 2025 17:20

complex prompt and review fixes

459f482

Merge branch 'main' of https://github.com/mckinsey/vizro into score_t…

c11fcb4

…ests_improvements

[pre-commit.ci] auto fixes from pre-commit.com hooks

e60a104

for more information, see https://pre-commit.ci

added reruns

9be7bd8

Merge branch 'score_tests_improvements' of https://github.com/mckinse…

72099a4

…y/vizro into score_tests_improvements

change complex prompt

dafbb16

change complex prompt

06c239a

option for ',' separator in aggregated report

97a1019

petar-qb reviewed Jan 17, 2025

View reviewed changes

review changes

ff97b31

lingyielia self-requested a review January 17, 2025 18:07

lingyielia approved these changes Jan 17, 2025

View reviewed changes

maxschulz-COL approved these changes Jan 20, 2025

View reviewed changes

l0uden added 6 commits January 20, 2025 13:19

Merge branch 'main' of https://github.com/mckinsey/vizro into score_t…

88f77eb

…ests_improvements

created one parametrized test

b1ae1b6

Merge branch 'main' of https://github.com/mckinsey/vizro into score_t…

c6a0cd0

…ests_improvements

separate anthropic test

c52f366

Merge branch 'main' of https://github.com/mckinsey/vizro into score_t…

ca19250

…ests_improvements

l0uden changed the title ~~[QA] Score tests improvements~~ [QA] Vizro AI dashboard tests improvements Feb 3, 2025

maxschulz-COL approved these changes Feb 4, 2025

View reviewed changes

vizro-ai/tests/e2e/test_dashboard.py Show resolved Hide resolved

l0uden added 2 commits February 4, 2025 13:04

added more models

cc943df

Merge branch 'main' of https://github.com/mckinsey/vizro into score_t…

b640aa8

…ests_improvements

l0uden and others added 2 commits February 4, 2025 13:51

comments. deleted local vizro run

d72326e

Merge branch 'main' into score_tests_improvements

ad9095d

l0uden merged commit 2426ef6 into main Feb 4, 2025
39 checks passed

l0uden deleted the score_tests_improvements branch February 4, 2025 14:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[QA] Vizro AI dashboard tests improvements #935

[QA] Vizro AI dashboard tests improvements #935

l0uden commented Dec 24, 2024 •

edited by petar-qb

Loading

petar-qb left a comment

lingyielia left a comment

l0uden commented Jan 16, 2025

lingyielia left a comment

maxschulz-COL left a comment

maxschulz-COL left a comment

l0uden commented Feb 4, 2025

[QA] Vizro AI dashboard tests improvements #935

[QA] Vizro AI dashboard tests improvements #935

Conversation

l0uden commented Dec 24, 2024 • edited by petar-qb Loading

Description

Notice

petar-qb left a comment

Choose a reason for hiding this comment

lingyielia left a comment

Choose a reason for hiding this comment

l0uden commented Jan 16, 2025

lingyielia left a comment

Choose a reason for hiding this comment

maxschulz-COL left a comment

Choose a reason for hiding this comment

maxschulz-COL left a comment

Choose a reason for hiding this comment

l0uden commented Feb 4, 2025

l0uden commented Dec 24, 2024 •

edited by petar-qb

Loading