Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuration-based microbenchmarks #242

Merged
merged 17 commits into from
Jan 10, 2024
Merged

Configuration-based microbenchmarks #242

merged 17 commits into from
Jan 10, 2024

Conversation

Anilm3
Copy link
Collaborator

@Anilm3 Anilm3 commented Jan 4, 2024

  • Support configuration-based multiple scenarios
  • Remove context destroy fixture from this particular benchmark. This style of benchmark will be an isolated one which perhaps will take advantage of some of the existing utilities, such as the object generator.
  • Add support for cbmf to replace json.
  • Perform multiple runs per scenario with different seeds (configurable).
  • Remove benchcmp and benchmerge as they will now be redundant.

Future work (to be done in a future PR):

  • Reconsider multithreaded tests in order to properly exercise contention, currently only independent fixtures are executed in parallel, but this should be modified to validate the same fixture concurrently. This should also allow for core-pinning of each thread.
  • Add memory usage metrics.
  • Add more scenarios for each of the operators, transformers, exclusion filters, processors, etc.

Related Jira: APPSEC-10679

Anilm3 added 2 commits January 4, 2024 17:46
 * Support configuration-based multiple scenarios
 * Remove context destroy fixture from this particular benchmark
 * Add support for cbmf
@codecov-commenter
Copy link

codecov-commenter commented Jan 4, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9f2205c) 83.28% compared to head (a481aa8) 83.28%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #242   +/-   ##
=======================================
  Coverage   83.28%   83.28%           
=======================================
  Files         106      106           
  Lines        4177     4177           
  Branches     1915     1915           
=======================================
  Hits         3479     3479           
  Misses        279      279           
  Partials      419      419           
Flag Coverage Δ
waf_test_none 83.18% <ø> (ø)
waf_test_sse2 83.25% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Anilm3 Anilm3 marked this pull request as ready for review January 8, 2024 17:02
@Anilm3 Anilm3 requested a review from a team as a code owner January 8, 2024 17:02
Copy link
Collaborator

@estringana estringana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing to highlight is that there are some files with bits of code commented. Not sure if that is on purpose or they should get deleted

estringana
estringana previously approved these changes Jan 9, 2024
@Anilm3 Anilm3 requested a review from cataphract January 10, 2024 09:10
cataphract
cataphract previously approved these changes Jan 10, 2024
benchmark/CMakeLists.txt Outdated Show resolved Hide resolved
benchmark/output_formatter.cpp Show resolved Hide resolved
benchmark/output_formatter.cpp Show resolved Hide resolved
@Anilm3 Anilm3 dismissed stale reviews from cataphract and estringana via a481aa8 January 10, 2024 17:20
@Anilm3 Anilm3 merged commit 8a0d79b into master Jan 10, 2024
23 checks passed
@Anilm3 Anilm3 deleted the anilm3/perf-cbmf-output branch January 10, 2024 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants