Improved benchmarking infrastructure (do not merge) #1964
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This makes a bunch of improvements to the benchmarking of cosmwasm-vm, such as running all cache tests with two different contracts as well as upgrading criterion.
It also adds a benchmark to a contract project in which it compared the Wasm performance to the native performance for some compute heavy operations: argon3 and BLS12-318 signature verification. This part is a bit problematic since the schema generator and the benchmarking create compiler isssues together.
A few first results is that for the chosen benchmarks, the Wasm runs 10x-20x slower than native for the given tasks. A reason might be simd optimizations on native or assembly optimizations. More tasks will be needed. The other observation is that wasm-opt can produce results that execute up to 24% faster by using -O4 instead of -Os.