Async optimization fixes + profiling scripts #3201
Closed
+85,377
−602
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…e 3.1.0 with its hotkeypub changes.
…g (such as is the case with BadOrigin) while retaining the list join for non-strings.
…/bump-bt-wallet Bump bittensor-wallet version
…rror-formatter Format Error with string docs
…ect-parrent-init `LoggingMachine` initialization updated to explicitly call both parent constructors
…ding-in-dynamic-info Fixed `moving_price` conversion from `I96F32` to float
…-crv3-logic Add new CRv4 logic
…ix-test-breakages UV Fix
Update changelog + version
…rand-major-version Major version locks drand
Bump version and update CHANGELOG.md
Release v9.12.0
…/update-type-hints-ws-shutdown-timer Update type hint/docstring for ws_shutdown_timer
…rnance-deadcode Commenting on the test until the new governance come
Bumping scalecodec to 1.2.12
Bumping version 9.12.1 + changelog
Release/v9.12.1
* Use uv pip for e2e dependency installation (opentensor#3109) * improve `subtensor.get_liquidity_list` * update `test_liquidity` (non functional) * add storage_keys * fix unit tests * apply default era.period to all extrinsics * apply default era.period to all subtensor extrinsic calls * fix *do tests * replace sn0 owner keys * Allow python 3.14 (opentensor#3122) * 9.12.2: changelog + version (opentensor#3125) changelog + version --------- Co-authored-by: Roman Chkhaidze <[email protected]> Co-authored-by: Roman <[email protected]>
…w-e2e-tests Improve e2e tests workflow
Author
@basfroman Thanks for the link. Could you clarify which section of the document is most relevant to the requested changes, or which specific part of the code I should focus on updating first? I want to make sure I implement the fix correctly. |
Collaborator
entire doc
Pls close this pr |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enhanced Summary: Performance Optimization for Bittensor Async Execution
This Pull Request (PR) delivers critical performance enhancements to the asynchronous execution path within
bittensor/core/axon.py. These targeted changes directly address and resolve significant bottlenecks observed during the handling of high-concurrency requests, leading to improved system responsiveness under load.Key Changes and Technical Details
Optimized Asynchronous Handling
The core logic within
axon.pyhas been refactored to optimize async execution, specifically focusing on reducing overhead and contention during concurrent processing.Comprehensive Profiling Tools
To enable precise measurement and future analysis, the following new scripts are included:
profile_bottleneck.py: A script designed for general performance profiling of the system under typical load.reproduce_async_bottleneck.py: A specialized script that reliably triggers and measures the specific high-concurrency slowdown addressed by this PR.Reproducibility Artifact
The
git_setup_log.txtfile is included to document the initial setup environment, ensuring tests and results can be accurately reproduced.Motivation and Impact
Profiling under load revealed repeated, significant slowdowns when processing concurrent forward calls (RPC workloads).
These optimizations successfully reduce latency and substantially improve throughput for all asynchronous RPC workloads, making the Bittensor Axon more stable and efficient in high-concurrency environments.
Verification and Testing
The performance improvements were thoroughly verified locally:
Next Steps
We welcome detailed feedback and suggestions during the code review to ensure the highest level of performance and code quality.