Skip to content

[Feature] Aggregation strategies #3209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: gh/vmoens/164/base
Choose a base branch
from
Open

[Feature] Aggregation strategies #3209

wants to merge 6 commits into from
+41 −7

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 16, 2025 •
edited
Loading

@vmoens
Update
dc9dfcb 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 16, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3209

Note: Links to docs will display an error until the docs builds have been completed.

❌ 11 New Failures, 4 Unrelated Failures

As of commit 553d1e9 with merge base 13434eb (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Oct 16, 2025
@vmoens
[Feature] Aggregation strategies
dd4ee78 
ghstack-source-id: 935815c
Pull-Request: #3209
This was referenced Oct 16, 2025
Merged
Merged
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 16, 2025
This was referenced Oct 16, 2025
Open
Open
@vmoens
Update
aac6063 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Oct 19, 2025
@vmoens
[Feature] Aggregation strategies
2292903 
ghstack-source-id: b3b9a86
Pull-Request: #3209
vmoens added a commit that referenced this pull request Oct 20, 2025
@vmoens
[Feature] Aggregation strategies
d1e6fb7 
ghstack-source-id: b3b9a86
Pull-Request: #3209
@vmoens vmoens mentioned this pull request Oct 23, 2025
Merged
@vmoens
Update
3907285 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 23, 2025
Open
@github-actions
Copy link

github-actions bot commented Oct 23, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}14$. Worsened: $\large\color{#d91a1a}11$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 84.0520μs 82.0747μs 12.1840 KOps/s 12.1540 KOps/s $\color{#35bf28}+0.25\%$
test_tensor_to_bytestream_speed[torch.save] 0.1426ms 0.1412ms 7.0801 KOps/s 6.9436 KOps/s $\color{#35bf28}+1.97\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1261s 0.1255s 7.9664 Ops/s 7.9386 Ops/s $\color{#35bf28}+0.35\%$
test_tensor_to_bytestream_speed[numpy] 2.8797μs 2.8763μs 347.6724 KOps/s 356.5603 KOps/s $\color{#d91a1a}-2.49\%$
test_tensor_to_bytestream_speed[safetensors] 41.7527μs 41.4285μs 24.1379 KOps/s 22.5489 KOps/s $\textbf{\color{#35bf28}+7.05\%}$
test_simple 0.5520s 0.5509s 1.8153 Ops/s 1.7350 Ops/s $\color{#35bf28}+4.63\%$
test_transformed 1.2264s 1.1344s 0.8815 Ops/s 0.8776 Ops/s $\color{#35bf28}+0.45\%$
test_serial 1.6678s 1.6644s 0.6008 Ops/s 0.5902 Ops/s $\color{#35bf28}+1.80\%$
test_parallel 1.1678s 1.0906s 0.9169 Ops/s 0.9352 Ops/s $\color{#d91a1a}-1.95\%$
test_step_mdp_speed[True-True-True-True-True] 0.4167ms 44.8915μs 22.2759 KOps/s 22.6103 KOps/s $\color{#d91a1a}-1.48\%$
test_step_mdp_speed[True-True-True-True-False] 0.4149ms 25.1482μs 39.7643 KOps/s 40.2946 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[True-True-True-False-True] 0.3492ms 24.8564μs 40.2310 KOps/s 40.1440 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[True-True-True-False-False] 49.4010μs 13.7918μs 72.5067 KOps/s 71.1421 KOps/s $\color{#35bf28}+1.92\%$
test_step_mdp_speed[True-True-False-True-True] 0.2114ms 47.7127μs 20.9588 KOps/s 20.8290 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-True-False-True-False] 0.4293ms 27.7182μs 36.0773 KOps/s 36.1928 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-True-False-False-True] 67.9610μs 27.5896μs 36.2455 KOps/s 36.1932 KOps/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-True-False-False-False] 50.5910μs 16.5979μs 60.2486 KOps/s 59.2669 KOps/s $\color{#35bf28}+1.66\%$
test_step_mdp_speed[True-False-True-True-True] 0.4070ms 50.3677μs 19.8540 KOps/s 19.8018 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-False-True-True-False] 63.0710μs 30.4532μs 32.8373 KOps/s 32.6224 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[True-False-True-False-True] 0.4453ms 27.4766μs 36.3946 KOps/s 35.4912 KOps/s $\color{#35bf28}+2.55\%$
test_step_mdp_speed[True-False-True-False-False] 43.6310μs 16.5101μs 60.5691 KOps/s 59.4521 KOps/s $\color{#35bf28}+1.88\%$
test_step_mdp_speed[True-False-False-True-True] 0.4525ms 52.5406μs 19.0329 KOps/s 18.7954 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[True-False-False-True-False] 0.4087ms 33.0098μs 30.2941 KOps/s 30.3312 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-False-False-False-True] 0.1078ms 29.0622μs 34.4089 KOps/s 32.8325 KOps/s $\color{#35bf28}+4.80\%$
test_step_mdp_speed[True-False-False-False-False] 41.8100μs 19.0632μs 52.4571 KOps/s 51.9888 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[False-True-True-True-True] 79.3420μs 49.9907μs 20.0037 KOps/s 19.6660 KOps/s $\color{#35bf28}+1.72\%$
test_step_mdp_speed[False-True-True-True-False] 60.2710μs 30.3420μs 32.9576 KOps/s 33.0036 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[False-True-True-False-True] 2.3401ms 31.3822μs 31.8651 KOps/s 31.0970 KOps/s $\color{#35bf28}+2.47\%$
test_step_mdp_speed[False-True-True-False-False] 42.2710μs 18.2029μs 54.9364 KOps/s 54.2828 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-True-False-True-True] 78.3610μs 52.5625μs 19.0250 KOps/s 18.7437 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-True-False-True-False] 61.6610μs 33.1962μs 30.1239 KOps/s 30.0714 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[False-True-False-False-True] 67.8310μs 33.9729μs 29.4352 KOps/s 28.9570 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[False-True-False-False-False] 48.0210μs 20.8327μs 48.0015 KOps/s 47.6157 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[False-False-True-True-True] 0.1058ms 55.4989μs 18.0184 KOps/s 17.7831 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[False-False-True-True-False] 66.1710μs 36.3966μs 27.4751 KOps/s 27.6777 KOps/s $\color{#d91a1a}-0.73\%$
test_step_mdp_speed[False-False-True-False-True] 67.8610μs 34.2764μs 29.1746 KOps/s 28.9234 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[False-False-True-False-False] 51.7810μs 21.0633μs 47.4759 KOps/s 47.6251 KOps/s $\color{#d91a1a}-0.31\%$
test_step_mdp_speed[False-False-False-True-True] 90.4420μs 58.5917μs 17.0673 KOps/s 17.2886 KOps/s $\color{#d91a1a}-1.28\%$
test_step_mdp_speed[False-False-False-True-False] 0.1066ms 38.6821μs 25.8518 KOps/s 26.4248 KOps/s $\color{#d91a1a}-2.17\%$
test_step_mdp_speed[False-False-False-False-True] 93.8120μs 36.1689μs 27.6480 KOps/s 27.5299 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-False-False-False-False] 52.7910μs 23.2925μs 42.9323 KOps/s 43.4355 KOps/s $\color{#d91a1a}-1.16\%$
test_values[generalized_advantage_estimate-True-True] 10.4504ms 10.2681ms 97.3888 Ops/s 99.3279 Ops/s $\color{#d91a1a}-1.95\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.8540ms 17.8329ms 56.0762 Ops/s 60.3968 Ops/s $\textbf{\color{#d91a1a}-7.15\%}$
test_values[td0_return_estimate-False-False] 0.2076ms 0.1325ms 7.5477 KOps/s 7.6602 KOps/s $\color{#d91a1a}-1.47\%$
test_values[td1_return_estimate-False-False] 28.8438ms 28.3998ms 35.2115 Ops/s 35.5450 Ops/s $\color{#d91a1a}-0.94\%$
test_values[vec_td1_return_estimate-False-False] 20.0734ms 18.1262ms 55.1687 Ops/s 56.2865 Ops/s $\color{#d91a1a}-1.99\%$
test_values[td_lambda_return_estimate-True-False] 42.9704ms 41.6910ms 23.9860 Ops/s 23.8360 Ops/s $\color{#35bf28}+0.63\%$
test_values[vec_td_lambda_return_estimate-True-False] 18.4221ms 17.9106ms 55.8327 Ops/s 55.8924 Ops/s $\color{#d91a1a}-0.11\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.9577ms 8.8317ms 113.2280 Ops/s 112.8668 Ops/s $\color{#35bf28}+0.32\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.0670ms 1.5644ms 639.2417 Ops/s 653.2315 Ops/s $\color{#d91a1a}-2.14\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5174ms 0.4266ms 2.3441 KOps/s 2.3627 KOps/s $\color{#d91a1a}-0.79\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 35.2180ms 34.5632ms 28.9325 Ops/s 33.2635 Ops/s $\textbf{\color{#d91a1a}-13.02\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.1404ms 1.7655ms 566.3960 Ops/s 564.4009 Ops/s $\color{#35bf28}+0.35\%$
test_dqn_speed[False-None] 6.4924ms 1.4458ms 691.6785 Ops/s 704.4080 Ops/s $\color{#d91a1a}-1.81\%$
test_dqn_speed[False-backward] 1.9637ms 1.9160ms 521.9195 Ops/s 517.8668 Ops/s $\color{#35bf28}+0.78\%$
test_dqn_speed[True-None] 0.7310ms 0.5186ms 1.9281 KOps/s 1.8888 KOps/s $\color{#35bf28}+2.08\%$
test_dqn_speed[True-backward] 1.0779ms 0.9802ms 1.0202 KOps/s 1.0237 KOps/s $\color{#d91a1a}-0.33\%$
test_dqn_speed[reduce-overhead-None] 0.7496ms 0.5011ms 1.9958 KOps/s 1.9605 KOps/s $\color{#35bf28}+1.80\%$
test_dqn_speed[reduce-overhead-backward] 0.9960ms 0.9414ms 1.0623 KOps/s 861.6158 Ops/s $\textbf{\color{#35bf28}+23.29\%}$
test_ddpg_speed[False-None] 3.1888ms 2.8526ms 350.5527 Ops/s 346.7393 Ops/s $\color{#35bf28}+1.10\%$
test_ddpg_speed[False-backward] 4.2601ms 4.1044ms 243.6397 Ops/s 239.8656 Ops/s $\color{#35bf28}+1.57\%$
test_ddpg_speed[True-None] 1.5600ms 1.3849ms 722.0513 Ops/s 719.1040 Ops/s $\color{#35bf28}+0.41\%$
test_ddpg_speed[True-backward] 2.7607ms 2.3848ms 419.3186 Ops/s 418.9393 Ops/s $\color{#35bf28}+0.09\%$
test_ddpg_speed[reduce-overhead-None] 1.7415ms 1.3671ms 731.4853 Ops/s 709.4892 Ops/s $\color{#35bf28}+3.10\%$
test_ddpg_speed[reduce-overhead-backward] 2.3945ms 2.3359ms 428.1047 Ops/s 417.4551 Ops/s $\color{#35bf28}+2.55\%$
test_sac_speed[False-None] 8.3666ms 7.9434ms 125.8903 Ops/s 126.7510 Ops/s $\color{#d91a1a}-0.68\%$
test_sac_speed[False-backward] 11.9209ms 11.3121ms 88.4011 Ops/s 90.0456 Ops/s $\color{#d91a1a}-1.83\%$
test_sac_speed[True-None] 2.2983ms 2.0955ms 477.2024 Ops/s 481.8367 Ops/s $\color{#d91a1a}-0.96\%$
test_sac_speed[True-backward] 4.2284ms 4.0135ms 249.1561 Ops/s 230.8113 Ops/s $\textbf{\color{#35bf28}+7.95\%}$
test_sac_speed[reduce-overhead-None] 2.1964ms 2.0675ms 483.6769 Ops/s 474.6436 Ops/s $\color{#35bf28}+1.90\%$
test_sac_speed[reduce-overhead-backward] 4.1414ms 4.0029ms 249.8177 Ops/s 237.3456 Ops/s $\textbf{\color{#35bf28}+5.25\%}$
test_redq_speed[False-None] 10.9161ms 10.2942ms 97.1418 Ops/s 97.4994 Ops/s $\color{#d91a1a}-0.37\%$
test_redq_speed[False-backward] 19.1457ms 17.8160ms 56.1294 Ops/s 56.7164 Ops/s $\color{#d91a1a}-1.04\%$
test_redq_speed[True-None] 4.8507ms 4.3318ms 230.8504 Ops/s 228.6448 Ops/s $\color{#35bf28}+0.96\%$
test_redq_speed[True-backward] 10.0209ms 9.6761ms 103.3475 Ops/s 104.6338 Ops/s $\color{#d91a1a}-1.23\%$
test_redq_speed[reduce-overhead-None] 4.6363ms 4.3000ms 232.5602 Ops/s 232.8170 Ops/s $\color{#d91a1a}-0.11\%$
test_redq_speed[reduce-overhead-backward] 10.0941ms 9.8467ms 101.5564 Ops/s 94.6844 Ops/s $\textbf{\color{#35bf28}+7.26\%}$
test_redq_deprec_speed[False-None] 11.6464ms 11.0729ms 90.3104 Ops/s 90.0062 Ops/s $\color{#35bf28}+0.34\%$
test_redq_deprec_speed[False-backward] 16.3491ms 15.9407ms 62.7326 Ops/s 62.9594 Ops/s $\color{#d91a1a}-0.36\%$
test_redq_deprec_speed[True-None] 3.9499ms 3.5688ms 280.2045 Ops/s 271.6570 Ops/s $\color{#35bf28}+3.15\%$
test_redq_deprec_speed[True-backward] 7.8964ms 7.4839ms 133.6203 Ops/s 125.6034 Ops/s $\textbf{\color{#35bf28}+6.38\%}$
test_redq_deprec_speed[reduce-overhead-None] 3.7719ms 3.5329ms 283.0529 Ops/s 272.3986 Ops/s $\color{#35bf28}+3.91\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.6212ms 7.4287ms 134.6131 Ops/s 130.6403 Ops/s $\color{#35bf28}+3.04\%$
test_td3_speed[False-None] 8.6416ms 8.0009ms 124.9867 Ops/s 125.0495 Ops/s $\color{#d91a1a}-0.05\%$
test_td3_speed[False-backward] 11.5354ms 10.8947ms 91.7880 Ops/s 91.7522 Ops/s $\color{#35bf28}+0.04\%$
test_td3_speed[True-None] 1.8380ms 1.7990ms 555.8492 Ops/s 560.5000 Ops/s $\color{#d91a1a}-0.83\%$
test_td3_speed[True-backward] 3.6525ms 3.5229ms 283.8530 Ops/s 249.4280 Ops/s $\textbf{\color{#35bf28}+13.80\%}$
test_td3_speed[reduce-overhead-None] 1.7832ms 1.7399ms 574.7588 Ops/s 570.8095 Ops/s $\color{#35bf28}+0.69\%$
test_td3_speed[reduce-overhead-backward] 3.7035ms 3.5278ms 283.4590 Ops/s 248.0198 Ops/s $\textbf{\color{#35bf28}+14.29\%}$
test_cql_speed[False-None] 29.1743ms 25.9627ms 38.5167 Ops/s 38.4795 Ops/s $\color{#35bf28}+0.10\%$
test_cql_speed[False-backward] 38.8566ms 35.3729ms 28.2703 Ops/s 28.1853 Ops/s $\color{#35bf28}+0.30\%$
test_cql_speed[True-None] 12.8894ms 12.3399ms 81.0379 Ops/s 80.6624 Ops/s $\color{#35bf28}+0.47\%$
test_cql_speed[True-backward] 18.5930ms 18.2288ms 54.8581 Ops/s 56.9050 Ops/s $\color{#d91a1a}-3.60\%$
test_cql_speed[reduce-overhead-None] 14.9600ms 12.3930ms 80.6906 Ops/s 81.4914 Ops/s $\color{#d91a1a}-0.98\%$
test_cql_speed[reduce-overhead-backward] 18.5997ms 18.1479ms 55.1029 Ops/s 55.2712 Ops/s $\color{#d91a1a}-0.30\%$
test_a2c_speed[False-None] 5.5988ms 5.3720ms 186.1496 Ops/s 182.2448 Ops/s $\color{#35bf28}+2.14\%$
test_a2c_speed[False-backward] 12.2888ms 11.8505ms 84.3845 Ops/s 82.7521 Ops/s $\color{#35bf28}+1.97\%$
test_a2c_speed[True-None] 4.1471ms 3.7122ms 269.3796 Ops/s 257.8654 Ops/s $\color{#35bf28}+4.47\%$
test_a2c_speed[True-backward] 11.0139ms 9.1793ms 108.9405 Ops/s 108.8117 Ops/s $\color{#35bf28}+0.12\%$
test_a2c_speed[reduce-overhead-None] 4.0759ms 3.6841ms 271.4369 Ops/s 272.8130 Ops/s $\color{#d91a1a}-0.50\%$
test_a2c_speed[reduce-overhead-backward] 9.1188ms 8.8002ms 113.6335 Ops/s 113.3426 Ops/s $\color{#35bf28}+0.26\%$
test_ppo_speed[False-None] 6.1585ms 5.9008ms 169.4674 Ops/s 164.7386 Ops/s $\color{#35bf28}+2.87\%$
test_ppo_speed[False-backward] 12.7114ms 12.2578ms 81.5807 Ops/s 78.8323 Ops/s $\color{#35bf28}+3.49\%$
test_ppo_speed[True-None] 4.0569ms 3.6541ms 273.6623 Ops/s 271.3078 Ops/s $\color{#35bf28}+0.87\%$
test_ppo_speed[True-backward] 8.7606ms 8.4951ms 117.7154 Ops/s 109.7614 Ops/s $\textbf{\color{#35bf28}+7.25\%}$
test_ppo_speed[reduce-overhead-None] 4.0183ms 3.6542ms 273.6553 Ops/s 273.9993 Ops/s $\color{#d91a1a}-0.13\%$
test_ppo_speed[reduce-overhead-backward] 9.1149ms 8.7590ms 114.1687 Ops/s 113.0237 Ops/s $\color{#35bf28}+1.01\%$
test_reinforce_speed[False-None] 5.0173ms 4.6858ms 213.4110 Ops/s 214.4807 Ops/s $\color{#d91a1a}-0.50\%$
test_reinforce_speed[False-backward] 8.2360ms 7.4756ms 133.7680 Ops/s 134.6344 Ops/s $\color{#d91a1a}-0.64\%$
test_reinforce_speed[True-None] 3.3239ms 2.8705ms 348.3665 Ops/s 339.4539 Ops/s $\color{#35bf28}+2.63\%$
test_reinforce_speed[True-backward] 7.8819ms 7.6567ms 130.6038 Ops/s 126.5252 Ops/s $\color{#35bf28}+3.22\%$
test_reinforce_speed[reduce-overhead-None] 3.0390ms 2.8635ms 349.2171 Ops/s 352.1590 Ops/s $\color{#d91a1a}-0.84\%$
test_reinforce_speed[reduce-overhead-backward] 8.0666ms 7.8737ms 127.0054 Ops/s 122.7371 Ops/s $\color{#35bf28}+3.48\%$
test_iql_speed[False-None] 26.3692ms 20.4321ms 48.9426 Ops/s 49.2907 Ops/s $\color{#d91a1a}-0.71\%$
test_iql_speed[False-backward] 32.7159ms 30.6961ms 32.5775 Ops/s 31.9287 Ops/s $\color{#35bf28}+2.03\%$
test_iql_speed[True-None] 9.0616ms 8.5196ms 117.3771 Ops/s 115.7572 Ops/s $\color{#35bf28}+1.40\%$
test_iql_speed[True-backward] 20.7042ms 17.1929ms 58.1637 Ops/s 59.8049 Ops/s $\color{#d91a1a}-2.74\%$
test_iql_speed[reduce-overhead-None] 9.0032ms 8.5000ms 117.6468 Ops/s 115.6582 Ops/s $\color{#35bf28}+1.72\%$
test_iql_speed[reduce-overhead-backward] 17.8918ms 17.1102ms 58.4448 Ops/s 58.4483 Ops/s $-0.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5325ms 6.0811ms 164.4438 Ops/s 165.4294 Ops/s $\color{#d91a1a}-0.60\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5408ms 0.3339ms 2.9951 KOps/s 3.4835 KOps/s $\textbf{\color{#d91a1a}-14.02\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6094ms 0.3195ms 3.1299 KOps/s 3.8582 KOps/s $\textbf{\color{#d91a1a}-18.88\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.2646ms 5.7638ms 173.4971 Ops/s 173.5779 Ops/s $\color{#d91a1a}-0.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7093ms 0.3277ms 3.0520 KOps/s 3.3424 KOps/s $\textbf{\color{#d91a1a}-8.69\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5475ms 0.3130ms 3.1952 KOps/s 3.4330 KOps/s $\textbf{\color{#d91a1a}-6.93\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6227ms 1.3520ms 739.6441 Ops/s 790.7034 Ops/s $\textbf{\color{#d91a1a}-6.46\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4996ms 1.2762ms 783.6054 Ops/s 860.0584 Ops/s $\textbf{\color{#d91a1a}-8.89\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.5980ms 6.0182ms 166.1640 Ops/s 169.0519 Ops/s $\color{#d91a1a}-1.71\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3195ms 0.4939ms 2.0246 KOps/s 2.1607 KOps/s $\textbf{\color{#d91a1a}-6.30\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8081ms 0.4089ms 2.4457 KOps/s 2.4094 KOps/s $\color{#35bf28}+1.51\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.0064ms 5.7441ms 174.0907 Ops/s 173.2215 Ops/s $\color{#35bf28}+0.50\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.2064ms 0.3382ms 2.9566 KOps/s 2.8041 KOps/s $\textbf{\color{#35bf28}+5.44\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5421ms 0.3180ms 3.1449 KOps/s 2.9030 KOps/s $\textbf{\color{#35bf28}+8.33\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9871ms 5.7510ms 173.8833 Ops/s 173.6856 Ops/s $\color{#35bf28}+0.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5570ms 0.2730ms 3.6630 KOps/s 3.6235 KOps/s $\color{#35bf28}+1.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5165ms 0.2527ms 3.9566 KOps/s 3.9045 KOps/s $\color{#35bf28}+1.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.1201ms 5.8872ms 169.8602 Ops/s 168.4417 Ops/s $\color{#35bf28}+0.84\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8207ms 0.4270ms 2.3420 KOps/s 1.9408 KOps/s $\textbf{\color{#35bf28}+20.67\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6216ms 0.4067ms 2.4590 KOps/s 2.0144 KOps/s $\textbf{\color{#35bf28}+22.07\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.6811ms 5.1171ms 195.4225 Ops/s 188.9325 Ops/s $\color{#35bf28}+3.44\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 8.7351ms 2.3310ms 429.0030 Ops/s 436.6725 Ops/s $\color{#d91a1a}-1.76\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.5925ms 1.2162ms 822.2469 Ops/s 838.2308 Ops/s $\color{#d91a1a}-1.91\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5088s 15.3371ms 65.2015 Ops/s 55.1424 Ops/s $\textbf{\color{#35bf28}+18.24\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 10.2668ms 2.0865ms 479.2693 Ops/s 463.9806 Ops/s $\color{#35bf28}+3.30\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.3307ms 1.1631ms 859.7372 Ops/s 973.8622 Ops/s $\textbf{\color{#d91a1a}-11.72\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.8615ms 5.3018ms 188.6165 Ops/s 183.3664 Ops/s $\color{#35bf28}+2.86\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 10.8743ms 2.2358ms 447.2576 Ops/s 464.0876 Ops/s $\color{#d91a1a}-3.63\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 3.7268ms 1.2760ms 783.7173 Ops/s 944.4261 Ops/s $\textbf{\color{#d91a1a}-17.02\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 34.7301ms 32.4475ms 30.8190 Ops/s 29.7644 Ops/s $\color{#35bf28}+3.54\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.5597ms 17.7028ms 56.4884 Ops/s 55.9421 Ops/s $\color{#35bf28}+0.98\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 36.5266ms 34.2661ms 29.1833 Ops/s 28.6017 Ops/s $\color{#35bf28}+2.03\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.0678ms 17.7290ms 56.4048 Ops/s 55.0563 Ops/s $\color{#35bf28}+2.45\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 36.7282ms 35.3626ms 28.2784 Ops/s 26.9826 Ops/s $\color{#35bf28}+4.80\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.8425ms 19.4016ms 51.5420 Ops/s 51.0949 Ops/s $\color{#35bf28}+0.88\%$

vmoens added 3 commits October 23, 2025 11:45
@vmoens
Update
aec1e8a 
[ghstack-poisoned]
@vmoens
Update
e1d813f 
[ghstack-poisoned]
@vmoens
Update
553d1e9 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens