Skip to content

[Feature] Trainer Infrastructure - Timing and Utilities #3188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 21 commits into
base: gh/vmoens/151/base
Choose a base branch
from
Open

[Feature] Trainer Infrastructure - Timing and Utilities #3188

wants to merge 21 commits into from
+242 −35

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 14, 2025 •
edited
Loading

@vmoens
Update
f553b66 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Oct 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3188

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 10 Unrelated Failures

As of commit 4a09103 with merge base 92c20cd (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Oct 14, 2025
Merged
Merged
Merged
Merged
Merged
Open
Open
Open
Open
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 14, 2025
vmoens added 7 commits October 16, 2025 15:00
@vmoens
Update
bda4145 
[ghstack-poisoned]
@vmoens
Update
be2c94b 
[ghstack-poisoned]
@vmoens
Update
876de4f 
[ghstack-poisoned]
@vmoens
Update
0ad3f7a 
[ghstack-poisoned]
@vmoens
Update
019f157 
[ghstack-poisoned]
@vmoens
Update
db37e8c 
[ghstack-poisoned]
@vmoens
Update
84b6f37 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 18, 2025
Open
vmoens added 4 commits October 18, 2025 19:00
@vmoens
Update
475044d 
[ghstack-poisoned]
@vmoens
Update
2dc2e22 
[ghstack-poisoned]
@vmoens
Update
c64e7f2 
[ghstack-poisoned]
@vmoens
Update
9bad747 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Oct 20, 2025
@vmoens
Update
a05a91d 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
c78b8d2 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
3579eaa 
[ghstack-poisoned]
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 20, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}19$. Worsened: $\large\color{#d91a1a}16$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 84.2502μs 82.9562μs 12.0545 KOps/s 12.1328 KOps/s $\color{#d91a1a}-0.64\%$
test_tensor_to_bytestream_speed[torch.save] 0.1421ms 0.1409ms 7.0955 KOps/s 7.0668 KOps/s $\color{#35bf28}+0.41\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1257s 0.1252s 7.9867 Ops/s 7.9818 Ops/s $\color{#35bf28}+0.06\%$
test_tensor_to_bytestream_speed[numpy] 2.8464μs 2.8431μs 351.7284 KOps/s 356.8438 KOps/s $\color{#d91a1a}-1.43\%$
test_tensor_to_bytestream_speed[safetensors] 42.3352μs 41.9852μs 23.8179 KOps/s 22.8497 KOps/s $\color{#35bf28}+4.24\%$
test_simple 0.5575s 0.5513s 1.8138 Ops/s 1.7266 Ops/s $\textbf{\color{#35bf28}+5.05\%}$
test_transformed 1.1078s 1.1062s 0.9040 Ops/s 0.8712 Ops/s $\color{#35bf28}+3.76\%$
test_serial 1.6699s 1.6657s 0.6003 Ops/s 0.5859 Ops/s $\color{#35bf28}+2.46\%$
test_parallel 1.1976s 1.1133s 0.8982 Ops/s 0.8836 Ops/s $\color{#35bf28}+1.66\%$
test_step_mdp_speed[True-True-True-True-True] 0.1391ms 45.0438μs 22.2006 KOps/s 22.4938 KOps/s $\color{#d91a1a}-1.30\%$
test_step_mdp_speed[True-True-True-True-False] 52.3330μs 25.4996μs 39.2163 KOps/s 40.4853 KOps/s $\color{#d91a1a}-3.13\%$
test_step_mdp_speed[True-True-True-False-True] 61.1530μs 25.6323μs 39.0133 KOps/s 39.6792 KOps/s $\color{#d91a1a}-1.68\%$
test_step_mdp_speed[True-True-True-False-False] 37.4420μs 14.0921μs 70.9618 KOps/s 71.8120 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[True-True-False-True-True] 84.4840μs 48.8517μs 20.4701 KOps/s 20.6612 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[True-True-False-True-False] 66.0630μs 28.6294μs 34.9291 KOps/s 35.5032 KOps/s $\color{#d91a1a}-1.62\%$
test_step_mdp_speed[True-True-False-False-True] 93.8440μs 28.4184μs 35.1885 KOps/s 35.2635 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[True-True-False-False-False] 48.6620μs 16.9549μs 58.9801 KOps/s 59.2895 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[True-False-True-True-True] 92.5550μs 51.8899μs 19.2716 KOps/s 19.2421 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[True-False-True-True-False] 64.1740μs 31.2842μs 31.9650 KOps/s 32.0199 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[True-False-True-False-True] 60.6330μs 28.4145μs 35.1933 KOps/s 34.9526 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-False-True-False-False] 46.7420μs 16.8147μs 59.4719 KOps/s 59.3418 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[True-False-False-True-True] 91.8550μs 54.1010μs 18.4839 KOps/s 18.4601 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[True-False-False-True-False] 88.6450μs 33.4646μs 29.8823 KOps/s 29.6149 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[True-False-False-False-True] 60.1720μs 30.9483μs 32.3119 KOps/s 32.2129 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-False-False-False] 47.4920μs 19.3810μs 51.5969 KOps/s 50.7424 KOps/s $\color{#35bf28}+1.68\%$
test_step_mdp_speed[False-True-True-True-True] 87.8840μs 51.6572μs 19.3584 KOps/s 19.2474 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-True-True-True-False] 56.4930μs 31.0020μs 32.2560 KOps/s 32.1352 KOps/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[False-True-True-False-True] 2.3487ms 32.4889μs 30.7798 KOps/s 30.6982 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-True-True-False-False] 47.9430μs 18.5457μs 53.9207 KOps/s 52.8223 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[False-True-False-True-True] 90.0950μs 53.9641μs 18.5308 KOps/s 18.3480 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-True-False-True-False] 83.6540μs 33.5674μs 29.7908 KOps/s 29.3855 KOps/s $\color{#35bf28}+1.38\%$
test_step_mdp_speed[False-True-False-False-True] 65.0440μs 35.3943μs 28.2531 KOps/s 28.6551 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[False-True-False-False-False] 49.7430μs 21.5373μs 46.4310 KOps/s 47.0050 KOps/s $\color{#d91a1a}-1.22\%$
test_step_mdp_speed[False-False-True-True-True] 0.1011ms 57.3273μs 17.4437 KOps/s 17.3789 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[False-False-True-True-False] 84.8940μs 36.3408μs 27.5173 KOps/s 27.6923 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-False-True-False-True] 68.9730μs 34.7513μs 28.7759 KOps/s 28.5115 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[False-False-True-False-False] 50.2030μs 21.3938μs 46.7426 KOps/s 47.0694 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-False-False-True-True] 99.9740μs 58.6622μs 17.0467 KOps/s 17.0106 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-False-False-True-False] 79.5340μs 38.8527μs 25.7382 KOps/s 25.9573 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[False-False-False-False-True] 68.7730μs 36.6618μs 27.2764 KOps/s 26.8959 KOps/s $\color{#35bf28}+1.41\%$
test_step_mdp_speed[False-False-False-False-False] 52.2220μs 23.6398μs 42.3016 KOps/s 41.5416 KOps/s $\color{#35bf28}+1.83\%$
test_values[generalized_advantage_estimate-True-True] 11.1961ms 10.2623ms 97.4444 Ops/s 97.5825 Ops/s $\color{#d91a1a}-0.14\%$
test_values[vec_generalized_advantage_estimate-True-True] 13.9245ms 11.0360ms 90.6128 Ops/s 90.3748 Ops/s $\color{#35bf28}+0.26\%$
test_values[td0_return_estimate-False-False] 0.2147ms 0.1305ms 7.6636 KOps/s 8.4010 KOps/s $\textbf{\color{#d91a1a}-8.78\%}$
test_values[td1_return_estimate-False-False] 29.1537ms 28.1883ms 35.4757 Ops/s 35.5936 Ops/s $\color{#d91a1a}-0.33\%$
test_values[vec_td1_return_estimate-False-False] 11.4262ms 11.1113ms 89.9987 Ops/s 90.2592 Ops/s $\color{#d91a1a}-0.29\%$
test_values[td_lambda_return_estimate-True-False] 49.7610ms 42.8115ms 23.3582 Ops/s 23.5126 Ops/s $\color{#d91a1a}-0.66\%$
test_values[vec_td_lambda_return_estimate-True-False] 11.4246ms 11.0842ms 90.2188 Ops/s 90.2326 Ops/s $\color{#d91a1a}-0.02\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.0278ms 8.8645ms 112.8098 Ops/s 112.4278 Ops/s $\color{#35bf28}+0.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7394ms 1.5143ms 660.3690 Ops/s 652.2345 Ops/s $\color{#35bf28}+1.25\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5306ms 0.4194ms 2.3841 KOps/s 2.3955 KOps/s $\color{#d91a1a}-0.48\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 29.9868ms 29.4407ms 33.9666 Ops/s 41.9125 Ops/s $\textbf{\color{#d91a1a}-18.96\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.1005ms 1.7039ms 586.9059 Ops/s 579.9949 Ops/s $\color{#35bf28}+1.19\%$
test_dqn_speed[False-None] 6.6981ms 1.4680ms 681.1829 Ops/s 693.0186 Ops/s $\color{#d91a1a}-1.71\%$
test_dqn_speed[False-backward] 2.0044ms 1.9420ms 514.9451 Ops/s 513.1556 Ops/s $\color{#35bf28}+0.35\%$
test_dqn_speed[True-None] 0.7522ms 0.5166ms 1.9358 KOps/s 1.8365 KOps/s $\textbf{\color{#35bf28}+5.40\%}$
test_dqn_speed[True-backward] 1.0294ms 0.9784ms 1.0221 KOps/s 903.2867 Ops/s $\textbf{\color{#35bf28}+13.15\%}$
test_dqn_speed[reduce-overhead-None] 0.9195ms 0.5205ms 1.9211 KOps/s 1.9390 KOps/s $\color{#d91a1a}-0.92\%$
test_dqn_speed[reduce-overhead-backward] 1.0287ms 0.9524ms 1.0499 KOps/s 897.2906 Ops/s $\textbf{\color{#35bf28}+17.01\%}$
test_ddpg_speed[False-None] 3.3123ms 2.9047ms 344.2645 Ops/s 336.1982 Ops/s $\color{#35bf28}+2.40\%$
test_ddpg_speed[False-backward] 4.3081ms 4.1665ms 240.0111 Ops/s 242.0269 Ops/s $\color{#d91a1a}-0.83\%$
test_ddpg_speed[True-None] 1.7563ms 1.3798ms 724.7629 Ops/s 705.1355 Ops/s $\color{#35bf28}+2.78\%$
test_ddpg_speed[True-backward] 2.5198ms 2.3541ms 424.7823 Ops/s 348.0854 Ops/s $\textbf{\color{#35bf28}+22.03\%}$
test_ddpg_speed[reduce-overhead-None] 2.2062ms 1.3923ms 718.2163 Ops/s 695.9684 Ops/s $\color{#35bf28}+3.20\%$
test_ddpg_speed[reduce-overhead-backward] 2.4105ms 2.3508ms 425.3959 Ops/s 369.7947 Ops/s $\textbf{\color{#35bf28}+15.04\%}$
test_sac_speed[False-None] 8.3933ms 7.9451ms 125.8631 Ops/s 124.9932 Ops/s $\color{#35bf28}+0.70\%$
test_sac_speed[False-backward] 12.3150ms 11.3479ms 88.1218 Ops/s 88.2795 Ops/s $\color{#d91a1a}-0.18\%$
test_sac_speed[True-None] 2.3984ms 2.0906ms 478.3279 Ops/s 469.9026 Ops/s $\color{#35bf28}+1.79\%$
test_sac_speed[True-backward] 4.1407ms 4.0117ms 249.2727 Ops/s 243.6742 Ops/s $\color{#35bf28}+2.30\%$
test_sac_speed[reduce-overhead-None] 2.5243ms 2.0799ms 480.7927 Ops/s 442.4562 Ops/s $\textbf{\color{#35bf28}+8.66\%}$
test_sac_speed[reduce-overhead-backward] 4.0941ms 4.0033ms 249.7946 Ops/s 205.7755 Ops/s $\textbf{\color{#35bf28}+21.39\%}$
test_redq_speed[False-None] 10.9195ms 10.3317ms 96.7893 Ops/s 95.6740 Ops/s $\color{#35bf28}+1.17\%$
test_redq_speed[False-backward] 18.3844ms 17.8704ms 55.9585 Ops/s 56.0032 Ops/s $\color{#d91a1a}-0.08\%$
test_redq_speed[True-None] 4.7655ms 4.3105ms 231.9917 Ops/s 213.8726 Ops/s $\textbf{\color{#35bf28}+8.47\%}$
test_redq_speed[True-backward] 10.1762ms 9.8213ms 101.8198 Ops/s 97.2156 Ops/s $\color{#35bf28}+4.74\%$
test_redq_speed[reduce-overhead-None] 4.5500ms 4.3227ms 231.3395 Ops/s 215.2767 Ops/s $\textbf{\color{#35bf28}+7.46\%}$
test_redq_speed[reduce-overhead-backward] 10.2261ms 9.9050ms 100.9593 Ops/s 99.9249 Ops/s $\color{#35bf28}+1.04\%$
test_redq_deprec_speed[False-None] 11.3947ms 10.8911ms 91.8180 Ops/s 90.0091 Ops/s $\color{#35bf28}+2.01\%$
test_redq_deprec_speed[False-backward] 16.1945ms 15.6353ms 63.9577 Ops/s 62.3074 Ops/s $\color{#35bf28}+2.65\%$
test_redq_deprec_speed[True-None] 3.9515ms 3.6134ms 276.7471 Ops/s 268.1399 Ops/s $\color{#35bf28}+3.21\%$
test_redq_deprec_speed[True-backward] 7.7212ms 7.5053ms 133.2384 Ops/s 127.6392 Ops/s $\color{#35bf28}+4.39\%$
test_redq_deprec_speed[reduce-overhead-None] 3.8294ms 3.6184ms 276.3646 Ops/s 271.1873 Ops/s $\color{#35bf28}+1.91\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.7975ms 7.4816ms 133.6614 Ops/s 129.8661 Ops/s $\color{#35bf28}+2.92\%$
test_td3_speed[False-None] 8.2313ms 7.9753ms 125.3864 Ops/s 125.0846 Ops/s $\color{#35bf28}+0.24\%$
test_td3_speed[False-backward] 11.7958ms 10.9237ms 91.5437 Ops/s 91.2632 Ops/s $\color{#35bf28}+0.31\%$
test_td3_speed[True-None] 1.8158ms 1.7730ms 564.0018 Ops/s 547.2908 Ops/s $\color{#35bf28}+3.05\%$
test_td3_speed[True-backward] 3.7287ms 3.5973ms 277.9824 Ops/s 254.7979 Ops/s $\textbf{\color{#35bf28}+9.10\%}$
test_td3_speed[reduce-overhead-None] 1.7893ms 1.7554ms 569.6845 Ops/s 564.3871 Ops/s $\color{#35bf28}+0.94\%$
test_td3_speed[reduce-overhead-backward] 3.8034ms 3.6628ms 273.0160 Ops/s 263.2813 Ops/s $\color{#35bf28}+3.70\%$
test_cql_speed[False-None] 28.9074ms 26.1254ms 38.2769 Ops/s 38.5249 Ops/s $\color{#d91a1a}-0.64\%$
test_cql_speed[False-backward] 40.8203ms 36.7583ms 27.2048 Ops/s 28.2583 Ops/s $\color{#d91a1a}-3.73\%$
test_cql_speed[True-None] 13.3683ms 12.4867ms 80.0854 Ops/s 81.9698 Ops/s $\color{#d91a1a}-2.30\%$
test_cql_speed[True-backward] 18.9242ms 18.5115ms 54.0204 Ops/s 58.8875 Ops/s $\textbf{\color{#d91a1a}-8.27\%}$
test_cql_speed[reduce-overhead-None] 13.1900ms 12.4884ms 80.0745 Ops/s 85.2047 Ops/s $\textbf{\color{#d91a1a}-6.02\%}$
test_cql_speed[reduce-overhead-backward] 19.1491ms 18.6628ms 53.5826 Ops/s 58.3425 Ops/s $\textbf{\color{#d91a1a}-8.16\%}$
test_a2c_speed[False-None] 5.6921ms 5.3410ms 187.2315 Ops/s 191.3205 Ops/s $\color{#d91a1a}-2.14\%$
test_a2c_speed[False-backward] 12.1208ms 11.8481ms 84.4017 Ops/s 84.2359 Ops/s $\color{#35bf28}+0.20\%$
test_a2c_speed[True-None] 4.0224ms 3.6772ms 271.9446 Ops/s 287.2905 Ops/s $\textbf{\color{#d91a1a}-5.34\%}$
test_a2c_speed[True-backward] 8.8576ms 8.6443ms 115.6827 Ops/s 111.4800 Ops/s $\color{#35bf28}+3.77\%$
test_a2c_speed[reduce-overhead-None] 3.8551ms 3.7237ms 268.5535 Ops/s 268.7395 Ops/s $\color{#d91a1a}-0.07\%$
test_a2c_speed[reduce-overhead-backward] 8.9629ms 8.7316ms 114.5271 Ops/s 112.7356 Ops/s $\color{#35bf28}+1.59\%$
test_ppo_speed[False-None] 6.3457ms 5.9778ms 167.2843 Ops/s 173.6152 Ops/s $\color{#d91a1a}-3.65\%$
test_ppo_speed[False-backward] 12.8320ms 12.4171ms 80.5340 Ops/s 82.1798 Ops/s $\color{#d91a1a}-2.00\%$
test_ppo_speed[True-None] 3.8181ms 3.6498ms 273.9902 Ops/s 272.6963 Ops/s $\color{#35bf28}+0.47\%$
test_ppo_speed[True-backward] 8.6652ms 8.4774ms 117.9607 Ops/s 114.7595 Ops/s $\color{#35bf28}+2.79\%$
test_ppo_speed[reduce-overhead-None] 4.0290ms 3.6131ms 276.7741 Ops/s 275.9609 Ops/s $\color{#35bf28}+0.29\%$
test_ppo_speed[reduce-overhead-backward] 9.1354ms 8.7339ms 114.4963 Ops/s 112.0885 Ops/s $\color{#35bf28}+2.15\%$
test_reinforce_speed[False-None] 4.7375ms 4.5241ms 221.0376 Ops/s 217.9016 Ops/s $\color{#35bf28}+1.44\%$
test_reinforce_speed[False-backward] 7.6568ms 7.3829ms 135.4490 Ops/s 134.4417 Ops/s $\color{#35bf28}+0.75\%$
test_reinforce_speed[True-None] 3.1627ms 2.8366ms 352.5307 Ops/s 344.7500 Ops/s $\color{#35bf28}+2.26\%$
test_reinforce_speed[True-backward] 8.0630ms 7.7522ms 128.9958 Ops/s 123.5970 Ops/s $\color{#35bf28}+4.37\%$
test_reinforce_speed[reduce-overhead-None] 3.0077ms 2.8332ms 352.9616 Ops/s 347.3154 Ops/s $\color{#35bf28}+1.63\%$
test_reinforce_speed[reduce-overhead-backward] 8.1614ms 7.8873ms 126.7858 Ops/s 117.1976 Ops/s $\textbf{\color{#35bf28}+8.18\%}$
test_iql_speed[False-None] 25.6936ms 20.1122ms 49.7211 Ops/s 51.1552 Ops/s $\color{#d91a1a}-2.80\%$
test_iql_speed[False-backward] 31.1654ms 30.5982ms 32.6817 Ops/s 33.2253 Ops/s $\color{#d91a1a}-1.64\%$
test_iql_speed[True-None] 8.9224ms 8.4807ms 117.9153 Ops/s 110.2898 Ops/s $\textbf{\color{#35bf28}+6.91\%}$
test_iql_speed[True-backward] 17.3059ms 16.8201ms 59.4526 Ops/s 58.7665 Ops/s $\color{#35bf28}+1.17\%$
test_iql_speed[reduce-overhead-None] 8.9941ms 8.5625ms 116.7889 Ops/s 116.7401 Ops/s $\color{#35bf28}+0.04\%$
test_iql_speed[reduce-overhead-backward] 18.5176ms 17.2545ms 57.9558 Ops/s 58.1826 Ops/s $\color{#d91a1a}-0.39\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5662ms 6.0354ms 165.6879 Ops/s 166.5572 Ops/s $\color{#d91a1a}-0.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6917ms 0.3351ms 2.9841 KOps/s 3.6018 KOps/s $\textbf{\color{#d91a1a}-17.15\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6449ms 0.3144ms 3.1809 KOps/s 3.7050 KOps/s $\textbf{\color{#d91a1a}-14.15\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0660ms 5.8097ms 172.1265 Ops/s 175.6514 Ops/s $\color{#d91a1a}-2.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5758s 0.7089ms 1.4105 KOps/s 3.5977 KOps/s $\textbf{\color{#d91a1a}-60.79\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5088ms 0.2526ms 3.9594 KOps/s 3.9243 KOps/s $\color{#35bf28}+0.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5857ms 1.3391ms 746.7548 Ops/s 792.0898 Ops/s $\textbf{\color{#d91a1a}-5.72\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6288ms 1.3099ms 763.4362 Ops/s 848.6351 Ops/s $\textbf{\color{#d91a1a}-10.04\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.1276ms 5.9613ms 167.7498 Ops/s 170.7850 Ops/s $\color{#d91a1a}-1.78\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0359ms 0.4601ms 2.1734 KOps/s 2.2888 KOps/s $\textbf{\color{#d91a1a}-5.04\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8442ms 0.4028ms 2.4828 KOps/s 2.4648 KOps/s $\color{#35bf28}+0.73\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9612ms 5.8219ms 171.7650 Ops/s 174.6637 Ops/s $\color{#d91a1a}-1.66\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.2748ms 0.2905ms 3.4418 KOps/s 853.8679 Ops/s $\textbf{\color{#35bf28}+303.08\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6563ms 0.3125ms 3.1998 KOps/s 3.1061 KOps/s $\color{#35bf28}+3.02\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9697ms 5.7452ms 174.0569 Ops/s 172.4003 Ops/s $\color{#35bf28}+0.96\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9438ms 0.2715ms 3.6828 KOps/s 3.1088 KOps/s $\textbf{\color{#35bf28}+18.46\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4387ms 0.2676ms 3.7371 KOps/s 3.6793 KOps/s $\color{#35bf28}+1.57\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.2430ms 6.0107ms 166.3692 Ops/s 165.9755 Ops/s $\color{#35bf28}+0.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.8346ms 0.4920ms 2.0326 KOps/s 2.2524 KOps/s $\textbf{\color{#d91a1a}-9.76\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8010ms 0.4091ms 2.4444 KOps/s 2.3569 KOps/s $\color{#35bf28}+3.71\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4965s 14.8868ms 67.1738 Ops/s 193.9858 Ops/s $\textbf{\color{#d91a1a}-65.37\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 3.8883ms 1.7323ms 577.2780 Ops/s 404.8386 Ops/s $\textbf{\color{#35bf28}+42.59\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.0912ms 1.0075ms 992.5719 Ops/s 789.9569 Ops/s $\textbf{\color{#35bf28}+25.65\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 7.7027ms 5.0550ms 197.8253 Ops/s 54.9185 Ops/s $\textbf{\color{#35bf28}+260.22\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 7.9397ms 2.0116ms 497.1281 Ops/s 519.2660 Ops/s $\color{#d91a1a}-4.26\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 8.0416ms 1.2364ms 808.7684 Ops/s 869.6314 Ops/s $\textbf{\color{#d91a1a}-7.00\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4729s 14.8761ms 67.2217 Ops/s 185.7013 Ops/s $\textbf{\color{#d91a1a}-63.80\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 10.7087ms 2.1168ms 472.4130 Ops/s 450.3285 Ops/s $\color{#35bf28}+4.90\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.1614ms 1.0017ms 998.3195 Ops/s 812.7722 Ops/s $\textbf{\color{#35bf28}+22.83\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 36.5245ms 33.1355ms 30.1791 Ops/s 30.1865 Ops/s $\color{#d91a1a}-0.02\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.6697ms 18.1924ms 54.9681 Ops/s 56.6764 Ops/s $\color{#d91a1a}-3.01\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 37.4515ms 34.5909ms 28.9093 Ops/s 29.3354 Ops/s $\color{#d91a1a}-1.45\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.7546ms 17.8233ms 56.1063 Ops/s 56.0182 Ops/s $\color{#35bf28}+0.16\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 38.9539ms 36.0280ms 27.7562 Ops/s 27.6314 Ops/s $\color{#35bf28}+0.45\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.3635ms 19.1341ms 52.2628 Ops/s 51.8423 Ops/s $\color{#35bf28}+0.81\%$

vmoens added 6 commits October 19, 2025 20:56
@vmoens
Update
30d4e53 
[ghstack-poisoned]
@vmoens
Update
e97f430 
[ghstack-poisoned]
@vmoens
Update
495a051 
[ghstack-poisoned]
@vmoens
Update
82ba9b9 
[ghstack-poisoned]
@vmoens
Update
d30b5f4 
[ghstack-poisoned]
@vmoens
Update
4a09103 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens