Skip to content

[BugFix] Fix GRPO tests and runs #3213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Oct 22, 2025
Merged

[BugFix] Fix GRPO tests and runs #3213

merged 7 commits into from
Oct 22, 2025
+527 −717

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 20, 2025 •
edited
Loading

@vmoens
Update
8b25d90 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Oct 20, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3213

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 24 Pending

As of commit 58f6807 with merge base a7707ca (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 20, 2025
This was referenced Oct 20, 2025
Merged
Open
Open
Open
Open
Open
Open
@vmoens
Update
9604f48 
[ghstack-poisoned]
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 20, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}19$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 83.5413μs 82.2138μs 12.1634 KOps/s 12.2423 KOps/s $\color{#d91a1a}-0.64\%$
test_tensor_to_bytestream_speed[torch.save] 0.1412ms 0.1400ms 7.1407 KOps/s 7.1496 KOps/s $\color{#d91a1a}-0.12\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1230s 0.1220s 8.1995 Ops/s 8.2682 Ops/s $\color{#d91a1a}-0.83\%$
test_tensor_to_bytestream_speed[numpy] 2.8007μs 2.7946μs 357.8325 KOps/s 337.8152 KOps/s $\textbf{\color{#35bf28}+5.93\%}$
test_tensor_to_bytestream_speed[safetensors] 41.9229μs 41.7881μs 23.9303 KOps/s 21.7899 KOps/s $\textbf{\color{#35bf28}+9.82\%}$
test_simple 0.6624s 0.5712s 1.7508 Ops/s 1.7621 Ops/s $\color{#d91a1a}-0.64\%$
test_transformed 1.2235s 1.1309s 0.8842 Ops/s 0.8878 Ops/s $\color{#d91a1a}-0.41\%$
test_serial 1.6628s 1.6613s 0.6020 Ops/s 0.5973 Ops/s $\color{#35bf28}+0.78\%$
test_parallel 1.2156s 1.1372s 0.8794 Ops/s 0.8835 Ops/s $\color{#d91a1a}-0.46\%$
test_step_mdp_speed[True-True-True-True-True] 0.1770ms 45.4297μs 22.0120 KOps/s 22.5204 KOps/s $\color{#d91a1a}-2.26\%$
test_step_mdp_speed[True-True-True-True-False] 57.0210μs 26.1720μs 38.2088 KOps/s 39.8824 KOps/s $\color{#d91a1a}-4.20\%$
test_step_mdp_speed[True-True-True-False-True] 55.0710μs 25.8000μs 38.7597 KOps/s 39.6003 KOps/s $\color{#d91a1a}-2.12\%$
test_step_mdp_speed[True-True-True-False-False] 40.4810μs 14.1934μs 70.4554 KOps/s 73.0589 KOps/s $\color{#d91a1a}-3.56\%$
test_step_mdp_speed[True-True-False-True-True] 81.1510μs 48.8572μs 20.4678 KOps/s 21.1322 KOps/s $\color{#d91a1a}-3.14\%$
test_step_mdp_speed[True-True-False-True-False] 80.5720μs 28.4901μs 35.0999 KOps/s 35.8560 KOps/s $\color{#d91a1a}-2.11\%$
test_step_mdp_speed[True-True-False-False-True] 60.3310μs 29.0377μs 34.4380 KOps/s 35.7489 KOps/s $\color{#d91a1a}-3.67\%$
test_step_mdp_speed[True-True-False-False-False] 47.9610μs 17.3701μs 57.5702 KOps/s 60.1724 KOps/s $\color{#d91a1a}-4.32\%$
test_step_mdp_speed[True-False-True-True-True] 85.3910μs 51.9368μs 19.2542 KOps/s 19.7063 KOps/s $\color{#d91a1a}-2.29\%$
test_step_mdp_speed[True-False-True-True-False] 69.3410μs 31.2118μs 32.0392 KOps/s 32.9053 KOps/s $\color{#d91a1a}-2.63\%$
test_step_mdp_speed[True-False-True-False-True] 63.1810μs 28.8511μs 34.6608 KOps/s 35.7147 KOps/s $\color{#d91a1a}-2.95\%$
test_step_mdp_speed[True-False-True-False-False] 48.5900μs 17.1140μs 58.4318 KOps/s 60.4909 KOps/s $\color{#d91a1a}-3.40\%$
test_step_mdp_speed[True-False-False-True-True] 84.1410μs 54.9164μs 18.2095 KOps/s 19.1057 KOps/s $\color{#d91a1a}-4.69\%$
test_step_mdp_speed[True-False-False-True-False] 76.0420μs 34.4871μs 28.9964 KOps/s 30.2241 KOps/s $\color{#d91a1a}-4.06\%$
test_step_mdp_speed[True-False-False-False-True] 66.9310μs 31.5341μs 31.7117 KOps/s 32.7202 KOps/s $\color{#d91a1a}-3.08\%$
test_step_mdp_speed[True-False-False-False-False] 48.2710μs 19.8617μs 50.3481 KOps/s 52.2617 KOps/s $\color{#d91a1a}-3.66\%$
test_step_mdp_speed[False-True-True-True-True] 81.4410μs 52.2493μs 19.1390 KOps/s 19.7243 KOps/s $\color{#d91a1a}-2.97\%$
test_step_mdp_speed[False-True-True-True-False] 65.1010μs 31.6879μs 31.5578 KOps/s 32.8181 KOps/s $\color{#d91a1a}-3.84\%$
test_step_mdp_speed[False-True-True-False-True] 2.3283ms 32.9362μs 30.3617 KOps/s 31.4320 KOps/s $\color{#d91a1a}-3.41\%$
test_step_mdp_speed[False-True-True-False-False] 50.0310μs 18.5849μs 53.8073 KOps/s 55.0187 KOps/s $\color{#d91a1a}-2.20\%$
test_step_mdp_speed[False-True-False-True-True] 85.8610μs 53.5292μs 18.6814 KOps/s 19.2350 KOps/s $\color{#d91a1a}-2.88\%$
test_step_mdp_speed[False-True-False-True-False] 61.5910μs 33.9485μs 29.4564 KOps/s 30.1711 KOps/s $\color{#d91a1a}-2.37\%$
test_step_mdp_speed[False-True-False-False-True] 69.8920μs 34.9735μs 28.5931 KOps/s 29.2194 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[False-True-False-False-False] 47.8810μs 21.6068μs 46.2817 KOps/s 47.8361 KOps/s $\color{#d91a1a}-3.25\%$
test_step_mdp_speed[False-False-True-True-True] 92.5310μs 57.2573μs 17.4650 KOps/s 17.9273 KOps/s $\color{#d91a1a}-2.58\%$
test_step_mdp_speed[False-False-True-True-False] 68.7810μs 36.7741μs 27.1930 KOps/s 27.3291 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[False-False-True-False-True] 65.6310μs 35.3246μs 28.3089 KOps/s 29.0031 KOps/s $\color{#d91a1a}-2.39\%$
test_step_mdp_speed[False-False-True-False-False] 54.2410μs 21.8174μs 45.8349 KOps/s 47.6440 KOps/s $\color{#d91a1a}-3.80\%$
test_step_mdp_speed[False-False-False-True-True] 0.1013ms 58.7035μs 17.0348 KOps/s 17.3715 KOps/s $\color{#d91a1a}-1.94\%$
test_step_mdp_speed[False-False-False-True-False] 65.7710μs 39.1092μs 25.5694 KOps/s 25.8398 KOps/s $\color{#d91a1a}-1.05\%$
test_step_mdp_speed[False-False-False-False-True] 71.4510μs 37.2908μs 26.8163 KOps/s 27.5981 KOps/s $\color{#d91a1a}-2.83\%$
test_step_mdp_speed[False-False-False-False-False] 50.0410μs 23.8004μs 42.0161 KOps/s 42.1786 KOps/s $\color{#d91a1a}-0.39\%$
test_values[generalized_advantage_estimate-True-True] 10.3209ms 9.9904ms 100.0958 Ops/s 101.5471 Ops/s $\color{#d91a1a}-1.43\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.9654ms 17.5100ms 57.1102 Ops/s 57.4497 Ops/s $\color{#d91a1a}-0.59\%$
test_values[td0_return_estimate-False-False] 0.2578ms 0.1300ms 7.6948 KOps/s 7.5431 KOps/s $\color{#35bf28}+2.01\%$
test_values[td1_return_estimate-False-False] 28.4134ms 27.8173ms 35.9488 Ops/s 36.4730 Ops/s $\color{#d91a1a}-1.44\%$
test_values[vec_td1_return_estimate-False-False] 17.9046ms 17.6104ms 56.7847 Ops/s 55.9994 Ops/s $\color{#35bf28}+1.40\%$
test_values[td_lambda_return_estimate-True-False] 42.0680ms 41.3811ms 24.1656 Ops/s 25.1011 Ops/s $\color{#d91a1a}-3.73\%$
test_values[vec_td_lambda_return_estimate-True-False] 17.8709ms 17.5813ms 56.8785 Ops/s 57.1141 Ops/s $\color{#d91a1a}-0.41\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.7808ms 8.6010ms 116.2652 Ops/s 118.8483 Ops/s $\color{#d91a1a}-2.17\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7005ms 1.4850ms 673.3907 Ops/s 685.2226 Ops/s $\color{#d91a1a}-1.73\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5233ms 0.4137ms 2.4171 KOps/s 2.4684 KOps/s $\color{#d91a1a}-2.08\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 34.5680ms 31.8428ms 31.4042 Ops/s 34.0756 Ops/s $\textbf{\color{#d91a1a}-7.84\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.9269ms 1.6978ms 588.9807 Ops/s 587.4626 Ops/s $\color{#35bf28}+0.26\%$
test_dqn_speed[False-None] 6.4028ms 1.4377ms 695.5650 Ops/s 701.7407 Ops/s $\color{#d91a1a}-0.88\%$
test_dqn_speed[False-backward] 1.9910ms 1.9389ms 515.7563 Ops/s 512.0544 Ops/s $\color{#35bf28}+0.72\%$
test_dqn_speed[True-None] 0.6921ms 0.5146ms 1.9434 KOps/s 1.9043 KOps/s $\color{#35bf28}+2.05\%$
test_dqn_speed[True-backward] 1.0907ms 0.9751ms 1.0256 KOps/s 859.9265 Ops/s $\textbf{\color{#35bf28}+19.26\%}$
test_dqn_speed[reduce-overhead-None] 0.5667ms 0.5029ms 1.9883 KOps/s 1.9632 KOps/s $\color{#35bf28}+1.28\%$
test_dqn_speed[reduce-overhead-backward] 0.9893ms 0.9475ms 1.0554 KOps/s 873.6504 Ops/s $\textbf{\color{#35bf28}+20.80\%}$
test_ddpg_speed[False-None] 3.2079ms 2.8925ms 345.7246 Ops/s 353.4208 Ops/s $\color{#d91a1a}-2.18\%$
test_ddpg_speed[False-backward] 4.2076ms 4.1145ms 243.0429 Ops/s 246.1230 Ops/s $\color{#d91a1a}-1.25\%$
test_ddpg_speed[True-None] 1.9563ms 1.3901ms 719.3633 Ops/s 731.7786 Ops/s $\color{#d91a1a}-1.70\%$
test_ddpg_speed[True-backward] 2.3935ms 2.3586ms 423.9834 Ops/s 382.7713 Ops/s $\textbf{\color{#35bf28}+10.77\%}$
test_ddpg_speed[reduce-overhead-None] 1.6579ms 1.3833ms 722.9324 Ops/s 730.8092 Ops/s $\color{#d91a1a}-1.08\%$
test_ddpg_speed[reduce-overhead-backward] 2.4018ms 2.3602ms 423.6891 Ops/s 381.5881 Ops/s $\textbf{\color{#35bf28}+11.03\%}$
test_sac_speed[False-None] 8.4010ms 7.9298ms 126.1068 Ops/s 127.4951 Ops/s $\color{#d91a1a}-1.09\%$
test_sac_speed[False-backward] 11.7158ms 11.2239ms 89.0959 Ops/s 89.9641 Ops/s $\color{#d91a1a}-0.97\%$
test_sac_speed[True-None] 2.1885ms 2.1049ms 475.0809 Ops/s 470.0928 Ops/s $\color{#35bf28}+1.06\%$
test_sac_speed[True-backward] 4.1479ms 4.0358ms 247.7798 Ops/s 241.1687 Ops/s $\color{#35bf28}+2.74\%$
test_sac_speed[reduce-overhead-None] 2.3032ms 2.1032ms 475.4710 Ops/s 462.5273 Ops/s $\color{#35bf28}+2.80\%$
test_sac_speed[reduce-overhead-backward] 4.1353ms 4.0394ms 247.5638 Ops/s 233.4184 Ops/s $\textbf{\color{#35bf28}+6.06\%}$
test_redq_speed[False-None] 15.1351ms 10.6192ms 94.1695 Ops/s 90.2065 Ops/s $\color{#35bf28}+4.39\%$
test_redq_speed[False-backward] 22.7327ms 18.0831ms 55.3003 Ops/s 53.5804 Ops/s $\color{#35bf28}+3.21\%$
test_redq_speed[True-None] 4.5713ms 4.3671ms 228.9864 Ops/s 231.8102 Ops/s $\color{#d91a1a}-1.22\%$
test_redq_speed[True-backward] 10.0520ms 9.7825ms 102.2232 Ops/s 97.8638 Ops/s $\color{#35bf28}+4.45\%$
test_redq_speed[reduce-overhead-None] 4.4800ms 4.3017ms 232.4661 Ops/s 228.1985 Ops/s $\color{#35bf28}+1.87\%$
test_redq_speed[reduce-overhead-backward] 10.3635ms 10.0711ms 99.2942 Ops/s 100.3642 Ops/s $\color{#d91a1a}-1.07\%$
test_redq_deprec_speed[False-None] 11.3619ms 10.9805ms 91.0709 Ops/s 90.5903 Ops/s $\color{#35bf28}+0.53\%$
test_redq_deprec_speed[False-backward] 16.5619ms 15.9972ms 62.5109 Ops/s 62.4769 Ops/s $\color{#35bf28}+0.05\%$
test_redq_deprec_speed[True-None] 3.9059ms 3.6064ms 277.2823 Ops/s 268.1262 Ops/s $\color{#35bf28}+3.41\%$
test_redq_deprec_speed[True-backward] 7.8898ms 7.6889ms 130.0572 Ops/s 119.0182 Ops/s $\textbf{\color{#35bf28}+9.28\%}$
test_redq_deprec_speed[reduce-overhead-None] 3.7425ms 3.5860ms 278.8658 Ops/s 269.2573 Ops/s $\color{#35bf28}+3.57\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.7440ms 7.6120ms 131.3713 Ops/s 124.7324 Ops/s $\textbf{\color{#35bf28}+5.32\%}$
test_td3_speed[False-None] 8.1577ms 7.9544ms 125.7167 Ops/s 127.0003 Ops/s $\color{#d91a1a}-1.01\%$
test_td3_speed[False-backward] 11.5346ms 10.8611ms 92.0720 Ops/s 93.0689 Ops/s $\color{#d91a1a}-1.07\%$
test_td3_speed[True-None] 1.7915ms 1.7719ms 564.3568 Ops/s 552.3938 Ops/s $\color{#35bf28}+2.17\%$
test_td3_speed[True-backward] 3.7449ms 3.6238ms 275.9525 Ops/s 236.9227 Ops/s $\textbf{\color{#35bf28}+16.47\%}$
test_td3_speed[reduce-overhead-None] 1.8020ms 1.7714ms 564.5356 Ops/s 570.2849 Ops/s $\color{#d91a1a}-1.01\%$
test_td3_speed[reduce-overhead-backward] 3.7373ms 3.6412ms 274.6335 Ops/s 274.8360 Ops/s $\color{#d91a1a}-0.07\%$
test_cql_speed[False-None] 28.7575ms 26.0632ms 38.3683 Ops/s 38.4168 Ops/s $\color{#d91a1a}-0.13\%$
test_cql_speed[False-backward] 38.7128ms 35.7393ms 27.9804 Ops/s 28.9138 Ops/s $\color{#d91a1a}-3.23\%$
test_cql_speed[True-None] 16.6416ms 12.6744ms 78.8989 Ops/s 82.0770 Ops/s $\color{#d91a1a}-3.87\%$
test_cql_speed[True-backward] 18.9373ms 18.5182ms 54.0010 Ops/s 52.9980 Ops/s $\color{#35bf28}+1.89\%$
test_cql_speed[reduce-overhead-None] 12.7537ms 12.4845ms 80.0993 Ops/s 75.4754 Ops/s $\textbf{\color{#35bf28}+6.13\%}$
test_cql_speed[reduce-overhead-backward] 18.4597ms 17.8832ms 55.9184 Ops/s 56.3549 Ops/s $\color{#d91a1a}-0.77\%$
test_a2c_speed[False-None] 5.7062ms 5.4081ms 184.9084 Ops/s 186.8786 Ops/s $\color{#d91a1a}-1.05\%$
test_a2c_speed[False-backward] 12.1768ms 11.8686ms 84.2562 Ops/s 84.4753 Ops/s $\color{#d91a1a}-0.26\%$
test_a2c_speed[True-None] 3.8991ms 3.6994ms 270.3107 Ops/s 269.4855 Ops/s $\color{#35bf28}+0.31\%$
test_a2c_speed[True-backward] 8.7222ms 8.5427ms 117.0591 Ops/s 114.0531 Ops/s $\color{#35bf28}+2.64\%$
test_a2c_speed[reduce-overhead-None] 4.0623ms 3.6846ms 271.4021 Ops/s 274.1838 Ops/s $\color{#d91a1a}-1.01\%$
test_a2c_speed[reduce-overhead-backward] 9.3295ms 8.8743ms 112.6847 Ops/s 112.9097 Ops/s $\color{#d91a1a}-0.20\%$
test_ppo_speed[False-None] 6.1154ms 5.8393ms 171.2524 Ops/s 172.6076 Ops/s $\color{#d91a1a}-0.79\%$
test_ppo_speed[False-backward] 13.0262ms 12.5144ms 79.9082 Ops/s 80.1513 Ops/s $\color{#d91a1a}-0.30\%$
test_ppo_speed[True-None] 3.7647ms 3.6165ms 276.5120 Ops/s 275.5394 Ops/s $\color{#35bf28}+0.35\%$
test_ppo_speed[True-backward] 8.5916ms 8.4614ms 118.1833 Ops/s 117.7587 Ops/s $\color{#35bf28}+0.36\%$
test_ppo_speed[reduce-overhead-None] 3.9476ms 3.5816ms 279.2024 Ops/s 275.6569 Ops/s $\color{#35bf28}+1.29\%$
test_ppo_speed[reduce-overhead-backward] 8.9244ms 8.7503ms 114.2821 Ops/s 107.6343 Ops/s $\textbf{\color{#35bf28}+6.18\%}$
test_reinforce_speed[False-None] 5.0842ms 4.5573ms 219.4261 Ops/s 212.7672 Ops/s $\color{#35bf28}+3.13\%$
test_reinforce_speed[False-backward] 7.6963ms 7.3930ms 135.2631 Ops/s 131.9075 Ops/s $\color{#35bf28}+2.54\%$
test_reinforce_speed[True-None] 3.0410ms 2.8602ms 349.6252 Ops/s 343.6580 Ops/s $\color{#35bf28}+1.74\%$
test_reinforce_speed[True-backward] 8.0761ms 7.6625ms 130.5051 Ops/s 130.9635 Ops/s $\color{#d91a1a}-0.35\%$
test_reinforce_speed[reduce-overhead-None] 3.2096ms 2.8456ms 351.4235 Ops/s 336.8534 Ops/s $\color{#35bf28}+4.33\%$
test_reinforce_speed[reduce-overhead-backward] 8.1111ms 7.8716ms 127.0387 Ops/s 119.4839 Ops/s $\textbf{\color{#35bf28}+6.32\%}$
test_iql_speed[False-None] 24.4245ms 19.8727ms 50.3204 Ops/s 50.0042 Ops/s $\color{#35bf28}+0.63\%$
test_iql_speed[False-backward] 31.1571ms 30.3397ms 32.9601 Ops/s 32.9014 Ops/s $\color{#35bf28}+0.18\%$
test_iql_speed[True-None] 8.6493ms 8.4225ms 118.7298 Ops/s 116.6306 Ops/s $\color{#35bf28}+1.80\%$
test_iql_speed[True-backward] 17.0135ms 16.6525ms 60.0511 Ops/s 59.7185 Ops/s $\color{#35bf28}+0.56\%$
test_iql_speed[reduce-overhead-None] 8.7084ms 8.4494ms 118.3514 Ops/s 117.4714 Ops/s $\color{#35bf28}+0.75\%$
test_iql_speed[reduce-overhead-backward] 17.3739ms 17.0394ms 58.6874 Ops/s 58.1198 Ops/s $\color{#35bf28}+0.98\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5287ms 5.9907ms 166.9255 Ops/s 167.9905 Ops/s $\color{#d91a1a}-0.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5462ms 0.2772ms 3.6074 KOps/s 3.5254 KOps/s $\color{#35bf28}+2.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5440ms 0.2567ms 3.8954 KOps/s 3.8521 KOps/s $\color{#35bf28}+1.12\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.4923ms 5.7637ms 173.4983 Ops/s 176.5389 Ops/s $\color{#d91a1a}-1.72\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.0543ms 0.2756ms 3.6278 KOps/s 3.6238 KOps/s $\color{#35bf28}+0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5960ms 0.2784ms 3.5925 KOps/s 3.6192 KOps/s $\color{#d91a1a}-0.74\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.4634ms 1.2351ms 809.6417 Ops/s 796.1631 Ops/s $\color{#35bf28}+1.69\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.3709ms 1.1673ms 856.7062 Ops/s 865.2583 Ops/s $\color{#d91a1a}-0.99\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 10.0669ms 6.0808ms 164.4514 Ops/s 170.7582 Ops/s $\color{#d91a1a}-3.69\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9799ms 0.4471ms 2.2366 KOps/s 2.2471 KOps/s $\color{#d91a1a}-0.47\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6488ms 0.4125ms 2.4240 KOps/s 2.4994 KOps/s $\color{#d91a1a}-3.02\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8278ms 5.7433ms 174.1167 Ops/s 174.1512 Ops/s $\color{#d91a1a}-0.02\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4890ms 0.2947ms 3.3937 KOps/s 2.9840 KOps/s $\textbf{\color{#35bf28}+13.73\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6818ms 0.2635ms 3.7950 KOps/s 3.1666 KOps/s $\textbf{\color{#35bf28}+19.85\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9200ms 5.6482ms 177.0486 Ops/s 176.6896 Ops/s $\color{#35bf28}+0.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6051ms 0.3251ms 3.0758 KOps/s 3.0740 KOps/s $\color{#35bf28}+0.06\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5406ms 0.3443ms 2.9044 KOps/s 3.2375 KOps/s $\textbf{\color{#d91a1a}-10.29\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.9274ms 5.8455ms 171.0723 Ops/s 171.0837 Ops/s $-0.01\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8412ms 0.4506ms 2.2192 KOps/s 2.1835 KOps/s $\color{#35bf28}+1.63\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7029ms 0.4411ms 2.2668 KOps/s 2.2513 KOps/s $\color{#35bf28}+0.69\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.6215ms 5.0632ms 197.5047 Ops/s 195.7393 Ops/s $\color{#35bf28}+0.90\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.7731ms 2.2423ms 445.9680 Ops/s 435.9950 Ops/s $\color{#35bf28}+2.29\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.8800ms 1.1436ms 874.4019 Ops/s 824.8702 Ops/s $\textbf{\color{#35bf28}+6.00\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5119s 15.2334ms 65.6454 Ops/s 56.2557 Ops/s $\textbf{\color{#35bf28}+16.69\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 10.3925ms 2.0646ms 484.3610 Ops/s 514.8174 Ops/s $\textbf{\color{#d91a1a}-5.92\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 3.3643ms 1.0734ms 931.6114 Ops/s 855.6867 Ops/s $\textbf{\color{#35bf28}+8.87\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 8.6344ms 5.2756ms 189.5502 Ops/s 185.8505 Ops/s $\color{#35bf28}+1.99\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 8.1156ms 2.1571ms 463.5913 Ops/s 458.8757 Ops/s $\color{#35bf28}+1.03\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.7769ms 1.0162ms 984.0836 Ops/s 740.9706 Ops/s $\textbf{\color{#35bf28}+32.81\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 34.9928ms 33.0316ms 30.2741 Ops/s 30.5947 Ops/s $\color{#d91a1a}-1.05\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.8153ms 17.2816ms 57.8651 Ops/s 57.8939 Ops/s $\color{#d91a1a}-0.05\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 36.4682ms 34.0556ms 29.3637 Ops/s 29.4825 Ops/s $\color{#d91a1a}-0.40\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.9553ms 18.3142ms 54.6024 Ops/s 55.0869 Ops/s $\color{#d91a1a}-0.88\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 38.8021ms 35.3548ms 28.2847 Ops/s 28.1178 Ops/s $\color{#35bf28}+0.59\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.9047ms 19.1320ms 52.2684 Ops/s 52.1842 Ops/s $\color{#35bf28}+0.16\%$

vmoens added 5 commits October 19, 2025 20:56
@vmoens
Update
5d400b4 
[ghstack-poisoned]
@vmoens
Update
a2c2973 
[ghstack-poisoned]
@vmoens
Update
ba0f221 
[ghstack-poisoned]
@vmoens
Update
675f0d9 
[ghstack-poisoned]
@vmoens
Update
58f6807 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Oct 22, 2025
@vmoens
[BugFix] Fix GRPO tests and runs
47ad9d8 
ghstack-source-id: a5b7078
Pull-Request: #3213
@vmoens vmoens merged commit 58f6807 into gh/vmoens/167/base Oct 22, 2025
73 of 89 checks passed
@vmoens vmoens deleted the gh/vmoens/167/head branch October 22, 2025 04:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens