New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

对GRPO num_generations有些不理解？ #137

Open

gmyFighting opened this issue Mar 9, 2025 · 1 comment

gmyFighting commented Mar 9, 2025

请问在GRPO中，显存占用和num_generations有关吗？为什么num_generations设置的越大，显存占用越高？

我的理解是计算优势生成的这些output都是不参与梯度回传的吧？相当于是off policy的过程，只用来推理就可以了。但是这样理解的话，伴随着num_generations增加的显存是来自于哪里呢？

1518630367 commented Mar 10, 2025

请问在GRPO中，显存占用和num_generations有关吗？为什么num_generations设置的越大，显存占用越高？

我的理解是计算优势生成的这些output都是不参与梯度回传的吧？相当于是off policy的过程，只用来推理就可以了。但是这样理解的话，伴随着num_generations增加的显存是来自于哪里呢？

batch_size

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment