New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pod gpu error cuMemoryAllocate failed res=2 #31

Open

beast-R opened this issue Oct 29, 2024 · 1 comment

beast-R commented Oct 29, 2024

k8s1.23
when my code run sometime later ，hami-core report an error
isolation network ，just photo

Author

beast-R commented Oct 29, 2024 •

edited

Loading

export GPU_CORE_UTILIZATION_POLICY=disable in my pod
and how to get hami-core NCCLWORK 6260 log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment