Skip to content

Conversation

ozhanozen
Copy link
Contributor

Description

When running Ray directly from tuner.py, Ray is not correctly initialized within invoke_tuning_run(). The two problems associated with this are discussed in #3532. To solve them, this PR:

  1. Removes ray_init() from util.get_gpu_node_resources(). Now, ray needs to be initialized before calling util.get_gpu_node_resources(). This change actually reverses Fixes the missing Ray initialization #3350, which was merged to add the missing initialization when using tuner.py, but it is safer to explicitly initialize Ray with the correct arguments outside of the util.get_gpu_node_resources().
  2. Moves Ray initialization within invoke_tuning_run() to be before util.get_gpu_node_resources() so we explicitly initialize it before and do not raise an exception later.
  3. Adds a warning when calling ray_init() if Ray was already initialized.

Fixes #3532

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Screenshots

Change 1:
Screenshot 2025-09-23 at 16 52 55

Change 2:
Screenshot 2025-09-23 at 16 52 33

Change 3:
Screenshot 2025-09-23 at 16 55 21

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@ozhanozen ozhanozen requested a review from ooctipus as a code owner September 23, 2025 14:59
@github-actions github-actions bot added bug Something isn't working isaac-lab Related to Isaac Lab team labels Sep 23, 2025
@garylvov
Copy link
Collaborator

LGTM thanks for catching this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working isaac-lab Related to Isaac Lab team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug Report] Wrong ray initialization within invoke_tuning_run()
2 participants