RHOAIENG-26482: Add GCS Fault Tolerance, Disable Usage Stats, Rename the RayJob scope RayCluster Config

[kryanbeane]

Jira link

RHOAIENG-26482

What changes have been made

• Added GCS fault tolerance, verification & future changes pending on the bug fix.
• Renamed RayJobClusterConfig to ManagedClusterConfig - made more sense to me as the Cluster is managed by RayJobs, and RayJob being in the name was confusing
• Disable usage stats and didn't add ability to overwrite it. We initially made it changeable by the user as we didn't want to add new fields the user couldn't change. With 3.0 we have the opportunity to just disable the usage stats.

Verification steps

• There are no verification steps yet - testing and merging this is dependant on the GCS Fault Tolerance bug being fixed first so this PR will sit until then.

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

[codecov]

Codecov Report

❌ Patch coverage is 83.33333% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.65%. Comparing base (3228739) to head (e5f6044).
⚠️ Report is 3 commits behind head on ray-jobs-feature.

Files with missing lines	Patch %	Lines
src/codeflare_sdk/ray/rayjobs/config.py	82.60%	4 Missing ⚠️

Additional details and impacted files

@@                 Coverage Diff                  @@
##           ray-jobs-feature     #880      +/-   ##
====================================================
- Coverage             93.67%   93.65%   -0.03%     
====================================================
  Files                    31       20      -11     
  Lines                  1724     1717       -7     
====================================================
- Hits                   1615     1608       -7     
  Misses                  109      109              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[openshift-ci-robot]

@kryanbeane: This pull request references RHOAIENG-26482 which is a valid jira issue.

In response to this:

Issue link

What changes have been made

Verification steps

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@kryanbeane: This pull request references RHOAIENG-26482 which is a valid jira issue.

In response to this:

Jira link

RHOAIENG-26482

What changes have been made

Verification steps

• There are no verification steps yet - testing and merging this is dependant on the GCS Fault Tolerance bug being fixed first so this PR will sit until then.

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[kryanbeane]

/hold

[openshift-ci-robot]

@kryanbeane: This pull request references RHOAIENG-26482 which is a valid jira issue.

In response to this:

Jira link

RHOAIENG-26482

What changes have been made

Verification steps

• There are no verification steps yet - testing and merging this is dependant on the GCS Fault Tolerance bug being fixed first so this PR will sit until then.

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@kryanbeane: This pull request references RHOAIENG-26482 which is a valid jira issue.

In response to this:

Jira link

RHOAIENG-26482

What changes have been made

Verification steps

• There are no verification steps yet - testing and merging this is dependant on the GCS Fault Tolerance bug being fixed first so this PR will sit until then.

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[kryanbeane]

/hold

[openshift-ci-robot]

@kryanbeane: This pull request references RHOAIENG-26482 which is a valid jira issue.

In response to this:

Jira link

RHOAIENG-26482

What changes have been made

Added GCS fault tolerance, verification & future changes pending on the bug fix.

Verification steps

• There are no verification steps yet - testing and merging this is dependant on the GCS Fault Tolerance bug being fixed first so this PR will sit until then.
• Renamed RayJobClusterConfig to ManagedClusterConfig - made more sense to me as the Cluster is managed by RayJobs, and RayJob being in the name was confusing
• Disable usage stats and didn't add ability to overwrite it. We initially made it changeable by the user as we didn't want to add new fields the user couldn't change. With 3.0 we have the opportunity to just disable the usage stats.

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@kryanbeane: This pull request references RHOAIENG-26482 which is a valid jira issue.

In response to this:

Jira link

RHOAIENG-26482

What changes have been made

• Added GCS fault tolerance, verification & future changes pending on the bug fix.
• • Renamed RayJobClusterConfig to ManagedClusterConfig - made more sense to me as the Cluster is managed by RayJobs, and RayJob being in the name was confusing

• Disable usage stats and didn't add ability to overwrite it. We initially made it changeable by the user as we didn't want to add new fields the user couldn't change. With 3.0 we have the opportunity to just disable the usage stats.

Verification steps

• There are no verification steps yet - testing and merging this is dependant on the GCS Fault Tolerance bug being fixed first so this PR will sit until then.

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@kryanbeane: This pull request references RHOAIENG-26482 which is a valid jira issue.

In response to this:

Jira link

RHOAIENG-26482

What changes have been made

• Added GCS fault tolerance, verification & future changes pending on the bug fix.
• Renamed RayJobClusterConfig to ManagedClusterConfig - made more sense to me as the Cluster is managed by RayJobs, and RayJob being in the name was confusing
• Disable usage stats and didn't add ability to overwrite it. We initially made it changeable by the user as we didn't want to add new fields the user couldn't change. With 3.0 we have the opportunity to just disable the usage stats.

Verification steps

• There are no verification steps yet - testing and merging this is dependant on the GCS Fault Tolerance bug being fixed first so this PR will sit until then.

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[chipspeak]

LGTM! My only thought (and it's purely devils advocate) is that I feel we may need to revisit the name of the RayJob's cluster config again in the future. It's a definite improvement but I feel the use of the use of managed might still raise cluster admin connotations rather than the 'managed by the RayJob' implication.

Again, the nittiest of nits. Just think it merits talking about!

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chipspeak

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [chipspeak]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kryanbeane]

/unhold

[kryanbeane]

Getting this merged to unblock other tickets. There is a bug ticket for the GCS Fault tolerance / ray head pod persistence ticket feature. We've added to fix the ray job GCS stuff as part of that ticket (should be the same fix)