Skip to content

[GLUTEN-12058][VL][MINOR] Disable GlutenAutoAdjustStageResourceProfile when off-heap memory is not enabled #12060

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wForget wants to merge 1 commit into
base: main
Choose a base branch
from
+6 −0
Open

[GLUTEN-12058][VL][MINOR] Disable GlutenAutoAdjustStageResourceProfile when off-heap memory is not enabled #12060

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .../src/main/scala/org/apache/spark/sql/execution/GlutenAutoAdjustStageResourceProfile.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,12 @@ case class GlutenAutoAdjustStageResourceProfile(glutenConf: GlutenConfig, spark:
if (!SQLConf.get.adaptiveExecutionEnabled) {
return plan
}
if (!sparkConf.getBoolean(GlutenCoreConfig.SPARK_OFFHEAP_ENABLED_KEY, defaultValue = false)) {
logWarning(
s"${GlutenCoreConfig.SPARK_OFFHEAP_ENABLED_KEY} is not enabled, " +
s"Gluten auto adjust stage resource profile will not work.")
return plan
}
// Starting here, the resource profile may differ between stages. Configure resource settings
// using the default profile to prevent any impact from the previous stage. If a new resource
// profile is applied, the settings will be updated accordingly.
Expand Down
Loading