Skip to content

[CORE] Preserve fallback tag for nodes without logicalLink in RemoveFallbackTagRule #12028

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
liujiayi771 merged 1 commit into from
May 9, 2026
Merged

[CORE] Preserve fallback tag for nodes without logicalLink in RemoveFallbackTagRule #12028

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion backends-velox/src/test/scala/org/apache/gluten/execution/FallbackSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,10 @@ class FallbackSuite extends VeloxWholeStageTransformerSuite with AdaptiveSparkPl
}
}
}
// Drain any pending events from previous tests before registering the listener.
// Spark's LiveListenerBus is async, so events posted but not yet dispatched will
// still be delivered to listeners added afterwards, contaminating `events` here.
GlutenSuiteUtils.waitUntilEmpty(spark.sparkContext)
spark.sparkContext.addSparkListener(listener)
withSQLConf(GlutenConfig.COLUMNAR_SORT_ENABLED.key -> "false") {
try {
Expand All @@ -345,7 +349,7 @@ class FallbackSuite extends VeloxWholeStageTransformerSuite with AdaptiveSparkPl
val fallbackReasons = events.flatMap(_.fallbackNodeToReason.values)
assert(fallbackReasons.nonEmpty)
assert(
fallbackReasons.exists(
fallbackReasons.forall(
_.contains("[FallbackByUserOptions] Validation failed on node Sort")))
} finally {
spark.sparkContext.removeSparkListener(listener)
Expand Down
16 changes: 15 additions & 1 deletion gluten-core/src/main/scala/org/apache/gluten/extension/columnar/FallbackTag.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,21 @@ object FallbackTags {

case class RemoveFallbackTagRule() extends Rule[SparkPlan] {
override def apply(plan: SparkPlan): SparkPlan = {
plan.foreach(FallbackTags.untag)
plan.foreach {
// Nodes without a logicalLink (e.g. SortExec/BroadcastExchange added by
// EnsureRequirements) have no place to forward the fallback reason via
// GlutenFallbackReporter. Keep the tag on the physical node so that
// GlutenExplainUtils.handleVanillaSparkPlan can still read it directly.
// We intentionally do NOT inject a synthetic logicalLink here, because
// AdaptiveSparkPlanExec.setLogicalLinkForNewQueryStage relies on these
// EnsureRequirements-generated nodes having no logicalLink in order to
// walk down to the real logical node; a synthetic link would poison
// AQE's stage-to-logical-plan mapping (breaking AQEPropagateEmptyRelation,
// ValidateSparkPlan-driven re-plans, etc.).
case p if FallbackTags.nonEmpty(p) && p.logicalLink.isEmpty =>
case p =>
FallbackTags.untag(p)
}
plan
}
}
Loading