Skip to content

Allow more control over enabling/disabling specific expressions #2344

@andygrove

Description

@andygrove

What is the problem the feature request solves?

Currently, we have a single config spark.comet.expression.allowIncompatible which enables or disables all incompatible expressions.

I would like to give the users the ability to enable or disable individual expressions, or to enable allowIncompat for specific expressions, dynamically using the Spark expression class name in the config key.

This will give us a workaround if we discover a bug in a particular expression implementation, such as #2338

Possible Examples:

spark.comet.expr.InitCap.enabled=true
spark.comet.expr.Rand.enabled=false
spark.comet.expr.Cast.allowIncompat=true

Describe the potential solution

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions