Description
GlutenConfig accepts a conf: SQLConf constructor parameter and stores it as an effectively final field. The widely-used factory GlutenConfig.get() passes SQLConf.get — the JVM thread-local static instance. Spark treats every key stored in that static instance as non-modifiable, so:
spark.conf.isModifiable("spark.gluten.*") → always false
even for keys that are genuinely dynamic (not marked .staticConf)
GlutenConfig should be refactored to get the value from SparkSession first so that it can correctly report the modifiable keys
Gluten version
None
Description
GlutenConfig accepts a
conf: SQLConfconstructor parameter and stores it as an effectively final field. The widely-used factory GlutenConfig.get() passesSQLConf.get— the JVM thread-local static instance. Spark treats every key stored in that static instance as non-modifiable, so:spark.conf.isModifiable("spark.gluten.*") → always falseeven for keys that are genuinely dynamic (not marked .staticConf)
GlutenConfig should be refactored to get the value from SparkSession first so that it can correctly report the modifiable keys
Gluten version
None