[CORE][VL] No Iceberg-specific switch to enable/disable Iceberg read and write offload

[jackylee-ch]

Description

Gluten currently has no Iceberg-specific configuration entry at all — neither for read nor for
write. Searching the whole repository for buildConf / buildStaticConf / registerConf keys
containing iceberg returns nothing.

As a result, operators who need to turn Iceberg offload off (bad data, a validation gap, a
regression after an upgrade) have to reach for a switch that is far broader than Iceberg:

Path	Only way to disable today	Problem
Read	spark.gluten.sql.columnar.batchscan	Generic DSv2 switch. Turning it off also falls back Paimon, Hudi and every other batch scan.
Write	spark.gluten.sql.enable.enhancedFeatures	A bundled flag (VeloxConfig.scala), documented as "iceberg native write and other features". Default true.

Details

Read. OffloadIcebergScan (gluten-iceberg/.../extension/OffloadIcebergScan.scala) is gated
only by Validators.FallbackByUserOptions:

case p: BatchScanExec if !glutenConf.enableColumnarBatchScan => fail(p)

There is no way to express "fall back Iceberg scans but keep other DSv2 scans offloaded".

Write. The five OffloadIcebergWrite rules
(backends-velox/src-iceberg/.../extension/OffloadIcebergWrite.scala) are gated by
VeloxBackend.supportAppendDataExec() and friends, all of which return
VeloxConfig.get.enableEnhancedFeatures(). Two consequences:

1. The enhancedFeatures flag conflates "was the native lib built with enhanced features" with
   "does the user want Iceberg native write". Today Iceberg write happens to be its only member, but
   the doc string already says "and other features" — a second member removes the ability to target
   Iceberg alone.
2. The build-time half of that gate is already gone: isEnhancedFeaturesEnabled in
   cpp/velox/jni/VeloxJniWrapper.cc now unconditionally returns true (the
   #ifdef GLUTEN_ENABLE_ENHANCED_FEATURES was dropped). Iceberg native write is therefore on by
   default, with the bundled SQL conf as the only remaining gate. test(iceberg): Revamp Iceberg tests #12660 similarly notes "Iceberg is
   not dependent on the enhanced tag anymore".
3. supportAppendDataExec() is also read once at rule-injection time
   (VeloxRuleApi.scala, to decide whether to inject RewriteCreateTableAsSelect), so flipping
   enhancedFeatures at runtime only affects the validator half — it is not a clean kill switch.

Iceberg write validation carries a long list of restrictions (Parquet only, no sort order, no
UUID/FIXED, no merge schema, no accept-any-schema, no float/double or nested partition columns,
no incompatible column names — see IcebergWriteExec.doValidateInternal). A write-path problem
means bad files rather than a slow query, so a switch that disables only Iceberg write, without
touching anything else, is worth having on its own.

Proposal

Add one switch per direction, both defaulting to true so behaviour is unchanged:

spark.gluten.sql.columnar.iceberg.enableNativeRead
spark.gluten.sql.columnar.backend.velox.iceberg.enableNativeWrite

The asymmetry in the key names is deliberate and follows where the code lives:

• OffloadIcebergScan is in gluten-iceberg/ and is shared by both backends
  (VeloxIcebergComponent and CHIcebergComponent both call it), so the read key must not carry a
  backend.velox prefix.
• OffloadIcebergWrite is in backends-velox/src-iceberg/ and is Velox-only. The write key mirrors
  the existing Delta precedent, spark.gluten.sql.columnar.backend.velox.delta.enableNativeWrite
  (VeloxDeltaConfig), which is checked with an early return in OffloadDeltaCommand.

Checks belong inside the offload rules (not at injection time), so the switches stay dynamic.
The write switch should be AND-ed with enhancedFeatures rather than replacing it.

One known gap to decide on: AllVeloxConfiguration only walks VeloxConfig.allEntries, so
component-level config objects are not picked up by the docs generator — which is why
VeloxDeltaConfig's key is absent from docs/velox-configuration.md today. Either accept the gap
or extend the generator to cover component configs.

Related but distinct: #11703 (mapping Iceberg table properties to Velox write configs), #8953
(Iceberg support tracker).

This enhancement description was written with the assistance of AI tooling.

Gluten version

main branch