JVM overrides don't work

[soenkeliebau]

Affected version

0.7.0

Current and expected behavior

When attempting to directy set jvm properties via config overrides like shown below the expected behavior is for these settings to be passed through to the jvm when starting the druid processes.

Instead these settings seem to be ignored and not used.

routers:
  roleGroups:
    default:
      config: {}
      configOverrides:
        jvm.config:
          veryimportantsetting: 300
      replicas: 1

Possible solution

No response

Additional context

No response

Environment

No response

Would you like to work on fixing this bug?

No response

[sbernauer]

@maltesander and I had a look.
We currently are not aware of a product that supports setting custom JVM args. We could easily add such functionality into e.g. druid-operator, we think it makes more sense to define a common struct in operator-rs, so that all jvm-based products jvm-overwrites are configured the same way.

Regarding the individual problem Mhd Redwan Alkurdi had

[he] says that the jvm is stuck at 256 MB and keeps falling over

The latest stable druid-operator version has the 256mb hard-coded.

druid-operator/rust/operator-binary/src/config.rs

Lines 29 to 30 in 35362af

	-Xms256m
	-Xmx256m

This cannot be changed.
In the currently nightly version 0.8.0-nightly we have introduced support for configurable compute resources for Druid in #298.
You can now configure the available cpu and memory for ever rolegroup (e.g. the coordinator)
The druid-operator will than automatically calculate the head size and set the corresponding jvm args for you:

druid-operator/rust/operator-binary/src/config.rs

Lines 29 to 30 in ec286c7

	-Xms{heap_in_mebi}m
	-Xmx{heap_in_mebi}m

So jvm arg overwrites are currently not possible but the 256mb can be adopted by using the druid-operator 0.8.0-nightly version.

stackablectl operator uninstall druid
kubectl apply -f https://raw.githubusercontent.com/stackabletech/druid-operator/main/deploy/manifests/crds.yaml
stackablectl operator install druid # will pull nightly version

The default coordinator memory limit is 2Gi, which will result in -Xms1638m and -Xmx1638m
If you further want to increase it you can use the following config (also see docs)

  coordinators:
    roleGroups:
      default:
        replicas: 2
        config:
          resources:
            cpu:
              min: 500m
              max: 2000m
            memory:
              limit: 8Gi

The Pods now have 8Gi and -Xms6553m. Please be aware that your Kubernetes has to be large enough to spin up such Pods (if not they will stay Pending).

[lfrancke]

This sounds like it shouldn't be expedite then but instead move back to an idea stage and will probably become an epic?

[soenkeliebau]

I'll close this in favor of stackabletech/operator-rs#489, yes.