Update Kafka images on the CI to 4.0 #3092
Description
Since Docker images for Kafka 4.0 were published, we're having issues with our configuration for Kafka on the CI. There's a series of errors related to environment variables.
Failure on the CI w/o changes:
kafka 19:03:02.02 ERROR ==> Kafka requires at least one process role to be set. Set the environment variable KAFKA_CFG_PROCESS_ROLES to configure the process roles for Kafka.
/opt/bitnami/scripts/libkafka.sh: line 350: KAFKA_CFG_PROCESS_ROLES: unbound variable
kafka 19:03:02.02 ERROR ==> KRaft mode requires an unique node.id, please set the environment variable KAFKA_CFG_NODE_ID
When KAFKA_CFG_PROCESS_ROLES was set to controller,broker, the documented value in the Docker image:
kafka 19:19:19.24 ERROR ==> Role 'controller' enabled but listener CONTROLLER not found in KAFKA_CFG_LISTENERS.
kafka 19:19:19.24 ERROR ==> Role 'controller' enabled but environment variable KAFKA_CFG_CONTROLLER_QUORUM_VOTERS was not provided.
kafka 19:19:19.24 ERROR ==> KRaft mode requires an unique node.id, please set the environment variable KAFKA_CFG_NODE_ID
When I tried setting the value of KAFKA_CFG_PROCESS_ROLES to '' based on this doc since we use Zookeeper:
kafka 19:38:02.92 ERROR ==> Kafka requires at least one process role to be set. Set the environment variable KAFKA_CFG_PROCESS_ROLES to configure the process roles for Kafka.
/opt/bitnami/scripts/libkafka.sh: line 350: KAFKA_CFG_PROCESS_ROLES: unbound variable
kafka 19:38:02.92 ERROR ==> KRaft mode requires an unique node.id, please set the environment variable KAFKA_CFG_NODE_ID
For now, the CI is locked to the last available 3.x version at the time of opening this issue, version 3.9.0.
These experiments can be found in #3091