[Bug]: Exposing a nodeport type listener does not work on DockerDesktop

[tdiesler]

Bug Description

I have this NodePort listener definition on local single node DockerDesktop ...

      # Accessing Kafka using node ports
      # https://strimzi.io/docs/operators/latest/deploying#proc-accessing-kafka-using-nodeports-str
      - name: external
        port: 9095
        type: nodeport
        tls: false

When I query the cluster status I get ...

kubectl -n kafka get kafka cluster -o=yaml

  listeners:
  - addresses:
    - host: cluster-kafka-bootstrap.kafka.svc
      port: 9092
    bootstrapServers: cluster-kafka-bootstrap.kafka.svc:9092
    name: plain
  - addresses:
    - host: cluster-kafka-bootstrap.kafka.svc
      port: 9093
    bootstrapServers: cluster-kafka-bootstrap.kafka.svc:9093
    certificates: ...
    name: tls
  - addresses:
    - host: 192.168.65.3
      port: 30172
    bootstrapServers: 192.168.65.3:30172
    name: external

and these services

kubectl -n kafka get svc

NAME                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                        AGE
cluster-dual-role-external-0       NodePort    10.99.47.223    <none>        9095:32265/TCP                                 12m
cluster-kafka-bootstrap            ClusterIP   10.96.83.142    <none>        9091/TCP,9092/TCP,9093/TCP                     12m
cluster-kafka-brokers              ClusterIP   None            <none>        9090/TCP,9091/TCP,8443/TCP,9092/TCP,9093/TCP   12m
cluster-kafka-external-bootstrap   NodePort    10.109.30.118   <none>        9095:30172/TCP                                 12m

Eventually I'd like to connect to the Kafka listener from an external Java client. Neither the above mentioned 192.168.65.3:30172 nor cluster-kafka-external-bootstrap:30172 works with ...

kafka-topics --list --bootstrap-server ${kafaHost}:${nodePort}

I tried mapping cluster-kafka-external-bootstrap to my host IP in /etc/hosts

Does this look right to you? What else could I try?

Steps to reproduce

Running operator on DockerDesktop MacOS, then do the above.

Expected behavior

Ideally, the single node strimzi cluster should work as it does with this bitnami deployment

---
# Kafka Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka
  labels:
    app.kubernetes.io/name: kafka
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: kafka
  template:
    metadata:
      labels:
        app.kubernetes.io/name: kafka
    spec:
      containers:
        - name: kafka
          image: bitnami/kafka:3.9.0
          ports:
            - containerPort: 9092
            - containerPort: 9093
          env:
            # KRaft settings
            - name: KAFKA_CFG_NODE_ID
              value: "0"
            - name: KAFKA_CFG_PROCESS_ROLES
              value: "controller,broker"
            - name: KAFKA_CFG_CONTROLLER_QUORUM_VOTERS
              value: "0@kafka-ctrl:9093"
            # Listeners & Security
            - name: KAFKA_CFG_LISTENERS
              value: "PLAINTEXT://:9092,CONTROLLER://:9093"
            - name: KAFKA_CFG_ADVERTISED_LISTENERS
              value: "PLAINTEXT://<my-external-ip-or-127.0.0.1>:30092"
            - name: KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP
              value: "CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT"
            - name: KAFKA_CFG_CONTROLLER_LISTENER_NAMES
              value: "CONTROLLER"

---
# Kafka Controller Service
apiVersion: v1
kind: Service
metadata:
  name: kafka-ctrl
  labels:
    app.kubernetes.io/name: kafka-ctrl
spec:
  type: ClusterIP
  ports:
    - port: 9093
      targetPort: 9093
  selector:
    app.kubernetes.io/name: kafka

---
# Kafka Listener Service
apiVersion: v1
kind: Service
metadata:
  name: kafka
  labels:
    app.kubernetes.io/name: kafka
spec:
  type: NodePort
  ports:
    - port: 9092
      targetPort: 9092
      nodePort: 30092
  selector:
    app.kubernetes.io/name: kafka

Strimzi version

0.45.0

Kubernetes version

Kubernetes 1.30.5

Installation method

Strimzi QuickStart

Infrastructure

DockerDesktop on MacOS M2

Configuration files and logs

No response

Additional context

No response

[scholzj]

This is not a Strimzi bug but a limitation of your environment. You can follow this blog post series - https://strimzi.io/blog/2019/04/17/accessing-kafka-part-1/ - and our documentation to understand how Kafka works and uses its discovery protocol and how you can configure it if needed.

[tdiesler]

As mentioned above, the bitnami/kafka image works. Likewise, all other services that I exposed via NodePort so far worked just fine. To me it seems very much related to the way strimzi exposes its nodeport services.

$ kafka-topics --create \
  --bootstrap-server localhost:30092 \
  --replication-factor 1 \
  --partitions 1 \
  --topic test

Created topic test.
$ echo 'Hello Kermit!' | kafka-console-producer --broker-list localhost:30092 --topic test
$ kafka-console-consumer --bootstrap-server localhost:30092 --topic test --from-beginning
Hello Kermit!

[tdiesler]

Here the my resource descriptor

# Original resource descriptor
# https://github.com/strimzi/strimzi-kafka-operator/blob/main/examples/kafka/kraft/kafka-single-node.yaml

# Kafka NodePool
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: dual-role
  labels:
    strimzi.io/cluster: cluster
spec:
  replicas: 1
  roles:
    - controller
    - broker
  storage:
    type: jbod
    volumes:
      - id: 0
        type: persistent-claim
        size: 100Gi
        deleteClaim: false
        kraftMetadata: shared

# Kafka Cluster
---
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: cluster
  annotations:
    strimzi.io/node-pools: enabled
    strimzi.io/kraft: enabled
spec:
  kafka:
    version: 3.9.0
    metadataVersion: 3.9-IV0
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      # Accessing Kafka using node ports
      # https://strimzi.io/docs/operators/latest/deploying#proc-accessing-kafka-using-nodeports-str
      - name: external
        port: 9095
        type: nodeport
        tls: false
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      default.replication.factor: 1
      min.insync.replicas: 1
  entityOperator:
    topicOperator: {}
    userOperator: {}

[tdiesler]

FWIW, here is plain strimzi-kafka deployment that works with nodeport on DockerDesktop
https://github.com/tdiesler/camel-cloud-examples/blob/main/camel-main/kafka-oauth/helm/templates/kafka.yaml