Added restproxy demo and c3 to clusterlinking demo

README.md

@@ -174,20 +174,21 @@ To run a lightweight a **Default** environment, follow the next steps:
 
 To add more use cases, we are leveraging the docker profiles. 
 
-To run replicator scenario, i.e. run `start.sh --profile replicator`. 
+Example, to run replicator scenario, run `start.sh --profile replicator`. 
 
 It's possible to combine profiles as well, i.e. `start.sh --profile schema-registry --profile ksqldb`.
 
 Currently supported profiles:
 - _clusterlinking_: add Cluster Linking between _kafka1-kafka2-kafka3-kafka4_ and a new cluster with 1 broker _broker-dest_
-- _connect_: it will add Kafka Connect with a datagen source connector and a file sink connector.
+- _connect_: it will add Kafka Connect with a datagen source connector and a file sink connector. It requires _schema-registry_ profile.
 - _consumer_: it will add a demo application implemented with Spring with full client metrics
 - _consumer-minimal_: it will add a demo application implemented with Spring with a limited number of client metrics
 - _jr_: it will add [JR](https://jrnd.io/) to generate random traffic for kafka.
 - _ksqldb_: it will add ksqldb server. It requires _schema-registry_ profile.
 - _kstream_: it will add a demo stateful kafka streams application with full client metrics (_TRACE_ level selected)
 - _kui_: it will add an instance of _kafka-ui_ for topics data visualizion (available on port 18080).
 - _replicator_: it will add a Kafka connect cluster with Confluent Replicator between _kafka1-kafka2-kafka3-kafka4_ and a new cluster with 1 broker _broker-dest_
+- _restproxy_: it will add Confluent Rest Proxy. It requires _schema-registry_ profile.
 - _schema-registry_: it will add Confluent Schema Registry.
 - _schema-registry-primary-secondary_: it will add 2 Confluent Schema Registry, primary and secondary.
 - _tieredstroage_: it will configure Confluent Platform to use Confluent Tiered Storage and a compatible S3 storage.

dev-toolkit/docker-compose.clusterlinking.yaml

@@ -36,3 +36,28 @@ services:
     volumes:
       - $PWD/jmx-exporter/jmx_prometheus_javaagent-1.1.0.jar:/tmp/jmx_prometheus_javaagent-1.1.0.jar
       - $PWD/jmx-exporter/kafka_broker.yml:/tmp/kafka_config.yml
+
+  control-center:
+    image: confluentinc/cp-enterprise-control-center:${CFLT_TAG}
+    hostname: control-center
+    container_name: control-center
+    healthcheck:
+      start_period: 10s
+      interval: 10s
+      retries: 20
+      test: curl --fail --silent http://control-center:9021 --output /dev/null || exit 1
+    ports:
+      - 9021:9021
+    environment:
+      CONTROL_CENTER_BOOTSTRAP_SERVERS: PLAINTEXT://broker-replicator-dst:29092
+      CONTROL_CENTER_REPLICATION_FACTOR: 1
+      CONTROL_CENTER_INTERNAL_TOPICS_PARTITIONS: 1
+      CONTROL_CENTER_MONITORING_INTERCEPTOR_TOPIC_REPLICATION: 1
+      CONTROL_CENTER_MONITORING_INTERCEPTOR_TOPIC_PARTITIONS: 1
+      CONTROL_CENTER_METRICS_TOPIC_REPLICATION: 1
+      CONTROL_CENTER_METRICS_TOPIC_PARTITIONS: 1
+      CONTROL_CENTER_STREAMS_CACHE_MAX_BYTES_BUFFERING: 100000000
+      CONTROL_CENTER_STREAMS_CONSUMER_REQUEST_TIMEOUT_MS: "960032"
+      CONTROL_CENTER_STREAMS_NUM_STREAM_THREADS: 1
+      CONTROL_CENTER_REST_LISTENERS: http://0.0.0.0:9021
+      PORT: 9021

dev-toolkit/docker-compose.connect.yaml

@@ -6,10 +6,7 @@ services:
     hostname: connect
     container_name: connect
     depends_on:
-      - kafka1
-      - kafka2
-      - kafka3
-      - kafka4
+      - schemaregistry
     ports:
       - "8083:8083"
     environment:

dev-toolkit/docker-compose.restproxy.yaml

@@ -0,0 +1,20 @@
+services:
+  restproxy:
+    image: confluentinc/cp-kafka-rest:${CFLT_TAG}
+    profiles:
+      - restproxy
+    hostname: restproxy
+    container_name: restproxy
+    depends_on:
+      - schemaregistry
+    ports:
+      - "8086:8086"
+    environment:
+      KAFKA_REST_HOST_NAME: restproxy
+      KAFKA_REST_BOOTSTRAP_SERVERS: kafka1:29092,kafka2:29092,kafka3:29092,kafka4:29092
+      KAFKA_REST_LISTENERS: http://0.0.0.0:8086
+      KAFKA_REST_SCHEMA_REGISTRY_URL: http://schemaregistry:8081
+      KAFKAREST_OPTS: "-javaagent:/tmp/jmx_prometheus_javaagent-1.1.0.jar=1234:/tmp/confluent_rest.yml"
+    volumes:
+      - $PWD/jmx-exporter/jmx_prometheus_javaagent-1.1.0.jar:/tmp/jmx_prometheus_javaagent-1.1.0.jar
+      - $PWD/jmx-exporter/confluent_rest.yml:/tmp/confluent_rest.yml

dev-toolkit/start.sh

@@ -169,6 +169,7 @@ DOCKER_COMPOSE_FILES="-f docker-compose.yaml \
   -f docker-compose.connect.yaml \
   -f docker-compose.kstream.yaml \
   -f docker-compose.kui.yaml \
+  -f docker-compose.restproxy.yaml
 "
 
 # if docker_args contains tieredstorage, then add the tieredstorage file

dev-toolkit/stop.sh

@@ -30,6 +30,7 @@ $DOCKER_COMPOSE_CMD \
     --profile clusterlinking \
     --profile kstream \
     --profile kui \
+    --profile restproxy \
     -f docker-compose.yaml \
     -f docker-compose.replicator.yaml \
     -f docker-compose.schema-registry.yaml \
@@ -42,6 +43,7 @@ $DOCKER_COMPOSE_CMD \
     -f docker-compose.connect.yaml \
     -f docker-compose.kstream.yaml \
     -f docker-compose.kui.yaml \
+    -f docker-compose.restproxy.yaml \
     down -v
 rm -rf jmx-exporter
 rm -rf assets

jmxexporter-prometheus-grafana/README.md

@@ -29,14 +29,14 @@ List of provided dashboards:
  - [Schema Registry cluster _(cp-demo,dev-toolkit)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#schema-registry-cluster)
  - [Kafka Connect cluster _(cp-demo,dev-toolkit)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#kafka-connect-cluster)
  - [ksqlDB cluster _(cp-demo,dev-toolkit)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#ksqldb-cluster)
- - [Kafka streams _(cp-demo)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#kafka-streams)
- - [Kafka streams RocksDB _(cp-demo)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#kafka-streams-rocksdb)
+ - [Kafka streams _(cp-demo,dev-toolkit)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#kafka-streams)
+ - [Kafka streams RocksDB _(cp-demo,dev-toolkit)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#kafka-streams-rocksdb)
  - [Librdkafka based client _(cp-demo)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#Librdkafka)
  - [Oracle CDC source Connector](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#oracle-cdc-source-connector)
  - [Debezium source Connectors](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#debezium-cdc-source-connectors)
  - [Mongo source and sink Connectors](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#mongo-source-and-sink-connectors)
  - [Cluster Linking _(dev-toolkit)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#cluster-linking)
- - [Rest Proxy _(cp-demo)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#rest-proxy)
+ - [Rest Proxy _(cp-demo,dev-toolkit)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#rest-proxy)
  - [KRaft overview _(dev-toolkit)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#kraft)
  - [Confluent RBAC _(cp-demo)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#rbac)
  - [Confluent Replicator _(dev-toolkit)_](https://github.com/confluentinc/jmx-monitoring-stacks/blob/main/jmxexporter-prometheus-grafana/README.md#replicator)