-
Installation and overall view of Kafka on OCP
-
Management tools included in AMQ streams in order to
-
create,
-
read,
-
update
-
delete topic in kafka (via UI or command line tool)
-
-
How to manage AMQ Streams (KAFKA) cluster
-
overall view of monitoring tool included in AMQ streams
-
security (kerberos, oauth2).
-
Architecture choices (single central cluster, collaborating projects, topic discovery, etc.)
-
Storage Available
-
Cluster Topology/Networking
-
ImageStreams/Images in cluster
-
Security/Auhorization
-
A user with cluster-admin role needs to be used, for example, system:admin.
-
-
oc tools
-
Download install_and_examples.zip
-
Add to OCP Cluster Registry the following images
-
registry.access.redhat.com/amqstreams-1/amqstreams10-clusteroperator-openshift:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-useroperator-openshift:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-topicoperator-openshift:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-entityoperatorstunnel-openshift:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-kafkamirrormaker:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-zookeeper-openshift:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-kafka-openshift:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-kafkaconnect-openshift:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-kafkaconnects2i-openshift:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-kafkainit-openshift:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-zookeeperstunnel-openshift:1.0.0
-
registry.access.redhat.com/amqstreams-1/amqstreams10-kafkastunnel-openshift:1.0.0
-
-
Ensure imagestreams are installed in openshift namespace
-
Run a script to install official imagestreams in openshift namespace such as this install-official-is-imagestreams.sh as cluster admin
-
To avoid minishift issues with registry authentication download 3.10 images:https://github.com/minishift/minishift/blob/master/addons/xpaas/v3.10/xpaas-streams/fis-image-streams.json also available at env/minishift-is.json
oc create -f env/ -n openshift
-
Image to be present in registry registry.access.redhat.com/jboss-fuse-6/fis-java-openshift:2.0
-
Scalability and performance
-
Designed for horizontal scalability
-
-
Message ordering guarantee
-
At partition level
-
-
Message rewind/replay
-
"Long term" storage
-
Allows to reconstruct application state by replaying the messages
-
Combined with compacted topics allows to use Kafka as key-value store
-
-
Cluster Operator: Responsible for deploying and managing Apache Kafka clusters within OpenShift cluster.
-
Cluster Operator watches the following resources:
-
A Kafka resource for the Kafka cluster.
-
A KafkaConnect resource for the Kafka Connect cluster.
-
A KafkaConnectS2I resource for the Kafka Connect cluster with Source2Image support.
-
-
To deploy a Kafka cluster, a Kafka resource with the cluster configuration has to be created within the OpenShift cluster. Based on what is declared inside of the Kafka resource, the Cluster Operator deploys a corresponding Kafka cluste
-
-
Topic Operator: Responsible for managing Kafka topics within a Kafka cluster running within OpenShift cluster.
-
User Operator: Responsible for managing Kafka users within a Kafka cluster running within OpenShift cluster.
minishift profile set amq-streams-workshop
$ minishift profile list - amq-streams-workshop Does Not Exist (Active) - minishift Stopped
minishift config set cpus 2 minishift config set memory 6144 minishift addons disable anyuid
minishift start
-
sudoer for docker service
sudo groupadd docker usermod -aG docker $USER
-
Issue with redhat.io registry authentication
TBD
-
Issue with profile for workshop
See kvm machines sudo virsh list --all ----------------------------- Id Name State --------------------------------------- 18 amq-streams-workshop running - minishift shut off
-
shutdown (if minishift hasn’t shutdown virtual machine properly)
sudo virsh shutdown amq-streams-workshop
-
Undefine
sudo virsh undefine 9b81c33f-739e-40ba-a0bc-b84389bf5fa7
-