AllegroGraph version 7.0.4 Docker image, Kubernetes and Helm features extended

[Wrobert]

Development/Release Notes

Author: Robert Wydler; LinkedIn: Profile
Below, an excerpt of the attached "README.md" file (the original can be found under README-original.md):

Introduction

Based on

This is an extension of the product "AllegroGraph for Multi-Master Replication" and is based on https://github.com/franzinc/agraph-examples.

Goals

1. Creating a docker image like Docker and AllegroGraph 7.0.4 but for Multi-Master Replication in AllegroGraph 7.0.4 based on Welcome to AllegroGraph examples.

2. Run standardized Multi-master Replication in AllegroGraph 7.0.4 using Google Kubernetes Engine, GKE.

3. Make it available for semantic AI research studies with help of Google's Free Tier and AllegroGraph's Free Editon and to learn more about both products.

4. Setup a Kubernetes namespace agraph-ns, Kubernetes Update-Strategy and the Horizontal Pod AutoScaler, HPA e.g. using Google's Cluster Autoscaler.

Affected Parts

Therefore, only the clustering parts of docker (folders: clustering/kubernetes/mmr/agmmr/*), kubernetes and helm (folders: clustering/kubernetes/mmr/helm/*) has been changed.

Important Constraints !!!

Currently ...

• ... it's only a developer study version and it's not ready for production!
• ... the prototype was only running in a very small (free) Google GKE test environment.
• ... fundamental changes had been made (e.g. from StateFull copy read only pods to Deployment copy read only pods) and Kubernetes' Horizontal Pod Autoscaler had been defined added (hpa.yaml).

Known Issue

After deployment, sometimes it may happen, that the repositories are not reachable by the agraph-mmr-copy-loadbalancer. That's because the agraph-mmr-controlling stateful set has to run before the agraph-mmr-copy deployment. I haven't figured out yet, how to arrange a sequential start in Helm.

You can circumvent the issue by deleting the agraph-mmr-copy pods in the Google Cloud Console or you can perform a graceful pod deletion with the following command:

kubectl delete pods <pod>

The agraph-mmr-copy pods will be recreated automatically by the deploment.

---

ToDo's for a Reliable Release

1. Change Helm to always start the stateful set agraph-mmr-controlling before the agraph-mmr-copy deployment.

For a reliable release of the Docker Image 7.0.4 together with Kubernetes and Helm, it's important to validate and test ...

1. ... the functionality of copy-ssl.yaml pods after the change from StatefulSet to Kubernetes Deployment pods.
2. ... the reliability and horizontal scalability of the pods (hpa.yaml) and of the Kubernetes cluster (e.g. GKE) in an operational environment with high query loads.
3. ... the interaction of the state-oriented controller pods with the stateless copy pods.
4. ... etc.

---

Overview of Changes

... you can find more about in the attached README.md file.