[scalardb-cluster] Support cert-manager in ScalarDB Cluster chart #263

kota2and3kan · 2024-04-26T09:21:23Z

Description

This PR updates ScalarDB Cluster chart to support cert-manager.

Please take a look!

Related issues and/or PRs

https://github.com/scalar-labs/docs-internal-orchestration/pull/24
- You can see the documents that describe how to use these new features in this PR.
[envoy] Support cert-manager in Scalar Envoy chart #262
- If you use self-signed CA, this chart deploy self-signed CA. And, Scalar Envoy chart refer it to generate certificate for Envoy.

Changes made

Mainly, this PR adds/updates the following features (manifests):

Deploy a Certificate resource for cert-manager by using charts/scalardb-cluster/templates/scalardb-cluster/certmanager.yaml.
Deploy self-signed CA and create certificate for ScalarDB Cluster if you don't specify any Issuer resources in issuerRef configuration.
- If you deploy Scalar Envoy as a sub chart of this ScalarDB Cluster chart without issuerRef configuration, the Envoy chart will use self-signed CA that is deployed by this chart to create certificate for Envoy.
```
[ScalarDB Cluster chart (this chart)] ---(Deploy)---> [Issuer (self-signed CA)] <---(Refer via issuerRef configuration)--- [Certificate resource] <---(Deploy)--- [Envoy chart]
```
Update deployment.spec.template.spec.volumes and deployment.spec.template.spec.containers.volumeMounts to remove subPath configuration.
- Cert-manager creates a secret resource that includes private key and certificate, and we use them by mounting them on pods. And, cert-manager automatically renew (update) the certificate in the secret resource before the certificate will be expired. However, if we use subPath to specify the mounted file name, Kubernetes does not apply the changes of the secret resource. So, to take advantage of cert-manager's automatically renew feature, we should not use subPath. This is because I removed subPath from the volumeMounts configuration.
Update file names of private key and certificate.
- From the perspective of cert-manager specification (strictly, TLS Secrets specification of Kubernetes), we must use the fixed names for private key and certificate files. This is because cert-manager creates a secret resource that includes private key and certificate files with fixed names based on the TLS Secrets specification. So, I updated the file names in existing manifests.
  - cert.pem -> tls.crt
  - key.pem -> tls.key
  - ca.pem -> ca.crt

Checklist

I have commented my code, particularly in hard-to-understand areas.
I have updated the documentation to reflect the changes.
Any remaining open issues linked to this PR are documented and up-to-date (Jira, GitHub, etc.).
Tests (unit, integration, etc.) have been added for the changes.
My changes generate no new warnings.
Any dependent changes in other PRs have been merged and published.

Additional notes (optional)

N/A

Release notes

Support cert-manager in ScalarDB Cluster chart. You can manage private key and certificate for TLS connections by using cert-manager.

kota2and3kan · 2024-04-26T09:23:54Z

charts/scalardb-cluster/templates/scalardb-cluster/certmanager.yaml

@@ -0,0 +1,83 @@
+{{- if .Values.scalardbCluster.tls.certManager.enabled }}
+{{- if not .Values.scalardbCluster.tls.certManager.issuerRef }}


If you don't specify Issuer in the issurRef configuration, this chart deploys self-signed CA to generate certificates.

If you specify the issuerRef configuration, this chart doesn't deploy self-signed CA.

kota2and3kan · 2024-04-26T09:31:11Z

charts/scalardb-cluster/templates/scalardb-cluster/certmanager.yaml

+  issuerRef:
+    {{- if .Values.scalardbCluster.tls.certManager.issuerRef }}
+    {{- toYaml .Values.scalardbCluster.tls.certManager.issuerRef | nindent 4 }}
+    {{- else }}
+    name: {{ include "scalardb-cluster.fullname" . }}-ca-issuer
+    {{- end }}


If you specify issuerRef, this chart uses that configuration in the Certificate resource. If you don't specify issuerRef, this chart uses a self-signed CA that is deployed by this chart.

kota2and3kan · 2024-04-26T09:31:33Z

charts/scalardb-cluster/templates/scalardb-cluster/deployment.yaml

-                {{- if .Values.scalardbCluster.tls.caRootCertSecret }}
-                - -tls-ca-cert=/tls/certs/ca-root-cert.pem
-                {{- end }}
+                - -tls-ca-cert=/tls/scalardb-cluster/certs/ca.crt


When you enable TLS, the -tls-ca-cert option is mandatory. In other words, {{- if .Values.scalardbCluster.tls.caRootCertSecret }} does not make sense. So, I removed that unnecessary condition.

kota2and3kan · 2024-04-26T09:34:31Z

charts/scalardb-cluster/templates/scalardb-cluster/deployment.yaml

+            {{- if .Values.scalardbCluster.tls.enabled }}
+            - name: scalardb-cluster-tls-volume
+              mountPath: /tls/scalardb-cluster/certs


I combine all private keys and certificates into one volume to remove the subPath configuration.

kota2and3kan · 2024-04-26T09:35:48Z

charts/scalardb-cluster/templates/scalardb-cluster/deployment.yaml

+        {{- if and .Values.scalardbCluster.tls.enabled .Values.scalardbCluster.tls.certManager.enabled }}
+        - name: scalardb-cluster-tls-volume
          secret:
-            secretName: {{ .Values.scalardbCluster.tls.caRootCertSecret }}
+            secretName: {{ include "scalardb-cluster.fullname" . }}-tls-cert


If you use cert-manager, it creates one secret resource that includes all private keys and certificates. So, it's enough to mount that one secret here.

kota2and3kan · 2024-04-26T09:39:11Z

charts/scalardb-cluster/templates/scalardb-cluster/deployment.yaml

+        {{- if and (.Values.scalardbCluster.tls.enabled) (not .Values.scalardbCluster.tls.certManager.enabled) }}
+        - name: scalardb-cluster-tls-volume
+          projected:
+            sources:
+              - secret:
+                  name: {{ .Values.scalardbCluster.tls.caRootCertSecret }}
+              - secret:
+                  name: {{ .Values.scalardbCluster.tls.certChainSecret }}
+              - secret:
+                  name: {{ .Values.scalardbCluster.tls.privateKeySecret }}


If you don't use cert-manager (and you want to manage each file separately), you have to create three secrets for the private key, certificate, and CA certificate.

In this case, I want to combine them into one volume to mount them easily under the same directory in the container. So, I use projected volume here.

kota2and3kan · 2024-04-26T09:43:48Z

charts/scalardb-cluster/values.yaml

+      dnsNames:
+        - localhost


I set localhost as a default value of the dnsNames. This is because:

The grpc_health_probe command accesses ScalarDB Cluster by specifying localhost like -addr=localhost:60053.

The grpc_health_probe command verifies the certificate by using Subject Alternative Name (SAN) instead of Common Name (CN) of the certificate. In other words, at least one SAN configuration is necessary. (I will update the document in another PR.)

komamitsu

LGTM! 👍
I didn't thoroughly review it, though.

brfrn169

LGTM! Thank you!

feeblefakie

LGTM! Thank you!

[scalardb-cluster] Support cert-manager

f840ac5

kota2and3kan added improvement scalardb cluster PR for ScalarDB Cluster chart labels Apr 26, 2024

kota2and3kan self-assigned this Apr 26, 2024

kota2and3kan commented Apr 26, 2024

View reviewed changes

kota2and3kan requested review from komamitsu, superbrothers, brfrn169 and feeblefakie April 30, 2024 07:11

kota2and3kan mentioned this pull request Apr 30, 2024

[scalardb-cluster] Support TLS for ServiceMonitor in ScalarDB Cluster chart #266

Merged

6 tasks

komamitsu approved these changes May 7, 2024

View reviewed changes

brfrn169 approved these changes May 9, 2024

View reviewed changes

superbrothers approved these changes May 12, 2024

View reviewed changes

kota2and3kan added 2 commits May 15, 2024 18:58

Update default duration of certificate

8eb5d03

Disable self-signed CA by default

d5bb8c6

feeblefakie approved these changes May 31, 2024

View reviewed changes

feeblefakie merged commit 503e5a4 into main May 31, 2024
14 checks passed

feeblefakie deleted the support-cert-manager-scalardb-cluster branch May 31, 2024 01:31

kota2and3kan added a commit that referenced this pull request Jun 10, 2024

[scalardb-cluster] Support cert-manager in ScalarDB Cluster chart (#263)

0e51ce3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[scalardb-cluster] Support cert-manager in ScalarDB Cluster chart #263

[scalardb-cluster] Support cert-manager in ScalarDB Cluster chart #263

kota2and3kan commented Apr 26, 2024 •

edited

Loading

kota2and3kan Apr 26, 2024

kota2and3kan Apr 26, 2024

kota2and3kan Apr 26, 2024

kota2and3kan Apr 26, 2024

kota2and3kan Apr 26, 2024

kota2and3kan Apr 26, 2024

kota2and3kan Apr 26, 2024

komamitsu left a comment

brfrn169 left a comment

feeblefakie left a comment

		@@ -0,0 +1,83 @@
		{{- if .Values.scalardbCluster.tls.certManager.enabled }}
		{{- if not .Values.scalardbCluster.tls.certManager.issuerRef }}

[scalardb-cluster] Support cert-manager in ScalarDB Cluster chart #263

[scalardb-cluster] Support cert-manager in ScalarDB Cluster chart #263

Conversation

kota2and3kan commented Apr 26, 2024 • edited Loading

Description

Related issues and/or PRs

Changes made

Checklist

Additional notes (optional)

Release notes

kota2and3kan Apr 26, 2024

Choose a reason for hiding this comment

kota2and3kan Apr 26, 2024

Choose a reason for hiding this comment

kota2and3kan Apr 26, 2024

Choose a reason for hiding this comment

kota2and3kan Apr 26, 2024

Choose a reason for hiding this comment

kota2and3kan Apr 26, 2024

Choose a reason for hiding this comment

kota2and3kan Apr 26, 2024

Choose a reason for hiding this comment

kota2and3kan Apr 26, 2024

Choose a reason for hiding this comment

komamitsu left a comment

Choose a reason for hiding this comment

brfrn169 left a comment

Choose a reason for hiding this comment

feeblefakie left a comment

Choose a reason for hiding this comment

kota2and3kan commented Apr 26, 2024 •

edited

Loading