Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[scalardb-cluster] Support cert-manager in ScalarDB Cluster chart #263

Merged
merged 3 commits into from
May 31, 2024
Merged

[scalardb-cluster] Support cert-manager in ScalarDB Cluster chart #263

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f840ac5
[scalardb-cluster] Support cert-manager
kota2and3kan Apr 26, 2024
8eb5d03
Update default duration of certificate
kota2and3kan May 15, 2024
d5bb8c6
Disable self-signed CA by default
kota2and3kan May 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions charts/scalardb-cluster/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,20 @@ Current chart version is `2.0.0-SNAPSHOT`
| scalardbCluster.strategy.rollingUpdate.maxSurge | string | `"25%"` | The number of pods that can be created above the desired amount of pods during an update |
| scalardbCluster.strategy.rollingUpdate.maxUnavailable | string | `"25%"` | The number of pods that can be unavailable during the update process |
| scalardbCluster.strategy.type | string | `"RollingUpdate"` | New pods are added gradually, and old pods are terminated gradually, e.g: Recreate or RollingUpdate |
| scalardbCluster.tls.caRootCertSecret | string | `""` | Name of the Secret containing the custom CA root certificate for TLS communication. |
| scalardbCluster.tls.certChainSecret | string | `""` | Name of the Secret containing the certificate chain file used for TLS communication. |
| scalardbCluster.tls.caRootCertSecret | string | `""` | Name of the Secret containing the custom CA root certificate for TLS communication. |
| scalardbCluster.tls.certChainSecret | string | `""` | Name of the Secret containing the certificate chain file used for TLS communication. |
| scalardbCluster.tls.certManager.dnsNames | list | `["localhost"]` | Subject Alternative Name (SAN) of a certificate. |
| scalardbCluster.tls.certManager.duration | string | `"8760h0m0s"` | Duration of a certificate. |
| scalardbCluster.tls.certManager.enabled | bool | `false` | Use cert-manager to manage private key and certificate files. |
| scalardbCluster.tls.certManager.issuerRef | object | `{}` | Issuer references of cert-manager. |
| scalardbCluster.tls.certManager.privateKey | object | `{"algorithm":"ECDSA","encoding":"PKCS1","size":256}` | Configuration of a private key. |
| scalardbCluster.tls.certManager.renewBefore | string | `"360h0m0s"` | How long before expiry a certificate should be renewed. |
| scalardbCluster.tls.certManager.selfSigned | object | `{"caRootCert":{"duration":"8760h0m0s","renewBefore":"360h0m0s"},"enabled":false}` | Configuration of a certificate for self-signed CA. |
| scalardbCluster.tls.certManager.selfSigned.caRootCert.duration | string | `"8760h0m0s"` | Duration of a self-signed CA certificate. |
| scalardbCluster.tls.certManager.selfSigned.caRootCert.renewBefore | string | `"360h0m0s"` | How long before expiry a self-signed CA certificate should be renewed. |
| scalardbCluster.tls.certManager.selfSigned.enabled | bool | `false` | Use self-signed CA. |
| scalardbCluster.tls.certManager.usages | list | `["server auth","key encipherment","signing"]` | List of key usages. |
| scalardbCluster.tls.enabled | bool | `false` | Enable TLS. You need to enable TLS when you use wire encryption feature of ScalarDB Cluster. |
| scalardbCluster.tls.overrideAuthority | string | `""` | The custom authority for TLS communication. This doesn't change what host is actually connected. This is intended for testing, but may safely be used outside of tests as an alternative to DNS overrides. For example, you can specify the hostname presented in the certificate chain file that you set by using `scalardbCluster.tls.certChainSecret`. This chart uses this value for startupProbe and livenessProbe. |
| scalardbCluster.tls.privateKeySecret | string | `""` | Name of the Secret containing the private key file used for TLS communication. |
| scalardbCluster.tls.privateKeySecret | string | `""` | Name of the Secret containing the private key file used for TLS communication. |
| scalardbCluster.tolerations | list | `[]` | Tolerations are applied to pods, and allow (but do not require) the pods to schedule onto nodes with matching taints. |
84 changes: 84 additions & 0 deletions charts/scalardb-cluster/templates/scalardb-cluster/certmanager.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
{{- if .Values.scalardbCluster.tls.certManager.enabled }}
{{- if .Values.scalardbCluster.tls.certManager.selfSigned.enabled }}
# Self-signed root CA
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: {{ include "scalardb-cluster.fullname" . }}-self-signed-issuer
namespace: {{ .Release.Namespace }}
labels:
{{- include "scalardb-cluster.labels" . | nindent 4 }}
spec:
selfSigned: {}
---
# Generate a CA Certificate used to sign certificates for the ScalarDB Cluster
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: {{ include "scalardb-cluster.fullname" . }}-root-ca-cert
namespace: {{ .Release.Namespace }}
labels:
{{- include "scalardb-cluster.labels" . | nindent 4 }}
spec:
isCA: true
secretName: {{ include "scalardb-cluster.fullname" . }}-root-ca-cert
secretTemplate:
labels:
{{- include "scalardb-cluster.labels" . | nindent 6 }}
commonName: self-signed-ca
duration: {{ .Values.scalardbCluster.tls.certManager.selfSigned.caRootCert.duration | quote }}
renewBefore: {{ .Values.scalardbCluster.tls.certManager.selfSigned.caRootCert.renewBefore | quote }}
privateKey:
algorithm: ECDSA
size: 256
issuerRef:
name: {{ include "scalardb-cluster.fullname" . }}-self-signed-issuer
kind: Issuer
group: cert-manager.io
---
# Create an Issuer that uses the above generated CA certificate to issue certs
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: {{ include "scalardb-cluster.fullname" . }}-ca-issuer
namespace: {{ .Release.Namespace }}
labels:
{{- include "scalardb-cluster.labels" . | nindent 4 }}
spec:
ca:
secretName: {{ include "scalardb-cluster.fullname" . }}-root-ca-cert
{{- end }}
---
# Generate a server certificate for the ScalarDB Cluster
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: {{ include "scalardb-cluster.fullname" . }}-tls-cert
namespace: {{ .Release.Namespace }}
labels:
{{- include "scalardb-cluster.labels" . | nindent 4 }}
spec:
secretName: {{ include "scalardb-cluster.fullname" . }}-tls-cert
secretTemplate:
labels:
{{- include "scalardb-cluster.labels" . | nindent 6 }}
duration: {{ .Values.scalardbCluster.tls.certManager.duration | quote }}
renewBefore: {{ .Values.scalardbCluster.tls.certManager.renewBefore | quote }}
privateKey:
{{- toYaml .Values.scalardbCluster.tls.certManager.privateKey | nindent 4 }}
usages:
{{- range .Values.scalardbCluster.tls.certManager.usages }}
- {{ . | quote }}
{{- end }}
dnsNames:
{{- range .Values.scalardbCluster.tls.certManager.dnsNames }}
- {{ . | quote }}
{{- end }}
issuerRef:
# If and only if you set `scalardbCluster.tls.certManager.selfSigned.enabled=true`, this chart automatically generates a self-signed CA and uses it.
{{- if .Values.scalardbCluster.tls.certManager.selfSigned.enabled }}
name: {{ include "scalardb-cluster.fullname" . }}-ca-issuer
{{- else }}
{{- toYaml .Values.scalardbCluster.tls.certManager.issuerRef | nindent 4 }}
{{- end }}
{{- end }}
50 changes: 18 additions & 32 deletions charts/scalardb-cluster/templates/scalardb-cluster/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,7 @@ spec:
- -addr=localhost:60053
{{- if .Values.scalardbCluster.tls.enabled }}
- -tls
{{- if .Values.scalardbCluster.tls.caRootCertSecret }}
- -tls-ca-cert=/tls/certs/ca-root-cert.pem
{{- end }}
- -tls-ca-cert=/tls/scalardb-cluster/certs/ca.crt
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you enable TLS, the -tls-ca-cert option is mandatory. In other words, {{- if .Values.scalardbCluster.tls.caRootCertSecret }} does not make sense. So, I removed that unnecessary condition.

{{- if .Values.scalardbCluster.tls.overrideAuthority }}
- -tls-server-name={{ .Values.scalardbCluster.tls.overrideAuthority }}
{{- end }}
Expand All @@ -82,9 +80,7 @@ spec:
- -addr=localhost:60053
{{- if .Values.scalardbCluster.tls.enabled }}
- -tls
{{- if .Values.scalardbCluster.tls.caRootCertSecret }}
- -tls-ca-cert=/tls/certs/ca-root-cert.pem
{{- end }}
- -tls-ca-cert=/tls/scalardb-cluster/certs/ca.crt
{{- if .Values.scalardbCluster.tls.overrideAuthority }}
- -tls-server-name={{ .Values.scalardbCluster.tls.overrideAuthority }}
{{- end }}
Expand All @@ -97,20 +93,9 @@ spec:
- name: scalardb-cluster-database-properties-volume
mountPath: /scalardb-cluster/node/scalardb-cluster-node.properties
subPath: scalardb-cluster-node.properties
{{- if .Values.scalardbCluster.tls.caRootCertSecret }}
- name: scalardb-cluster-tls-ca-root-volume
mountPath: /tls/certs/ca-root-cert.pem
subPath: ca-root-cert
{{- end }}
{{- if .Values.scalardbCluster.tls.certChainSecret }}
- name: scalardb-cluster-tls-cert-chain-volume
mountPath: /tls/certs/cert-chain.pem
subPath: cert-chain
{{- end }}
{{- if .Values.scalardbCluster.tls.privateKeySecret }}
- name: scalardb-cluster-tls-private-key-volume
mountPath: /tls/certs/private-key.pem
subPath: private-key
{{- if .Values.scalardbCluster.tls.enabled }}
- name: scalardb-cluster-tls-volume
mountPath: /tls/scalardb-cluster/certs
Comment on lines +96 to +98
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I combine all private keys and certificates into one volume to remove the subPath configuration.

{{- end }}
{{- with .Values.scalardbCluster.extraVolumeMounts }}
{{- toYaml . | nindent 12 }}
Expand All @@ -119,20 +104,21 @@ spec:
- name: scalardb-cluster-database-properties-volume
configMap:
name: {{ include "scalardb-cluster.fullname" . }}-node-properties
{{- if .Values.scalardbCluster.tls.caRootCertSecret }}
- name: scalardb-cluster-tls-ca-root-volume
{{- if and .Values.scalardbCluster.tls.enabled .Values.scalardbCluster.tls.certManager.enabled }}
- name: scalardb-cluster-tls-volume
secret:
secretName: {{ .Values.scalardbCluster.tls.caRootCertSecret }}
secretName: {{ include "scalardb-cluster.fullname" . }}-tls-cert
Comment on lines +107 to +110
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you use cert-manager, it creates one secret resource that includes all private keys and certificates. So, it's enough to mount that one secret here.

{{- end }}
{{- if .Values.scalardbCluster.tls.certChainSecret }}
- name: scalardb-cluster-tls-cert-chain-volume
secret:
secretName: {{ .Values.scalardbCluster.tls.certChainSecret }}
{{- end }}
{{- if .Values.scalardbCluster.tls.privateKeySecret }}
- name: scalardb-cluster-tls-private-key-volume
secret:
secretName: {{ .Values.scalardbCluster.tls.privateKeySecret }}
{{- if and (.Values.scalardbCluster.tls.enabled) (not .Values.scalardbCluster.tls.certManager.enabled) }}
- name: scalardb-cluster-tls-volume
projected:
sources:
- secret:
name: {{ .Values.scalardbCluster.tls.caRootCertSecret }}
- secret:
name: {{ .Values.scalardbCluster.tls.certChainSecret }}
- secret:
name: {{ .Values.scalardbCluster.tls.privateKeySecret }}
Comment on lines +112 to +121
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't use cert-manager (and you want to manage each file separately), you have to create three secrets for the private key, certificate, and CA certificate.

In this case, I want to combine them into one volume to mount them easily under the same directory in the container. So, I use projected volume here.

{{- end }}
{{- with .Values.scalardbCluster.extraVolumes }}
{{- toYaml . | nindent 8 }}
Expand Down
62 changes: 62 additions & 0 deletions charts/scalardb-cluster/values.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,68 @@
"certChainSecret": {
"type": "string"
},
"certManager": {
"type": "object",
"properties": {
"dnsNames": {
"type": "array",
"items": {
"type": "string"
}
},
"duration": {
"type": "string"
},
"enabled": {
"type": "boolean"
},
"issuerRef": {
"type": "object"
},
"privateKey": {
"type": "object",
"properties": {
"algorithm": {
"type": "string"
},
"encoding": {
"type": "string"
},
"size": {
"type": "integer"
}
}
},
"renewBefore": {
"type": "string"
},
"selfSigned": {
"type": "object",
"properties": {
"caRootCert": {
"type": "object",
"properties": {
"duration": {
"type": "string"
},
"renewBefore": {
"type": "string"
}
}
},
"enabled": {
"type": "boolean"
}
}
},
"usages": {
"type": "array",
"items": {
"type": "string"
}
}
}
},
"enabled": {
"type": "boolean"
},
Expand Down
37 changes: 34 additions & 3 deletions charts/scalardb-cluster/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -278,9 +278,40 @@ scalardbCluster:
enabled: false
# -- The custom authority for TLS communication. This doesn't change what host is actually connected. This is intended for testing, but may safely be used outside of tests as an alternative to DNS overrides. For example, you can specify the hostname presented in the certificate chain file that you set by using `scalardbCluster.tls.certChainSecret`. This chart uses this value for startupProbe and livenessProbe.
overrideAuthority: ""
# -- Name of the Secret containing the custom CA root certificate for TLS communication.
# -- Name of the Secret containing the custom CA root certificate for TLS communication.
caRootCertSecret: ""
# -- Name of the Secret containing the certificate chain file used for TLS communication.
# -- Name of the Secret containing the certificate chain file used for TLS communication.
certChainSecret: ""
# -- Name of the Secret containing the private key file used for TLS communication.
# -- Name of the Secret containing the private key file used for TLS communication.
privateKeySecret: ""
certManager:
# -- Use cert-manager to manage private key and certificate files.
enabled: false
# -- Configuration of a certificate for self-signed CA.
selfSigned:
# -- Use self-signed CA.
enabled: false
caRootCert:
# -- Duration of a self-signed CA certificate.
duration: "8760h0m0s"
# -- How long before expiry a self-signed CA certificate should be renewed.
renewBefore: "360h0m0s"
# -- Duration of a certificate.
duration: "8760h0m0s"
# -- How long before expiry a certificate should be renewed.
renewBefore: "360h0m0s"
# -- Configuration of a private key.
privateKey:
algorithm: ECDSA
encoding: PKCS1
size: 256
# -- List of key usages.
usages:
- server auth
- key encipherment
- signing
# -- Subject Alternative Name (SAN) of a certificate.
dnsNames:
- localhost
Comment on lines +314 to +315
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I set localhost as a default value of the dnsNames. This is because:

  • The grpc_health_probe command accesses ScalarDB Cluster by specifying localhost like -addr=localhost:60053.
  • The grpc_health_probe command verifies the certificate by using Subject Alternative Name (SAN) instead of Common Name (CN) of the certificate. In other words, at least one SAN configuration is necessary. (I will update the document in another PR.)

# -- Issuer references of cert-manager.
issuerRef: {}
Loading