Internode encryption should support cert-manager for secret management

[burmanm]

What is missing?
Like we modified the webhooks secret handling, we should also allow simplified management for internode encryption certificates. This might only require some documentation and test changes.

Why do we need it?
The internode encryption is a requested feature, but somewhat complicated and fragile in the current version.

┆Issue is synchronized with this Jira Story by Unito
┆Issue Number: CASS-58

[bradfordcp]

What would the estimate be if we limit the cluster to a single certificate for all pods?

[bradfordcp]

Hey team! Please add your planning poker estimate with ZenHub @burmanm @Miles-Garnsey @jsanda

[Miles-Garnsey]

AFAIK, we already support cert-manager for secrets management. There is just a hitch when it comes to rotating the certs.

I think any additional tests should probably be implemented under the heading of the cert-rotation work.

[jsanda]

We use cert-manage with the webhooks, but there is work to be done to be able to use it for internode encryption.

[Miles-Garnsey]

To simply enable encryption is relatively straightforward, however getting rotation working is a much more substantial piece of work. I've provided a revised estimate of 15 days to make sure rotation occurs without downtime. I think this estimate will be pretty fuzzy and we'd want this to become an epic with more precise estimates under each item.

One problem here lies in differences between DSE, Cassandra 3.x and Cassandra 4.x which will need to be catered to.