Sends and labels `VReplicationClass`, `VSnapClass`, `StorageClass` to client #2727

raaizik · 2024-08-01T13:02:06Z

Changes

Sends two VRCs (one for image flattening) and one VGRC. Depends on Adds metadata to external ceph resources #2620
Labels VRepClass, VSnapClass and StorageClass

RHSTOR-5753, RHSTOR-5794, RHSTOR-5754

raaizik · 2024-08-01T13:03:20Z

/cc @Madhu-1

raaizik · 2024-08-06T10:35:18Z

/cc @rewantsoni

rewantsoni

For Volume*ReplicationClass we are sending only the parameters field like we did for Volume*SnapshotClass and StorageClass, Should we send the entire spec for it or maybe the entire object that will allow us to add labels/annotations required (ramen labels, reclaimspace annotation) from the provider itself and the client would be responsible for adding missing fields like ProvisionerName and namespace for secrets and then creating/upating it.

@nb-ohad WDYT?

rewantsoni · 2024-08-06T10:52:33Z

services/provider/server/server.go

+					Data: mustMarshal(map[string]string{
+						"replication.storage.openshift.io/replication-secret-name": provisionerSecretName,


For flattening VRC, we require the special parameter flattenMode: force to be added, we should add it from the provider side

@raaizik I think this was missed

rewantsoni · 2024-08-06T10:53:00Z

services/provider/server/server.go

+					Data: mustMarshal(map[string]string{
+						"replication.storage.openshift.io/replication-secret-name": provisionerSecretName,
+					}),


Along with the replication secret name, we also require mirroringMode key that can we sent from the provider, could you add that as well in all the V*RC?
Ref: https://github.com/red-hat-storage/odf-multicluster-orchestrator/blob/main/controllers/drpolicy_controller.go#L170-L178

rewantsoni · 2024-08-06T10:55:15Z

services/provider/server/server.go

+					Data: mustMarshal(map[string]string{
+						"replication.storage.openshift.io/group-replication-secret-name": provisionerSecretName,
+					}),


For flattening VGRC, we require the special parameter flattenMode: force to be added, we should add it from the provider side

openshift-ci · 2024-08-06T14:32:21Z

@raaizik: GitHub didn't allow me to request PR reviews from the following users: Rakshith-R.

Note that only red-hat-storage members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @Rakshith-R

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

raaizik · 2024-08-06T14:33:58Z

/cc @Rakshith-R

openshift-ci · 2024-08-06T14:34:01Z

@raaizik: GitHub didn't allow me to request PR reviews from the following users: Rakshith-R.

Note that only red-hat-storage members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @Rakshith-R

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

services/provider/server/server.go

Rakshith-R

"replication.storage.openshift.io/is-default-class": "true",
This should be added as a annotation

and
"replication.storage.openshift.io/flatten-mode": "force",

This as a label

hope both of the above will be handled accordingly in consumer side.

Other than the above, everything else looks good to me.

raaizik · 2024-08-07T13:35:48Z

"replication.storage.openshift.io/is-default-class": "true", This should be added as a annotation

and "replication.storage.openshift.io/flatten-mode": "force",

This as a label

hope both of the above will be handled accordingly in consumer side.

Other than the above, everything else looks good to me.

@Rakshith-R I'm aware and it will.

nb-ohad · 2024-08-07T19:50:23Z

For VolumeReplicationClass we are sending only the parameters field like we did for VolumeSnapshotClass and StorageClass, Should we send the entire spec for it or maybe the entire object that will allow us to add labels/annotations required (ramen labels, reclaimspace annotation) from the provider itself and the client would be responsible for adding missing fields like ProvisionerName and namespace for secrets and then creating/upating it.

@nb-ohad WDYT?

I will start with the later proposal first:

maybe the entire object that will allow us to add labels/annotations required

We do not want to do that, an object contains not just desired state but also status and some metadata that is local to the provider cluster. In internal discussions, we already acknowledged the need to send labels and annotations but not as part of the serialized spec we send (the data field of the ExternalResource struct). The plan is to add it to the metadata of the ExternalResource object, similar to the kind and name. Until that is implemented we will keep on having logic in the client to add the missing metadata.

Should we send the entire spec
This depends on the structure of the type, we originally used data because that was the desired state for configmaps and secrets. If the Volume*ReplicationClass has a formal spec field it will make sense to serialize it instead of a map[string]string.
As a matter of fact, we are already doing something similar for the ClusterResourceQuota CR (https://github.com/red-hat-storage/ocs-operator/blob/main/services/provider/server/server.go#L376-L399)

raaizik · 2024-08-26T11:02:09Z

/test ocs-operator-bundle-e2e-aws

raaizik · 2024-08-26T12:23:36Z

/test ocs-operator-bundle-e2e-aws

raaizik · 2024-09-05T10:33:07Z

/test ocs-operator-bundle-e2e-aws

nb-ohad · 2024-09-08T11:02:04Z

/lgtm

rewantsoni · 2024-11-04T16:46:27Z

services/provider/server/server.go

 			rns := &rookCephv1.CephBlockPoolRadosNamespace{}
 			err = s.client.Get(ctx, types.NamespacedName{Name: cephRes.Name, Namespace: s.namespace}, rns)
 			if err != nil {
 				return nil, status.Errorf(codes.Internal, "failed to get %s CephBlockPoolRadosNamespace. %v", cephRes.Name, err)
 			}
-
+			mirroringEnabled := len(rns.Spec.Mirroring.Mode) > 0


Could you change it to If rns.Spec.Mirroring is not nil then mirroring is enabled

I think this is more precise check (it checks both)

What do you mean by it checks both?
We need to set this if mirroring is enabled, checking only if it is not nil should be sufficient

rewantsoni · 2024-11-04T16:48:54Z

services/provider/server/server.go

+	switch kind {
+	case "VolumeReplicationClass", "VolumeGroupReplicationClass":
+		if isMirroringReplicationEnabled {
+			replicationID := replicationID


I think this is redundant

rewantsoni · 2024-11-04T19:01:45Z

services/provider/server/server.go

 			rns := &rookCephv1.CephBlockPoolRadosNamespace{}
 			err = s.client.Get(ctx, types.NamespacedName{Name: cephRes.Name, Namespace: s.namespace}, rns)
 			if err != nil {
 				return nil, status.Errorf(codes.Internal, "failed to get %s CephBlockPoolRadosNamespace. %v", cephRes.Name, err)
 			}
-
+			mirroringEnabled := len(rns.Spec.Mirroring.Mode) > 0


What do you mean by it checks both?
We need to set this if mirroring is enabled, checking only if it is not nil should be sufficient

rewantsoni · 2024-11-04T19:09:03Z

services/provider/server/server.go

+		}
+	case "StorageClass":
+		if isMirroringReplicationEnabled {
+			labels["ramendr.openshift.io/replicationid"] = replicationID


storageClass doesn't need the replicationID label

rewantsoni · 2024-11-05T14:00:56Z

services/provider/server/server.go

+					Data: mustMarshal(map[string]string{
+						"replication.storage.openshift.io/replication-secret-name": provisionerSecretName,


@raaizik I think this was missed

rewantsoni · 2024-11-05T14:10:24Z

LGTM

rewantsoni · 2024-11-05T14:46:44Z

services/provider/server/server.go

+					Labels: getExternalResourceLabels("VolumeGroupReplicationClass", replicationEnabled, false, replicationID, storageID),
+				},


Suggested change

Labels: getExternalResourceLabels("VolumeGroupReplicationClass", replicationEnabled, false, replicationID, storageID),

},

Labels: getExternalResourceLabels("VolumeSnapshotClass", replicationEnabled, false, replicationID, storageID),

},

rewantsoni · 2024-11-05T14:47:13Z

services/provider/server/server.go

 				&pb.ExternalResource{
 					Name: "cephfs",
 					Kind: "VolumeGroupSnapshotClass",
 					Data: mustMarshal(map[string]string{
 						"csi.storage.k8s.io/group-snapshotter-secret-name": provisionerSecretName,
-					})},
+					}),
+					Labels: getExternalResourceLabels("VolumeGroupReplicationClass", replicationEnabled, false, replicationID, storageID),


Suggested change

Labels: getExternalResourceLabels("VolumeGroupReplicationClass", replicationEnabled, false, replicationID, storageID),

Labels: getExternalResourceLabels("VolumeGroupSnapshotClass", replicationEnabled, false, replicationID, storageID),

Madhu-1 · 2024-11-06T11:06:49Z

services/provider/server/server.go

+				&pb.ExternalResource{
+					Name: "ceph-rbd",
+					Kind: "VolumeGroupReplicationClass",
+					Data: mustMarshal(map[string]string{
+						"replication.storage.openshift.io/group-replication-secret-name": provisionerSecretName,
+						"mirroringMode": "snapshot",
+					}),
+					Labels: getExternalResourceLabels("VolumeGroupReplicationClass", mirroringEnabled, false, replicationID, storageID),
+					Annotations: map[string]string{
+						"replication.storage.openshift.io/is-default-class": "true",
+					},
+				},
+				&pb.ExternalResource{
+					Name: "ceph-rbd-flatten",
+					Kind: "VolumeGroupReplicationClass",
+					Data: mustMarshal(map[string]string{
+						"replication.storage.openshift.io/group-replication-secret-name": provisionerSecretName,
+						"mirroringMode": "snapshot",
+						"flattenMode":   "force",
+					}),
+					Labels:      getExternalResourceLabels("VolumeGroupReplicationClass", mirroringEnabled, true, replicationID, storageID),
+					Annotations: map[string]string{},
+				},


we dont have voluemegroupreplicationclass in 4.18 isnt it? can we remove this and add it later?

Madhu-1 · 2024-11-06T11:07:45Z

services/provider/server/server.go

 				&pb.ExternalResource{
 					Name: "cephfs",
 					Kind: "VolumeGroupSnapshotClass",
 					Data: mustMarshal(map[string]string{
 						"csi.storage.k8s.io/group-snapshotter-secret-name": provisionerSecretName,
-					})},
+					}),
+					Labels: getExternalResourceLabels("VolumeGroupSnapshotClass", replicationEnabled, false, replicationID, storageID),


IMO we should use consts for all class kinds, we are passing it as string in multiple places.

If you mean replace it in all places not just the function call -- I did that at first, but @nb-ohad opposed to the idea

Madhu-1 · 2024-11-06T11:08:45Z

services/provider/server/server.go

+		}
+	case "VolumeSnapshotClass", "VolumeGroupSnapshotClass":
+		if isMirroringReplicationEnabled {
+			labels["ramendr.openshift.io/storageID"] = storageID


can we have a const for the label keys?

- Sends two V*RCs (one for image flattening) for RDR - Labels V*RepClass, V*SnapClass and StorageClass Signed-off-by: raaizik <[email protected]>

openshift-ci bot requested a review from Madhu-1 August 1, 2024 13:03

raaizik force-pushed the RHSTOR-5753 branch 2 times, most recently from cb85825 to eeb74d1 Compare August 6, 2024 10:31

openshift-ci bot requested a review from rewantsoni August 6, 2024 10:35

rewantsoni suggested changes Aug 6, 2024

View reviewed changes

openshift-ci bot assigned rewantsoni Aug 6, 2024

raaizik force-pushed the RHSTOR-5753 branch from eeb74d1 to 9e2b5c5 Compare August 6, 2024 14:31

Rakshith-R reviewed Aug 7, 2024

View reviewed changes

services/provider/server/server.go Outdated Show resolved Hide resolved

raaizik force-pushed the RHSTOR-5753 branch 3 times, most recently from ffd5355 to 5c428c0 Compare August 7, 2024 12:47

raaizik requested review from Rakshith-R and rewantsoni August 7, 2024 12:47

Rakshith-R reviewed Aug 7, 2024

View reviewed changes

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 12, 2024

raaizik force-pushed the RHSTOR-5753 branch from 5c428c0 to 7982c8e Compare August 26, 2024 10:07

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 26, 2024

raaizik force-pushed the RHSTOR-5753 branch from 7982c8e to 1f19940 Compare September 4, 2024 12:35

openshift-ci bot assigned nb-ohad Sep 8, 2024

raaizik force-pushed the RHSTOR-5753 branch 4 times, most recently from fe9ea70 to 6c69b93 Compare November 4, 2024 16:40

rewantsoni suggested changes Nov 4, 2024

View reviewed changes

raaizik requested a review from rewantsoni November 4, 2024 17:09

raaizik force-pushed the RHSTOR-5753 branch 3 times, most recently from a8d44cb to 16961db Compare November 4, 2024 18:22

rewantsoni suggested changes Nov 4, 2024

View reviewed changes

raaizik force-pushed the RHSTOR-5753 branch 2 times, most recently from d65b1f2 to cae0219 Compare November 5, 2024 13:52

raaizik requested a review from rewantsoni November 5, 2024 13:52

rewantsoni suggested changes Nov 5, 2024

View reviewed changes

raaizik force-pushed the RHSTOR-5753 branch from cae0219 to 99c215b Compare November 5, 2024 14:05

raaizik requested a review from rewantsoni November 5, 2024 14:10

raaizik force-pushed the RHSTOR-5753 branch from 99c215b to c41a037 Compare November 5, 2024 14:41

raaizik changed the title ~~Sends V*ReplicationClass to client~~ Sends and labels V*ReplicationClass, V*SnapClass, StorageClass to client Nov 5, 2024

rewantsoni suggested changes Nov 5, 2024

View reviewed changes

raaizik force-pushed the RHSTOR-5753 branch from c41a037 to 858a3d4 Compare November 5, 2024 15:09

raaizik requested review from rewantsoni and Rakshith-R November 5, 2024 15:10

raaizik mentioned this pull request Nov 5, 2024

Generates VolRepClasses based on the RamenDR replication schedule resource red-hat-storage/ocs-client-operator#177

Draft

Madhu-1 reviewed Nov 6, 2024

View reviewed changes

Sends StorageClasses to client

20b2746

- Sends two V*RCs (one for image flattening) for RDR - Labels V*RepClass, V*SnapClass and StorageClass Signed-off-by: raaizik <[email protected]>

raaizik force-pushed the RHSTOR-5753 branch from 858a3d4 to 20b2746 Compare November 10, 2024 15:29

raaizik requested a review from Madhu-1 November 10, 2024 15:29

raaizik changed the title ~~Sends and labels V*ReplicationClass, V*SnapClass, StorageClass to client~~ Sends and labels VSnapReplicationClass, V*SnapClass, StorageClass to client Nov 10, 2024

raaizik changed the title ~~Sends and labels VSnapReplicationClass, V*SnapClass, StorageClass to client~~ Sends and labels V*ReplicationClass, V*SnapClass, StorageClass to client Nov 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sends and labels `VReplicationClass`, `VSnapClass`, `StorageClass` to client #2727

Sends and labels `VReplicationClass`, `VSnapClass`, `StorageClass` to client #2727

raaizik commented Aug 1, 2024 •

edited

Loading

raaizik commented Aug 1, 2024

raaizik commented Aug 6, 2024

rewantsoni left a comment •

edited

Loading

rewantsoni Aug 6, 2024

rewantsoni Nov 5, 2024

rewantsoni Aug 6, 2024

rewantsoni Aug 6, 2024

openshift-ci bot commented Aug 6, 2024

raaizik commented Aug 6, 2024

openshift-ci bot commented Aug 6, 2024

Rakshith-R left a comment

raaizik commented Aug 7, 2024 •

edited

Loading

nb-ohad commented Aug 7, 2024

raaizik commented Aug 26, 2024

raaizik commented Aug 26, 2024

raaizik commented Sep 5, 2024

nb-ohad commented Sep 8, 2024

rewantsoni Nov 4, 2024

raaizik Nov 4, 2024

rewantsoni Nov 4, 2024

rewantsoni Nov 4, 2024

rewantsoni Nov 4, 2024

rewantsoni Nov 4, 2024

rewantsoni Nov 5, 2024

rewantsoni commented Nov 5, 2024

rewantsoni Nov 5, 2024

rewantsoni Nov 5, 2024

Madhu-1 Nov 6, 2024

Madhu-1 Nov 6, 2024

raaizik Nov 6, 2024

Madhu-1 Nov 6, 2024

		Data: mustMarshal(map[string]string{
		"replication.storage.openshift.io/replication-secret-name": provisionerSecretName,

		Labels: getExternalResourceLabels("VolumeGroupReplicationClass", replicationEnabled, false, replicationID, storageID),
		},

Sends and labels V*ReplicationClass, V*SnapClass, StorageClass to client #2727

Are you sure you want to change the base?

Sends and labels V*ReplicationClass, V*SnapClass, StorageClass to client #2727

Conversation

raaizik commented Aug 1, 2024 • edited Loading

Changes

raaizik commented Aug 1, 2024

raaizik commented Aug 6, 2024

rewantsoni left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-ci bot commented Aug 6, 2024

raaizik commented Aug 6, 2024

openshift-ci bot commented Aug 6, 2024

Rakshith-R left a comment

Choose a reason for hiding this comment

raaizik commented Aug 7, 2024 • edited Loading

nb-ohad commented Aug 7, 2024

raaizik commented Aug 26, 2024

raaizik commented Aug 26, 2024

raaizik commented Sep 5, 2024

nb-ohad commented Sep 8, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rewantsoni commented Nov 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Sends and labels `VReplicationClass`, `VSnapClass`, `StorageClass` to client #2727

Sends and labels `VReplicationClass`, `VSnapClass`, `StorageClass` to client #2727

raaizik commented Aug 1, 2024 •

edited

Loading

rewantsoni left a comment •

edited

Loading

raaizik commented Aug 7, 2024 •

edited

Loading