Skip to content

fix(tenantresource): ensure original map is not modified in prepareAdditionalMetadata #1572

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Aug 18, 2025

Conversation

yehlo
Copy link
Contributor

@yehlo yehlo commented Aug 8, 2025

Saw that the original map was returned when additionalMetadata was specified leading to the controller adding the labels and annotations as specified here: https://github.com/projectcapsule/capsule/blob/v0.10.0/controllers/resources/processor.go#L133 to the globaltenantresource.spec.resources.additionalMetadata.

Leading to the tenantresource.spec looking like this:

spec:
  pruningOnDelete: true
  resources:
  - additionalMetadata:
      annotations:
        foo: bar
        capsule.clastix.io/tenant: tenant-name
      labels:
        foo: bar
        capsule.clastix.io/resources: "0"
        capsule.clastix.io/tenant: tenant-name

I made sure to just create a new map and return this instead.

@yehlo yehlo changed the title fix(tenantresource): ensure original map is not modified in prepareAd… TenantResource additionalMetadata bugfix Aug 8, 2025
@yehlo yehlo changed the title TenantResource additionalMetadata bugfix fix(tenantresource): additionalMetadata bugfix Aug 8, 2025
@yehlo yehlo changed the title fix(tenantresource): additionalMetadata bugfix fix(tenantresource): ensure original map is not modified in prepareAdditionalMetadata Aug 8, 2025
@yehlo yehlo force-pushed the fix/tenantresource-metadata branch from 8c6a07d to ff20f02 Compare August 11, 2025 06:10
@Svarrogh1337
Copy link
Collaborator

Svarrogh1337 commented Aug 12, 2025

@yehlo the idea behind prepareAdditionalMetadata is that we either return empty map or the original one if it's defined in the spec. We are not doing any modification at this point, so making copy of the map is unnecessary.
This was done in order to prevent #1387 ( fixed in #1413 ).

Do you see any misbehaviour with this approach?

@yehlo
Copy link
Contributor Author

yehlo commented Aug 12, 2025

Hi @Svarrogh1337 If the original map is returned instead of a copy it will be updated through the reconciliation leading to the globalTenantResource object receiving an update.

This leads to the original map being mutated here:
annotations: https://github.com/projectcapsule/capsule/blob/v0.10.0/controllers/resources/processor.go#L133
labels: https://github.com/projectcapsule/capsule/blob/v0.10.0/controllers/resources/processor.go#L135-L136

As the map is a pointer to the actual resource received here https://github.com/projectcapsule/capsule/blob/v0.10.0/controllers/resources/global.go#L51-L61 the map within k8s will be mutated aswell.

This leads to the following misbehaviour:

I create a globalTenantResource that looks like this:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: my-replication
spec:
  pruningOnDelete: true
  resources:
  - additionalMetadata:
      annotations:
        foo: "bar"
      labels:
        foo: "bar"
    namespacedItems:
    - apiVersion: v1
      kind: Configmap
      namespace: my-conf
      selector:
        matchLabels:
          do-replicate: "true"
  resyncPeriod: 60s
  tenantSelector:
    matchLabels:
      receive-replicated: "true"

While reconciling all tenants matching the tenantSelector the processor.go will add labels specific to the tenant to the map. (s. linked up)

While this map will then be added as metadata on resource creation, which results in the expected behaviour creating a configmap like this:

apiVersion: v1
kind: LimitRange
metadata:
  annotations:
    foo: "bar"
    capsule.clastix.io/tenant: my-tenant # mutated default tenant annotation
  creationTimestamp: "2025-08-08T06:37:28Z"
  labels:
    foo: "bar"
    capsule.clastix.io/resources: "0" # mutated default tenant label
    capsule.clastix.io/tenant: my-tenant # mutated default tenant label
  name: my-conf
  namespace: my-tenant-namespace
data:
  foo: "bar"

It will also result in the original globalTenantResource being mutated with the tenant processed (as the map is a pointer) leading to the resource looking like this:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: my-replication
spec:
  pruningOnDelete: true
  resources:
  - additionalMetadata:
      annotations:
        foo: "bar"
        capsule.clastix.io/tenant: my-tenant # added due to the map being a pointer
      labels:
        foo: "bar"
        capsule.clastix.io/resources: "0" # added due to the map being a pointer
        capsule.clastix.io/tenant: my-tenant # added due to the map being a pointer
    namespacedItems:
    - apiVersion: v1
      kind: Configmap
      namespace: my-conf
      selector:
        matchLabels:
          do-replicate: "true"
  resyncPeriod: 60s
  tenantSelector:
    matchLabels:
      receive-replicated: "true"

While this does not impact the replicated resource it results in diffs within GitOps processes resulting in endless reapplies.

Note

Please note that this only happens if additionalMetadata was specified on your globalTenantResource object

Hope this helps clarifying what I tried to solve with this PR.

@yehlo yehlo force-pushed the fix/tenantresource-metadata branch 2 times, most recently from c493375 to c9a5a30 Compare August 12, 2025 15:27
@oliverbaehler
Copy link
Collaborator

@Svarrogh1337 why do we need that function in the first place? As @yehlo the map given as args is always a reference, so any modification to it behaives like you know it from pointers.

Cant we just drop the function and directly loop or even deepycopy in the metadata check? It really doesnt add much value to abstract this to a function.

Probably even simpler is if we just pass it directly to createOrUpdate and check for nil in there. Blablabla. Just the above.

@Svarrogh1337
Copy link
Collaborator

@Svarrogh1337 why do we need that function in the first place? As @yehlo the map given as args is always a reference, so any modification to it behaives like you know it from pointers.

Cant we just drop the function and directly loop or even deepycopy in the metadata check? It really doesnt add much value to abstract this to a function.

Probably even simpler is if we just pass it directly to createOrUpdate and check for nil in there. Blablabla. Just the above.

I was thinking of dropping it in favour of DeepCopy indeed. @yehlo wdyt?

@yehlo
Copy link
Contributor Author

yehlo commented Aug 14, 2025

@Svarrogh1337 sure I don't mind though maps don't support the DeepCopy() method.

I can import the maps module and use it's Clone() method and check for nil on the supplied values.

Removing the prepareMetadata would end in something like this (not tested):

if spec.AdditionalMetadata != nil {
  if spec.AdditionalMetadata.Annotations != nil {
    objAnnotations = maps.Clone(spec.AdditionalMetadata.Annotations)
  } else {
    objAnnotations = make(map[string]string)
  }
  if spec.AdditionalMetadata.Labels != nil {
    objLabels = maps.Clone(spec.AdditionalMetadata.Labels)
  } else {
    objLabels = make(map[string]string)
  }
} 

I'd recommend keeping the method to keep it DRY and either stick to looping over keys (as seen in this PR) or use maps.Clone().

I might aswell just be overthinking.

Let me know what you'd prefer and I'll update accordingly.

@yehlo yehlo force-pushed the fix/tenantresource-metadata branch from 6c66d92 to ac33d22 Compare August 18, 2025 06:15
yehlo and others added 6 commits August 18, 2025 08:18
…ad of returning reference

Signed-off-by: Joshua Leuenberger <[email protected]>
…psule#1569)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

chore(tenantresource): make linter happy

Signed-off-by: Joshua Leuenberger <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…jectcapsule#1573)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…psule#1575)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
@yehlo yehlo force-pushed the fix/tenantresource-metadata branch from ac33d22 to 620c489 Compare August 18, 2025 06:20
@yehlo
Copy link
Contributor Author

yehlo commented Aug 18, 2025

Alright I now left it as is and fixed the commit message to not include camelCase.

@oliverbaehler
Copy link
Collaborator

Good, i will add e2e tests for this in my refactor branch. Thanks for the contribution! 🚀

@oliverbaehler oliverbaehler merged commit 67b5c3e into projectcapsule:main Aug 18, 2025
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants