What is the best way to remove some objects imported from Helm?

[felixscheinost]

Hi, thanks for picking up this project!

I am importing an existing Helm chart, which doesn't have the ability to turn some feature off which I want to turn off.

Currrently I am doing something like that:

{ kubenix, ... }: let
  helmObjects = (kubenix.evalModules {
    module = { kubenix, ... }: {
      imports = with kubenix.modules; [ k8s helm ];
      kubernetes.helm.releases.test = {
         # [...]
      };
    };
  }).config.kubernetes.objects;
  filteredHelmObjects = builtins.filter (x: true) helmObjects;
in {
  kubernetes.objects = filteredHelmObjects;
}

Is that the preferred approach or am I missing something?

[adrian-gierakowski]

You can probably set the specific objects to null, if you know their type and name:

kubernetes.resources.deployments.my-name = mkForce null

[hall]

Thanks for the discussion @felixscheinost and @adrian-gierakowski!

I think filtering and (eventually if it cannot be done at the same time/implementation) being able to arbitrarily modify resources.would be great. Early on, I was considering support for helmfile almost solely to leverage their jsonpatch support but would much rather it be done in a more nix-like way (rather than wrapping yet another tool).

At first glance filtering seems the more generic/safe approach but am open arguments why nullifying might be better or also necessary (e.g., maybe filtering should only modify resources and setting them to null would be the answer to deleting with a filter).

Happy to review a PR here.

[felixscheinost]

btw. with this small change it is be possible to do the following to delete a resource

Oh cool! That is indeed only a small change. Hmm, I think I would still rather go with the filtering approach as like you said the chance for breaking something else is lower.

e.g., maybe filtering should only modify resources and setting them to null would be the answer to deleting with a filter

That's a cool idea. Instead of having a filter: (Resource) -> Bool we could have a map: (Resource) -> Resource? so returning null would drop the resource and returning non-null would also give the ability to modify the resource. We should probably mkMerge the result of the map so that it behaves like e.g. overrideAttrs, right?

[hall]

the chance for breaking something else is lower

The README.md file does say to expect breaking changes 🙂 I'm OK doing so for good design this early on and will create a CHANGELOG.md file anyway and start tagging versions beforehand to help mitigate surprises.

We should probably mkMerge the result of the map so that it behaves like e.g. overrideAttrs, right?

That sounds reasonable. I haven't sat down to think through any sort of implementation yet. Ideally, we could create a few test cases to help ensure functionality matches expectation but I myself am not terribly familiar with the state of testing here so I certainly wouldn't let it slow you down (since we can always go back and add them later).

[adrian-gierakowski]

That's a cool idea. Instead of having a filter: (Resource) -> Bool we could have a map: (Resource) -> Resource? so returning null would drop the resource and returning non-null would also give the ability to modify the resource.

The overriding via merging is already possible since that's what the module system is designed to do. Also there is overrides option in the helm module which value is merged with each individual resource produced by the hem chart:

kubenix/modules/helm.nix

Lines 117 to 120 in b2feaf6

	"${apiVersion.group}"."${apiVersion.version}".${object.kind}."${name}" = mkMerge ([
	object
	]
	++ release.overrides);

This allows to easily add overrides of props which are common to all produced resources, like metadata.labels or annotations. For targeting individual resources you can simply add something like the following to you module definitions:

kubernetes.api.resources.apps.v1.Deployment.test.some.prop.to.override = mkForce value;

The only thing that is currently not possible to do is to delete existing resources. To solve this we have a few option:

1. filtering function at the helm chat level (helm module specific)
2. allow resources to be nullified (this would work for any resources, not just ones created with helm module)
3. filtering function at the k8s module level, similar to 1, but could but more general

Here's an example of how the function could be implemented at the k8s module level. I've implemented it some time ago on my fork to allow things like hashing config map names. Note that this modifies the final object list, which means that if you build kubenix manifests together with other modules, which reference kubenix resources (like kubernetes.resources.deployments.test.metadata.name), these other modules would not see the modification made by this function.

[felixscheinost]

The overriding via merging is already possible since that's what the module system is designed to do.

You're right. It wouldn't make sense to add another way to do the same thing, just worse ;)

What if I wanted to try to modify e.g. all Secrets though? Would I be able to do that without encountering infinite recursion? I assume that's the reason you implemented objectPostProcessingFunc to not operate on the submodules but instead on the final list of objects?

To solve this we have a few option:

When reading this list I think I would actually like the more generic options most. There's probably no reason to make this Helm specific.

allow resources to be nullified (this would work for any resources, not just ones created with helm module)

What I like about this option is that it seems more declarative than setting some filter function somewhere, so more in line with the NixOS module system. I don't like null values in general though as you now have to consider null values whenever mapping over the kubernetes.api.resources.

Would it make sense to instead of allowing nullability add a "synthetic" attribute to all resources, like kubenix.enabled or something like that? Which is true by default and can be set to false to omit the resource?