feedback on complex config manipulation use-case

[errordeveloper]

I found this provider when I was looking for a simple-ish way to take an existing Kubernetes manifests and customise them a little based on some terraform parameters, e.g. insert environment variables.

My key requirements is that the given Kubernetes manifests is sourced from somewhere and forking it or making a change upstream is not viable. I also don't want to shell out to kustomize or other tools for this purpose, at least for the simple thing I need to get done.

I tried using Terraform merged function, until I realise it's only a shallow merge. I found this provider and realised that it generally fits the bill. I really like it and appreciate the work put into it.

Here are a few things I thought I should share with the author:

• including a YAML parser might be quite handy
• passing Terraform objects as variables results in all attributes being read-only
  • using _.clone doesn't solve that
  • it's possible to add new properties, but still not possible to update an existing property

To do what I had to do, I ended up passing an object as JSON data (re-encoded from YAML source by Terraform), and parsing it in JavaScript. It's a bit inconvenient/unintuitive, I wished to be able to modify the object directly (having to copy is fine, of course).

[errordeveloper]

Here is a concrete example to illustrate what I said above: cilium/openshift-terraform-upi@9e22c4f

[apparentlymart]

Hi @errordeveloper! Thanks for this feedback.

While I was implementing this I had more of a functional programming style in mind -- derive a new value from the inputs -- and so I must admit I think to try situations where I'd modify the input data in-place like that. I'm not sure why cloning with underscore doesn't give you something mutable; I guess that must be an unfortunate interaction between whatever Underscore does in its cloning operation and how I implemented the conversion from Terraform data types into JavaScript values. 🤔

I'm not likely to have much time to spend on this provider in the near future, but if I do get a chance to iterate on it some more then I'll try to understand why cloning doesn't work the way you tried.

In the mean time, it might help to know that the programming model I was using for the situations I used this for was to use underscore's primitives to construct an entirely new value based on the input.

For example, you can derive a new object from an input object using _.merge with the input object and another object containing the properties you want to add or override. That's similar of course to Terraform's own merge function, but you can combine it with Underscore's other functions to achieve a deep merge with more flexibility than the Terraform language alone can achieve.

[errordeveloper]

Hi! Thanks for thinking about this :)

I gathered that the idea was like you described... however, when I looked at underscore docs, I couldn't find _.merge there and actually got a bit confused, cause I vaguely recollect using it in the past.

[apparentlymart]

Interesting! I remember doing something like this too myself, but perhaps I was doing it with something built into the JavaScript engine, rather than with Underscore. Perhaps it was Object.assign, or similar.

[errordeveloper]

On Mon, 30 Aug 2021, 10:28 pm Martin Atkins, ***@***.***> wrote: Interesting! I remember doing something like this too myself, but perhaps I was doing it with something built into the JavaScript engine, rather than with Underscore. Perhaps it was Object.assign <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign>, or similar.

I couldn't use `Object.assign` either, I forgot whether it was missing front the provider or something else happen, but I definitely tried it.