Versioning: Make each module have it's own version number

[qvistgaard]

As the size of the addons repo increases I find it harder make sense of which versions are relevant, especially when not working with the modules all time.

The biggest problem I see right now is, when the version changes, it is hard to keep a overview if it is necessary for me to to update. It leaves users with a lot of questions?

• Do I need to update this?
• Minor version changed: 🤔 Does it mean I get a new feature?
• Major version change: 😰 Uuuhhh. shit upgrading this breaks my shit, can use loose data, and does it even affect the module I'm looking at right now.

My proposition is to change the build process, giving each module it's own version numbers, just like helm stable does. When each package is build we publish them in a S3 bucket, and fetch them using s3 or https.

There are some side effects of this approach, both good and bad:

• 👍 We can change visibility of the repo to internal
• 👍 We can save build minutes if we check if the version already have been build
• 👍 We can auto generate a main readme.md with a list of links and recent versions
• 👍 As the repo grow, it will be easier to keep track of updates to the modules
• ✋ We need some sort if way to update versions / metadata
• 👎 Changelog generation will not work as great at it does today

[anarsen]

I agree and think this has been a problem for a while now.

I can't think of a sensible way to version modules within the same repository though. I think we're looking at splitting them into separate repos.
While fetching modules from S3 is supported by Terraform, what is really gained compared to just keeping each add-on module in its own repos?

I'm not convinced yet of either approach, so let's keep discussing!

[qvistgaard]

I can't think of a sensible way to version modules within the same repository though. I think we're looking at splitting them into separate repos.

In my opinion, if we split them we will have to deal with security every time we create a new module, and if you think about it we do not necessarily create a git tag every time we tag a module.

While fetching modules from S3 is supported by Terraform, what is really gained compared to just keeping each add-on module in its own repos?

I believe we get more ease of use, if you have to create a new repo every time you create a tiny module, it is going to be a hassle.

Regarding the version history, if we enforce that every time a module is update you have to update the metadata, for example set a new version number before a pr is merged, that way we can use the history of the metadata file to track "tags"

This approach have been observed to work in the wild, just take helm/stable as a example, it works exactly like this.

We could also look into using simple https to store our modules or even look into having a private terraform module repo.

[anarsen]

A lot of software is distributed in the way you propose. I just like to think of Terraform modules being versioned the same way in Nuuday as in the OSS community :-)

In any case, I think this will fall under the Kubernetes Services and Tools squad which was formed today in the Infrastructure and Platforms tribe. This is an interesting discussion, but you should pick it up with @scoof and his team members.

[qvistgaard]

A lot of software is distributed in the way you propose. I just like to think of Terraform modules being versioned the same way in Nuuday as in the OSS community :-)

As my suggestion is part of the official Terraform documentation I don't see why we cannot do it this way. Anyway, if no one has any thing else, I would like to start looking into this and update the pipeline. This is a pressing issue for me, and I have several addons like terraform-aws-eks-grafana that I would like to bring into this repo, and I Don't want to do this before this has been addressed in one way or the other.

[anarsen]

I didn’t say you couldn’t :-)

But do talk to the Kubertes Services and Tools squad about it as they own this now.

[qvistgaard]

Ok, so I have setup a basic pipeline that uses this approach. Before we can start using it we need to agree on somewhere we can put all these modules, maybe, someone in your end can provide a s3 bucket?