CLCs are unversioned

[ajeddeloh]

Issue Report

Container Linux Configs (CLCs) are not currently versioned. This means there is no clear contract about how deprecations or behavior changes are expected to work. CLCs should be versioned.

Versioning proposal:

• Use semver
• Support translating old config versions to the latest config version (in the same way Ignition handles this). This means a version 1.0.0 CLC will always be usable and be supported indefinitely.
• If we do ever need to drop support for a config version, it should have a clear deprecation window with plenty of notice. This is not expected to happen.
• Release new spec major versions instead of deprecating old fields. This means to update a 1.x.y config to a 2.0.0 config changes to the config other than just the version will be necessary.

[bgilbert]

We'll also need to handle existing, unversioned CLCs. We probably don't want to support them indefinitely, since it'd be easy to to accidentally omit the version declaration and thus select the wrong CLC version. Proposal: retroactively assign version 1.0.0 to the current CLC spec, then accept unversioned CLCs until a well-defined end date (e.g. a year) with a warning advising users to add a version declaration by then. Afterward, drop support for unversioned CLCs.

[lucab]

Related question: do we plan to let ct ingest multiple types of configurations for different OSes, or do we plan to have dedicated per-OS tools?

If the former, it may make sense to not use semver and instead adopt a top-level type+version kind (pretty much the same as this), and internally translate to the latest per-OS config and then to ignition.

[ajeddeloh]

We haven't decided that yet. I agree adding some kind of type+version kind is probably a good idea. Otherwise people will think configs for different distros are interchangable. We could even swing something where different top level sections (e.g. flannel, etcd, etc) are supported on some distros but not others.

[lucab]

[Fleshing some more details from my cryptic comment above]

Ignition config spec prescribes a mandatory SemVer-compatible field called ignition.version with the following guarantees:

the semantic version number of the spec. The spec version must be compatible with the latest version (2.2.0). Compatibility requires the major versions to match and the spec version be less than or equal to the latest version. -experimental versions compare less than the final version with the same number, and previous experimental versions are not accepted.

I'm a fan of semver for libraries/APIs, but for this kind of configuration I'd prefer instead an approach with an explicit tag that simply describes what's the format of the configuration object. For an example, torcx config schemas work that way.
The idea is that the internal representation is anyway different from the input, and the parser only needs a simple way to detect the input type.

Changes (breaking or not) are handled by incremental revision numbers and older revisions are accepted for compatibility (until some planned purge-point, if desired). This means there are no special micro-compatible versions and deprecated/moved/renamed fields can be directly changed on the next revision bump. Users can keep their old configs working at old revisions, or jump into new revisions with new features and deprecations.

In practice, for ct this would look like:

• day-0: kind: cl-config-v0
• day-1: kind: cl-config-v1-exp
• day-10: kind: cl-config-v1
• day-20: kind: fcos-config-v0-exp
• ...

[ajeddeloh]

Is your "day-20" kind supposed to be fcos-config-v2-exp? If not, I don't follow.

Assuming it is: How do you think that should be implemented? I'd prefer to not have to do a massive translation dance like we do with Ignition if we can avoid it.

[crawford]

Why bother fixing this when we know that Container Linux has a limited lifetime? Can we just leave it as is and fix the issue in the successor to Container Linux?

[ajeddeloh]

Because the CL lifetime is long enough to warrant it and we may want to make changes. Better than either breaking people unexpectedly or carrying a monstrosity of a codebase to handle weird backwards compat constraints (e.g. moving fields around).

[crawford]

Because the CL lifetime is long enough to warrant it and we may want to make changes.

I'm not convinced that the lifetime is long enough. We can also just agree not to make breaking changes to the config going forward. This is the approach we've taken up to this point and it has worked fine.

This seems like a tremendous amount of effort for very little gain.

[lucab]

Clarified out-of-band: fcos- and cl- are separate genealogy which can live and progress in parallel.

Yes, it would probably entail an external->internal translation dance (on top of the later ignition transpilation), but I don't know how much work this requires on ct internal. I did a similar refactoring on torcx (splitting external input and internal state) and it wasn't a lot of work, but I understand ct is much heavier on config parsing/processing logic.

This came up as a result of the first breaking change proposed in coreos/container-linux-config-transpiler#158 (comment). In the best case, this rework can be directly used to support FCOS and CL the same way. In the worst case, this only allows us to snapshot/freeze the current config format.