extending Go backward compatibility

[rsc]

This discussion is about backward compatibility, meaning new versions of Go compiling older Go code. For the problem of old versions of Go compiling newer Go code, see this other discussion about forward compatibility.

Go 1 introduced Go's compatibility promise, which says that old programs will by and large continue to run correctly in new versions of Go. There is an exception for security problems and certain other implementation overfitting. For example, code that depends on a given type not implementing a particular interface may change behavior when the type adds a new method, which we are allowed to do.

We now have about ten years of experience with Go 1 compatibility. In general it works very well for the Go team and for users. However, there are also practices we've developed since then that it doesn't capture (specifically GODEBUG settings), and there are still times when users’ programs break. I think it is worth extending our approach to try to break programs even less often, as well as to explicitly codify GODEBUG settings and clarify when they are and are not appropriate.

As background, I've been talking to the Kubernetes team about their experiences with Go. It turns out that Go's been averaging about one Kubernetes-breaking change per year for the past few years. I don't think Kubernetes is an outlier here: I expect most large projects have similar experiences. Once per year is not high, but it's not zero either, and our goal with Go 1 compatibility is zero.

Here are some examples of Kubernetes-breaking changes that we've made:

• Go 1.17 changed net.ParseIP to reject addresses with leading zeros, like 0127.0000.0000.0001. Go interpreted them as decimal, following some RFCs, while all BSD-derived systems interpret them as octal. Rejecting them avoids taking part in parser misalignment bugs. (Here is an arguably exaggerated security report.)

  Kubernetes clusters may have stored configs using such addresses, so this bug required them to make a copy of the parsers in order to keep accessing old data. In the interim, they were blocked from updating to Go 1.17.

• Go 1.15 changed crypto/x509 not to fall back to a certificate's CN field to find a host name when the SAN field was omitted. The old behavior was preserved when using GODEBUG=x509ignoreCN=0. Go 1.17 removed support for that setting.

  The Go 1.15 change broke a Kubernetes test and required a warning to users in Kubernetes 1.19 release notes.

  The Kubernetes 1.23 release notes warned users who were using the GODEBUG override that it was gone.

• Go 1.18 dropped support for SHA1 certificates, with a GODEBUG=x509sha1=1 override. We announced removal of that setting for Go 1.19 but changed plans on request from Kubernetes. SHA1 certificates are apparently still used by some enterprise CAs for on-prem Kubernetes installations.

• Go 1.19 changed LookPath behavior to remove an important class of security bugs, but the change may also break existing programs, so we included a GODEBUG=execerrdot=0 override.

  The impact of this change on Kubernetes is still uncertain: the Kubernetes developers flagged it as risky enough to warrant further investigation.

These kinds of behavioral changes don't only cause pain for Kubernetes developers and users. They also make it impossible to update older, long-term-supported versions of Kubernetes to a newer version of Go. Those older versions don't have the same access to performance improvements and bug fixes. Again, this is not specific to Kubernetes. I am sure lots of projects are in similar situations.

As the examples show, over time we've adopted a practice of being able to opt out of these risky changes using GODEBUG settings. The examples also show that we have probably been too aggressive about removing those settings. But the settings themselves have clearly become an important part of Go's compatibility story.

Other important compatibility-related GODEBUG settings include:

• GODEBUG=asyncpreemptoff=1 disables signal-based goroutine preemption, which occasionally uncovers operating system bugs.
• GODEBUG=cgocheck=0 disables the runtime's cgo pointer checks.
• GODEBUG=cpu.<extension>=off disables use of a particular CPU extension at run time.
• GODEBUG=http2client=0 disables client-side HTTP/2.
• GODEBUG=http2server=0 disables server-side HTTP/2.
• GODEBUG=netdns=cgo forces use of the cgo resolver.
• GODEBUG=netdns=go forces use of the Go DNS resolver

Programs that need one to use these can usually set the GODEBUG variable in func init of package main, but for runtime variables, that's too late: the runtime reads the variable early in Go program startup, before any of the user program has run yet. For those programs, the environment variable must be set in the execution environment. It cannot be “carried with” the program.

Another problem with the GODEBUGs is that you have to know they exist. If you have a large system written for Go 1.17 and want to update to Go 1.18's toolchain, you need to know which settings to flip to keep as close to Go 1.17 semantics as possible.

I believe that we should make it even easier and safer for large projects like Kubernetes to update to new Go releases. In particular, I think we should probably:

• Commit to gating these kinds of allowed but potentially-breaking runtime behavior changes by GODEBUG settings. Some settings may persist forever (execerrdot=0 seems like one). Others may be retired after a multiyear window (x509sha1=1 seems like one), instead of the “one release” window we’ve attempted in the past.
• Export a counter of the number of code executions that behaved differently from standard Go due to a GODEBUG setting, through the runtime/metrics interface. (For example, /godebug/execerrdot=0 would increment each time that setting suppresses an ErrDot error.) Programs can then watch that counter themselves.
• Provide some easy way for main packages to set their default GODEBUG in source code. (That's already partly possible today with os.Setenv at the start of main, but that approach fails for runtime settings and for settings consulted at package init time. Note that main's init runs after all other package inits.)
• Provide some easy way for programs to say “set the GODEBUGs to look like as much like Go 1.X as possible” for some earlier release of Go.
• Note that none of this lowers the bar on making incompatible changes: we still avoid as much as possible, following https://go.dev/doc/go1compat.

GODEBUG may not be the mechanism I'd design today, but it's what we have and it doesn't seem bad enough to be worth adding a second way, so I'm going to assume it stays. Then the two things we need are (1) a way to set individual GODEBUG defaults in package main, and (2) a way to make the default GODEBUGs match an earlier version of Go.

For (1), I am thinking about something like

//go:debug x509sha1=1

in any package main source file. These would be pulled out by the go command and linked into the binary for processing at startup (before any Go code runs). The GODEBUG environment variable would still override these, of course.

For (2), I am thinking about having the go line in the go.mod of package main's module, which already defines the exact language semantics of package main's Go source files, also define the default GODEBUG settings. So if package main's go.mod says go 1.17, SHA1 certificates still work, and os/exec does not generate ErrDot.

As noted above, some GODEBUG settings will stick around forever (for example, execerrdot=1, netdns=go), while we will want to retire others. When a newer version of Go is compiling code written for an older version, if a GODEBUG has been retired, the behavior will depend on whether it is named explicitly (as in (1)) or implicitly (as in (2)). If a retired GODEBUG is mentioned explicitly, the build should fail. If it is only implied by the earlier Go version, then build should succeed, on the assumption that the vast majority of programs that say “go 1.17” are saying it because they were written in that era, not because they require support for SHA1 certificates.

I think these two changes would go a long way toward making it even easier and safer to update to new Go toolchains, because it separates the update from the riskiest behavior changes and makes those changes easy to temporarily opt out of and also to debug.

Note that this mechanism would be inappropriate to use for new, incompatible features, because the settings in package main are affecting the entire binary, and it is unlikely that all the packages in a large program would agree on which version of a large feature they want. In contrast, for “extra-backwards compatibility shims” like these, especially in the context where you're keeping older code running, affecting the whole binary is appropriate and does not cause problems.

One thing people ask occasionally is whether Go will ever add LTS (long-term support) releases. I've always thought of Go 1 as the LTS release of Go, but subtle, necessary breaking changes like the ones listed above contradicted that idea. Being able to use a new Go toolchain but still get more faithful “old” semantics in these cases brings us much closer to Go 1 as LTS.

There is a question about what to do if go.mod says a newer version of Go than the toolchain being used for the build. In that case the older toolchain does not know what the newer version does differently. I am filing a separate discussion about this forward compatibility problem.

This is a discussion, not a proposal. I haven't implemented this nor even worked out all the implications. I'm curious what people think and what concerns they have. Thanks!

[liggitt]

For (2), I am thinking about having the go line in the go.mod of package main's module, which already defines the exact language semantics of package main's Go source files, also define the default GODEBUG settings. So if package main's go.mod says go 1.17, SHA1 certificates still work, and os/exec does not generate ErrDot.

I like the simplicity of this a lot. For what it's worth, "behave like go1.x" is actually what I expected the go directive to do when I first learned about modules, and is what it already accomplishes for the main module w.r.t. language features and go command behavior. Expanding that to include whatever runtime compatibility switches exist seems sensible to me.

[BenTheElder]

As mentioned in #55090 (reply in thread) it also seems like that was the intent. rust editions (mentioned in that comment) also seem like really clear precedent for this. I wonder if anyone can weigh in on how well that's working in the rust ecosystesm.

[apparentlymart]

A significant detail about Rust editions is that they cannot typically change the standard library, and instead focus on the language. I gather this is because the library code is linked once into the program and shared between all callers.

This is in turn because editions are a per-crate decision rather than a main-crate decision. This avoids splitting the ecosystem by allowing multiple language versions to exist in the program at once.

The current treatment of the go directive, e.g. activating type parameters in 1.18, seems to roughly match the intent of Rust additions. This further idea of making the main program's selected version also affect library behavior doesn't have precedent in Rust editions as far as I know.

I don't mean to say that it isn't a good idea just because Rust doesn't do it! I think it is an interesting idea to investigate, and given that we're talking about a small number of very specific changes made for exceptional reasons only (as opposed to becoming a broad way to evolve the standard library in incompatible ways) it seems nice and pragmatic to me.

[randall77]

The GODEBUG environment variable would still override these, of course.

We'd need to have the GODEBUG settings override field-by-field, probably. So if you did

//go:debug: x509sha1=1

In front of main, and then ran your program by setting GODEBUG=asyncpreemptoff=1, we'd want the x509sha1 setting to still survive. Only if you set GODEBUG=x509sha1=off would the default setting be overridden.

[rsc]

Indeed. That's exactly right: the //go:debug lines would set the default values for specific settings, which could then be overridden individually by $GODEBUG.

[josharian]

Overall, a hearty +1.

One downside to the implicit go.mod behavior is that getting all security improvements from a new Go version now requires actively editing go.mod.

OTOH, this matches MVS pretty well, so maybe that’s a point in its favor.

[rsc]

Indeed, and I think security deprecations are generally different from security fixes.

[josharian]

That does also raise a question about security point releases. Suppose the LookPath behavior was bad enough that it needed to get fixed in a dot release, but it still needs backwards compatibility support. Now the implicit go.mod line behavior isn't really enough to provide stability, because go.mod specifies only the major Go version.

Does this mean that some classes of security fixes (those with backwards compatibility issues) are only possible to make in major Go releases?

[rsc]

Does this mean that some classes of security fixes (those with backwards compatibility issues) are only possible to make in major Go releases?

Yes, and I believe that is true already. One of the hard rules of point releases is it really must not break anyone, because we never want someone to be unable to add an urgent security fix due to some breakage in that same point release or an earlier one in the sequence. That applies to the security fixes themselves too.

LookPath is a good example. There was a reported bug affecting go toolchain programs, and we fixed the bug by making the LookPath change in a forked copy of os/exec specifically for those programs. We left the toolchain-wide fix for a major Go release precisely because of the compatibility issue.

The same is true of net.ParseIP. We decided it was an important security-hardening fix but on balance inappropriate for a point release because of the potential for breakage.

It's hard for me to think of a security problem that would be so critical that it must be fixed in a point release and simultaneously so broad that the fix fundamentally must break unaffected user programs as collateral damage. To date I believe we've always found a way to avoid such a fix, and I think the onus is on those of us preparing security releases to continue to do that.

[josharian]

Thanks. I think this is an important enough point that, if this gets written up as a proposal, it should be called out explicitly.

[xdg]

FWIW, the go.mod approach is similar to what Perl (which also strongly prioritizes backwards compatibility) is doing. See the middle of this blog post from the Perl Steering Council for some examples.

[rsc]

Thanks. Yes, indeed. I'm not sure I had seen this Perl thing before, but it's also similar to what Rust is doing with editions. Enabling this kind of backward compatibility story was definitely one of the reasons we put the go version into the go.mod in the first place. (We didn't have the GODEBUG things in mind at the time, just language versions, which we already use it for.)

[josharian]

Right now, if a dependency has a higher go.mod Go version number than package main, but the code compiles, everything is fine. Would that change? If not, how does a dependency indicate that it depends on new behavior? If so, it seems like this might rapidly cause dependency management pain, because everything has a Go toolchain version dependency. (I know this is your other discussion topic, I just wanted to point out that the answer there might turn out to be very important to assessing the viability here.)

[rsc]

It would change, but as you note that change is #55092. I think we have to consider them at the same time, because they are two sides of the same coin, but it also made sense to split the discussions, because the two different directions have different answers. I think we could do this idea without #55092, but it is a stronger story if we include #55092 too.

[prattmic]

Should we add a new name for GODEBUG / //go:debug (as an alias, probably)? These uses of GODEBUG are clearly not for debugging (as many of the initial GODEBUG options are), so it seems potentially confusing, especially having the go.mod version set default GODEBUG values.

This is very minor, but perhaps a chance to make a small improvement.

[kgaughan]

If you want to borrow a term from the FreeBSD ports system, 'knob' could work, though that might not be ideal in some varieties of English. Other possibilities would be 'switch' and 'tweak'.

[daenney]

GOBACKWARDS? It's a bit tongue-in-cheek, but it reasonable implies you're enabling a previous/older behaviour.

[daenney]

Alternatively, GOBEHAVIOR or GOCAPABILITY might have some potential, and could be used for both gating a new experiment in a release as well as an old code path.

[rsc]

Alternatively, GOBEHAVIOR or GOCAPABILITY might have some potential, and could be used for both gating a new experiment in a release as well as an old code path.

This is exactly what we don't want to use it for (and why I excluded GOFEATURE), because it has the potential for splitting the ecosystem like Python 2 vs Python 3. It is very important that this only be used for things that very very very few people will need. Otherwise it's too large a break and shouldn't be done at all.

[prattmic]

I agree the name is hard, as I was planning to put a suggestion in my original message but couldn't think of a good one at the time. :)

The best I've got right now is GOCOMPAT, but maybe GODEBUG is nice just in that it looks like it shouldn't be used!

[willfaught]

This seems like adding another "knob" to build configurations. As @josharian pointed out, there seem to be complexity in resolving how multiple modules with conflicting GODEBUG settings are merged together. Plus, it means that GODEBUG becomes a real, official part of the module system. Just saying "go 1.x" is so simple; I would hate to give that up.

Let me push back and ask whether we should even have GODEBUG in the first place. Various settings are listed above, and they seem to relate to security decisions, like SHA1, or just deciding to break things, like making HTTP/2 the default (if I understand that correctly).

In the former case, those breaking changes are part of the compat promise; old programs breaking with newer Go versions that make those kinds of changes is unfortunate, but expected, and even desired. Broken programs in that context reveal security flaws in them. I'm glad Kubernetes broke in those cases. If they want the benefits of newer Go versions, then they can pay the cost of avoiding those security flaws. This very well may mean that Kubernetes has to adopt a similar breaking change policy for security concerns that Go has. I suppose it's worth debating whether Go should impose that on package makers, but in my opinion, it should. No one should be entitled to upgrade without paying a price in this context.

In the latter case, perhaps those changes shouldn't have been made at all. Instead of making HTTP/2 the default, we could have instead added an http2 package, or added an optional HTTP/2 mode to clients and servers. I can't recall whether the DNS resolver change was security-related, but if it wasn't, that also could have been an opt-in setting in the runtime or net packages. ParseIP probably shouldn't have been changed, since it was working as intended without a security flaw.

Perhaps GODEBUG shouldn't exist at all. If behavior is changed, and it's compatible with the compat promise, then there should be a really good reason for it, and Kubernetes or whoever should be happy that it happened. Perhaps the reason why @rsc was so surprised by how often Kubernetes was broken was because the Kube team knew that, in most cases, most of those breaking changes happened for good reasons, and they accepted it. (Just conjecture. I don't claim to know or speak for them.)

[BenTheElder]

Kubernetes adopts new Go versions relatively quickly we even tend to jump on RCs these days, this isn't about Kubernetes avoiding fixing something ...

But when Go makes breaking changes to say stdlib crypto it is problematic for the our minor version branches as end users are expecting no breaking changes within the lifecycle of those minor versions, and as evidenced in the examples in the original post here sometimes upgrading Go fundamentally makes breaking changes no matter what Kubernetes does.

This is about providing a way for projects like us to upgrade to the latest go without shipping breaking behavioral changes to our end users.

[willfaught]

This is about providing a way for projects like us to upgrade to the latest go without shipping breaking behavioral changes to our end users.

Right. I'm saying that maybe we shouldn't do that, for the reasons I gave. Kube could wait until its next major version release to upgrade Go and absorb the breaking changes, if that's what Kube's versioning policy requires. This problem affects a tiny percentage of Go users, according to @rsc, so I'm questioning the value, cost, and complexity of making this change to module semantics and the build system just to accommodate them when they already have a viable workaround, and things are working as intended.

[dims]

if "things are working as intended", then please update https://go.dev/doc/go1compat to reflect the reality.

Having worked on some of these failures in the kubernetes code base when we "jump on RCs" i think that the go1compat is no longer valid.

No, this is not just about kubernetes alone. other folks are affected too, see example in #51991

Either we work towards making go1compat true or accept that it is false and drop it. let's not pretend please.

[BenTheElder]

Kube could wait until its next major version release to upgrade Go and absorb the breaking changes, if that's what Kube's versioning policy requires.

The versioning policy is just semver. Kubernetes minor versions just last longer than Go's, by user demand.

Users need longer support cycles for their clusters with no breaking changes in on order to run critical infrastructure.

Thus far that requires sticking to eventually unsupported go versions that are not receiving any patches, or sometimes it can be worked around (e.g. by forking the standard library and patching usage to use the forked package).

This problem affects a tiny percentage of Go users, according to @rsc,

I don't see a statement to that effect. I do see this (emphasis mine):

It turns out that Go's been averaging about one Kubernetes-breaking change per year for the past few years. I don't think Kubernetes is an outlier here: I expect most large projects have similar experiences. Once per year is not high, but it's not zero either, and our goal with Go 1 compatibility is zero.

I think rsc is pretty equipped to determine if the complexity is justified for the Go project.

[willfaught]

This problem affects a tiny percentage of Go users, according to @rsc

I don't see a statement to that effect.

Here you go (from this thread):

Continuing the SHA1 example, changing the default to not accept SHA1 certificates makes Go more secure and breaks hardly anyone. The benefits for the 99.99% of users far outweigh the inconvenience for the 0.01%. Should we use semver and start calling it Go 2.0.0 because we no longer accept SHA1 certificates? That makes everyone expect a breakage and then for almost everyone, nothing breaks. That makes no sense.

[micahhausler]

Strong +1 on the ability to preserve backward compatible runtime behavior, I'm glad to see this discussion happening.

I'd also like to see some stated period for how many minor versions a behavior breaking change will be toggleable. Specifically for Kubernetes, there is a ~1yr community support period for release branches. These are not aligned with the Go release cadence so only maintaining compatibility for 1 year in Go will still not satisfy the Kubernetes case.

I would conjecture that a large portion of real-world Kubernetes usage is through distributors which often release 3-6 months behind upstream Kubernetes, and support that release for 12-15 months. This means a particular Kubernetes release's compatibility for end users right now needs just shy of 2 years of compatibility. Distributors and providers certainly could/should be faster to release which would ease the tail end of this, but if Go doesn't provide 18mo to 2 years of support the alternative for distributors is to either live with outdated versions of Go or backport CVE patches to a fork of Go.

[rsc]

I think we could probably commit to saying that any flag stays for at least two years. As I noted, some will stay forever. And also any flag should have usage metrics exported (counting executions that would have run differently with the default behavior, when that behavior is actively suppressed), which should make it easy for deployments of Kubernetes or other systems to watch for when they can safely stop using the flag.

[liggitt]

I really like the idea of runtime metrics for hits to non-default GODEBUG code paths. This helps identify uses that should be fixed, and allows gathering data if there are uses that can't be fixed that weren't anticipated and should inform speed of removal of the compatibility switch.

[willfaught]

Could there be any conflicts between main module info or debug info embedded in binaries built by a newer or older toolchain than the local one installed? For example, what happens if a future Go version uses DWARF-99, or adds a new field to the module info? Would those work with dlv or go version -m commands built with Go v1.older?

[bcmills]

I think this comment belongs on #55092..?

[davecb]

If we look at the go line in the go.mod file to set the ground-rules, should the GODEBUG lines that declare exceptions to the rule reside there as well?

[liggitt]

I prefer the main package override approach, since that gives flexibility if specific binaries within the module need to override defaults differently in more fine-grained ways.

[davecb]

Should we then move the go version there as well?
It seems silly to say "for the definitive answer, see A and then B" (;-))

[davecb]

More importantly, should we have multiple conflicting defaults?

[bcmills]

It seems silly to say "for the definitive answer, see A and then B"

Unfortunately that is already the case anyway.

If my go.mod says go 1.18 but I have a file with a constraint //go:build !go1.18, that file will today only be used when building with Go 1.17 and lower, so it would be at least incoherent to use a Go 1.18 feature in that file.

Similarly, if my go.mod says go 1.18 but I have a file with a constraint //go:build go1.19, that file will be used only when building with Go 1.19 and above, even though the module overall uses Go 1.18 language semantics.

(#30791 is closely related.)

[davecb]

Ow! Sorry to hear that, I know from experience how much "fun" it can be to maintain (;-))

[vpereira01]

Can you expand on the mindset/goal behind wanting to update go while at the same not wanting the new go version behaviors?
I understand partially the ideas behind this but at the same time I also feel like I'm missing something.

Is the problem

1. that behavior changes exist?
2. that it's hard to handle behavior changes?
3. that go does not really honor go version in go.mod?
   • In the sense that go compiler version go 1.35, which removed a GODEBUG flag, when compiling a module with go.mod 1.20 does not produce the behavior of go compiler 1.34
4. Something else?

[liggitt]

Can you expand on the mindset/goal behind wanting to update go while at the same not wanting the new go version behaviors?

Go-based projects need a way to ship maintenance patch updates rebuilt with go versions that fix go runtime vulnerabilities, without picking up go runtime deprecations or significant runtime default behavior changes (e.g. ParseIP changes, default disablement of SHA-1 support, default client change from http/1 to http/2).

Currently, the minor version updates required to get go vulnerability fixes also regularly bring the latter.

[jaredpar]

Go-based projects need a way to ship maintenance patch updates rebuilt with go versions that fix go runtime vulnerabilities, without picking up go runtime deprecations or significant runtime default behavior changes

Indeed. A Go toolset has effectively one year of maintenance attached to it with the N-1 model. A lot of projects, particularly at the corporate level, end up with servicing lifetimes much longer than that. At Microsoft a 3 year model is quite common. Those projects want to be shipping minimal changes throughout the lifetime of the product but taking a new Go toolset can force them to make unwanted significant changes

[davecb]

At Sun we found that bug fixes and deprecation interacted badly. We originally used semantic versioning at the whole-file level, and ran into exactly the kind of conflicts you describe. Fortunately you're in effect dealing with the bugs at the interface level, so you don't just deadlock.

(If you're interested, this is also related to the version problem that Mr Cox described in https://research.swtch.com/version-sat. My take on it is https://leaflessca.wordpress.com/2017/02/12/dll-hell-and-avoiding-an-np-complete-problem/, and Sun's was David J. Brown's https://www.usenix.org/legacy/publications/library/proceedings/als00/2000papers/papers/full_papers/browndavid/browndavid_html/
This stuff used to be my life (;-))

[vpereira01]

Go-based projects need a way to ship maintenance patch updates rebuilt with go versions that fix go runtime vulnerabilities, without picking up go runtime deprecations or significant runtime default behavior changes (e.g. ParseIP changes, default disablement of SHA-1 support, default client change from http/1 to http/2).

Makes sense 👍

Excluding http/2 change, the changes listed as an example were made for security reasons which make this discussion more difficult. The distinction is more/less clarified here #55090 (reply in thread)

Given the effort required to maintain behaviors that were deemed insecure and the security risk of maintaining those behaviors I think that kind of backwards compatibility should be short lived, as in, 3 point releases (deprecated/hidden in three releases, removed on the forth).

[davecb]

If we're talking about a significant number of the GODDEBUG (GOBUG?) settings being for security problems, then we probably need to spend some time considering using the rewriting tools to either provide corrections directly, or to provide blocks of candidate corrections.
At Sun we did the latter, to help as much as we could to avoid someone "depending on the bug" (:-()

[jxsl13]

Imo implicit behavior and hidden magic should be avoided as much as possible.
I would love if we had an optional go.mod section which can be written by a command like go mod features which then generates all key value pairs that can be tweaked in the current version of Go followed by a comment containing an url to the feature documentation.

[davecb]

Yes: a go mod features command is a clean way to ensure you get predictable behavior. I'd probably put the whole table in as a go mod list option, like git stash list, but that's a niggle.

go mod edit sounds like the beginning of a go idiom. Let's have less magic and more idioms: elegant ones!

[fazalmajid]

One example of a breaking change: the change in 1.13 strconv behavior to allow underscores as separators introduced regressions in some of our parsing code where underscore was used to separate fields in a URL format.

[mvdan]

I fully understand that the power to choose the Go version semantics lies on the main package and module, but I wonder how this affects libraries. As a library author, I will only be testing with one version's semantics. If a downstream project chooses newer or older semantics, the library could be silently broken and the toolchain wouldn't help either developer see that, I think.

[ChrisHines]

I don't understand this concern. When we test libraries the library is the main module, so wouldn't it still have control over the versions of the toolchain used for its tests?

[mvdan]

Right, but the library then gets used in other projects, which may choose a different version as the main module/package.

[ChrisHines]

True, but how is that different than the status quo?

[mvdan]

Oh, it's not. My point is that, since the proposal is formalizing "Go version semantics", I wonder what we could do to surface potential incompatibilities at build time before they could cause run-time breakages down the line.

As a strawman idea: a bit of static analysis that could show a warning if a library uses exec.LookPath with go 1.17 but the main module then overrides the version with go 1.19, thus potentially breaking the library.

[rsc]

This discussion has been very helpful. Thanks everyone!

[kevpar]

Wanted to add another datapoint on a Go change that broke a downstream consumer:

16f0f9c changed the default Windows os.OpenFile behavior to create the file as read-only if the u+w permission bit is unset. Previously the permission bits were completely unused on Windows, so we had a place in containerd that passed 0. After updating Go suddenly this call was creating a read-only file unintentionally.

This change wasn't something we could back-out with a GODEBUG setting, but perhaps it should have been.

[rsc]

Thanks for the example. I agree that change may have been a mistake.

[rsc]

This is now a proposal at #56986.