interp can't handle regexp.MustCompile because it gives up on panic()

[dkegel-fastly]

interp very sensibly avoids evaluating init functions with side effects.
panic() has the side effect of halting the program.
This means that regexp.MustCompile() is always executed purely at runtime instead of compiletime... leading to slow startup for edge applications, where init runs for every request.

Thinking it through, though, it would be completely reasonable for panic()
inside an init function to halt compilation. It would give a better
developer experience - invalid regexps in init functions would halt
compilation! - and a better runtime experience, particularly on the edge.

[aykevl]

This means that regexp.MustCompile() is always executed purely at runtime instead of compiletime... leading to slow startup for edge applications, where init runs for every request.

As long as the panic call itself is not reached (which is the case for valid regexp expressions), this is not what is causing it to run at runtime. There may be other reasons that prevent regexp.MustCompile from being run at compile time, though.

Thinking it through, though, it would be completely reasonable for panic()
inside an init function to halt compilation. It would give a better
developer experience - invalid regexps in init functions would halt
compilation! - and a better runtime experience, particularly on the edge.

True... but you'd also see the panic when you run the program for the first time. And can then fix the issue easily. We could do this at compile time, but I don't think the added complexity is worth it (interp is already far too complex).

[dkegel-fastly]

I currently use this wrapper to avoid the problem:

in my code, I call RegexpMustCompile instead of regexp.MustCompile in top level variable initializations.

During plain old go development, this uses panic(), and the regexps are all valid, so there is no panic.

During tinygo builds for production, it skips the panic(), and there is no regexp compilation overhead at startup (verified with profiling). This saves us a millisecond or two, which is rather important.

util/regexp.go:

package util

import "regexp"

// RegexpMustCompile is just like regexp.MustCompile, but does not panic if the
// build tag tinygo.interp.regexp.mustcompile is set.
// That way, in production builds, tinygo will precompile
// the regexp at build time, reducing startup time.
//
// This is just here to work around the fact that panic()
// causes tinygo's interp phase to give up and kick the
// panic to runtime.  A future enhancement to
// tinygo might let us go back to using regexp.MustCompile.
func RegexpMustCompile(str string) (r *regexp.Regexp) {
        r, err := regexp.Compile(str)
        if err != nil {
                BuildPanic(err.Error())
        }
        return r
}

util/panic_dev.go:

//go:build !tinygo.interp.regexp.mustcompile

package util

// BuildPanic is a build-time panic; it does nothing if the build tag
// tinygo.interp.regexp.mustcompile is set.
// This is solely for things that can be checked at build time, e.g. RegexpMustCompile
func BuildPanic(message string) {
        panic(message)
}

util/panic_nodev.go:

//go:build tinygo.interp.regexp.mustcompile

package util

// BuildPanic is a build-time panic; it does nothing if the build tag
// tinygo.interp.regexp.mustcompile is set.
// This is solely for things that can be checked at build time, e.g. RegexpMustCompile
func BuildPanic(_ string) {}

[dkegel-fastly]

I suppose the benefit of having the panic at build time would go away if the slow compiles were fixed. We regularly wait two minutes for compiles because of #4980 and/or #2090 (and longer in CI). But the init time performance benefit would remain.

[aykevl]

I checked, and a line like this:

var re = regexp.MustCompile("asdf")

indeed can't run at compile time, but it's not because of the panic. Something much deeper in the call stack happens that causes interp to abort.

In fact, code like this:

func init() {
        regexp.Compile("asdf")
}

also aborts. It doesn't get processed at compile time.
Are you sure your workaround actually does what it is meant to do?

[aykevl]

You can debug it with a small patch/hack:

--- a/interp/interp.go
+++ b/interp/interp.go
@@ -114,6 +114,7 @@ func Run(mod llvm.Module, timeout time.Duration, debug bool) error {
                        return errorAt(call, "interp: expected all instructions in "+initAll.Name()+" to be *.init() calls")
                }
                r.pkgName = initName[:len(initName)-len(".init")]
+               r.debug = r.pkgName == "main"
                fn := call.CalledValue()
                if r.debug {
                        fmt.Fprintln(os.Stderr, "call:", fn.Name())

It will dump a bunch of output of what happens during interp time (and can be quite lengthy if you're not careful!).

[aykevl]

With a small fix (see #5055) I was able to pinpoint the exact line that causes the revert:

https://github.com/golang/go/blob/go1.24.0/src/regexp/syntax/parse.go#L927

Not sure why it reverts there, yet, but it's something.

[aykevl]

Oh. I see. interp is choking on functions with defer. That makes sense. It doesn't stop right away, but eventually it hits a point where it can't continue.

Specifically, I think it's this defer:
https://github.com/golang/go/blob/go1.24.0/src/regexp/syntax/parse.go#L891

This is probably fixable.