Support cross compiling

[Un1q32]

Closes #48

Handles signum.h by pre-generating the header for different operating systems (does it ever differ by architecture?)
Handles tryexec tests by assuming success when cross compiling.
Handles operating system detection by requiring the user to specify it with an environment variable.
Handles POSIX/XOPEN versions by guessing 200112 and 600.
Made the getcwd(NULL, 0) check default to not supported to be safe, and added an environment variable to override it.
Added the --host option to configure, this adds a cross prefix to cc and ar, so if you pass --host=aarch64-linux-musl then configure will look for aarch64-linux-musl-gcc and aarch64-linux-musl-ar

[silvanshade]

Thanks for this.

I have a use case where I need cross-compilation for yash and this PR seems to work.

One suggested addition would be to support the usual --build option for configure:

./configure --build=x86_64-unknown-linux-gnu --host=aarch64-unknown-linux-gnu

The --build option will often be passed by default when cross-compiling with various tooling, such as nix, and the OS can be determined from this value instead of needing a separate environment var.

[Un1q32]

the OS can be determined from this value instead of needing a separate environment var.

--build specifies the system we're building on, not the target. The $OS environment variable specifies the target OS.
I don't think there's anywhere in the configure script that needs to know a triple for the build environment, but I can make the build option be accepted but ignored for autoconf compatibility.

Determining the OS from the triple would be complex, since the triple is in the format {cpu}-{vendor}-{os}{version}-{abi} but sometimes the abi or vendor or both are ommited, so for an example I don't know an easy way to tell if x86_64-linux-gnu means an OS of linux or gnu, or if arm64-apple-darwin22 means an os of darwin or apple. I think autoconf solves this by having a list of known values used as the vendor to check if the os is the 2nd or 3rd field, but then the list would have to be continuously updated and it's a lot easier to just make the user specify it in an environment variable.

[silvanshade]

the OS can be determined from this value instead of needing a separate environment var.

--build specifies the system we're building on, not the target.

Right.

The $OS environment variable specifies the target OS. I don't think there's anywhere in the configure script that needs to know a triple for the build environment, but I can make the build option be accepted but ignored for autoconf compatibility.

Okay, I see now.

When I first tried using this, I was cross-compiling with nix, which as I mentioned passes the --build= flag by default, which in this case gave an error. After modifying the build script to omit that flag, then the yash configure complained about not being able to determine the OS, and I just assumed (though it seemed odd) that it had something to do with --build.

Maybe it would be clearer if it were named $HOST_OS or even $YASH_HOST_OS?

Determining the OS from the triple would be complex, since the triple is in the format {cpu}-{vendor}-{os}{version}-{abi} but sometimes the abi or vendor or both are ommited, so for an example I don't know an easy way to tell if x86_64-linux-gnu means an OS of linux or gnu, or if arm64-apple-darwin22 means an os of darwin or apple.

Theoretically, I agree that extracting the correct information for the triple with perfect accuracy has a number of issues.

In practice, I don't think it's really a problem though, due to the regularity and small set of actual triples that are encountered on a normal basis.

I think the vast majority will be of the form {arch}-apple-darwin or {arch}-unknown-freebsd or {arch}-unknown-linux-{gnu,musl}. I don't know that the darwin vs apple distinction is really meaningful for this case, is it?

A best-effort heuristic could catch probably 95% of the cases and then for the remaining ambiguous cases you could fall back to requiring the explicit environment variable or another flag or something.

But it's just a suggestion.

I think autoconf solves this by having a list of known values used as the vendor to check if the os is the 2nd or 3rd field, but then the list would have to be continuously updated and it's a lot easier to just make the user specify it in an environment variable.

True, it's not a huge burden, but the downside is that it still requires an extra configuration step for consumers.

For the case of nix which I was referencing before, the yash cross-compile would have "just worked" (with this patch) like most other autoconf-based packages, if:

1. the --build= flag were accepted (even if ignored)
2. the --host= flag were used to determine the host OS

[Un1q32]

I think the vast majority will be of the form {arch}-apple-darwin or {arch}-unknown-freebsd or {arch}-unknown-linux-{gnu,musl}. I don't know that the darwin vs apple distinction is really meaningful for this case, is it?

Omitting vendor is extremely common, like half of all linux cross toolchains do it, and the remaining have lots of different vendors, unknown, pc, redhat, and suse are just 4 of the ones I know of. The only solution is to have either a list of allowed operating systems or a list of allowed vendors that we can use to check which fields are which. Failing only on edge cases isn't viable either since we would still need to have a list of operating systems or vendors to determine what is and isn't an edge case.

[silvanshade]

Omitting vendor is extremely common, like half of all linux cross toolchains do it, and the remaining have lots of different vendors, unknown, pc, redhat, and suse are just 4 of the ones I know of.

I guess I don't understand why this is a problem.

Why does whether or not the vendor is included in the triple matter for determining the OS? Are you intending to eventually use that information for some implementations?

Failing only on edge cases isn't viable either since we would still need to have a list of operating systems or vendors to determine what is and isn't an edge case.

Isn't that already effectively the case given that there OS-specific signum implementations in the PR?

Consider a very simple approach like this:

if `OS` is set, use that value
else if `--host=<triple>` contains substring `darwin` then `OS=darwin`
else if `--host=<triple>` contains substring `freebsd` then `OS=freebsd`
else if `--host=<triple>` contains substring `linux` then `OS=linux`
else if `--host=<triple>` contains substring `openbsd` then `OS=openbsd`
otherwise error and request user set `OS`

Wouldn't that work?

Given that (as far as I'm aware) no triple contains multiple OS values, that should always be unambiguous, right?

[Un1q32]

I don't want to do that because as it is right now all you need to do to allow a new OS to be cross compiled for is put a file in the root of the build folder called <os>-signum.h, but having a list would mean you also have to update that list and that's more work. Maybe I could auto generate the list of allowed OSes by checking what *-signum.h files exist.

[silvanshade]

but having a list would mean you also have to update that list and that's more work.

Fair point.

Maybe I could auto generate the list of allowed OSes by checking what *-signum.h files exist.

This seems like it should suffice, and cheap enough to generate on the fly in the configure script just from the file listings and then checking for the *-signum.h prefixes as substrings in host.

[silvanshade]

but having a list would mean you also have to update that list and that's more work.

Fair point.

Maybe I could auto generate the list of allowed OSes by checking what *-signum.h files exist.

This seems like it should suffice, and cheap enough to generate on the fly in the configure script just from the file listings and then checking for the *-signum.h prefixes as substrings in host.

I added a commit that implements this in my local branch: silvanshade/yash:target-os-from-host

[Un1q32]

I could probably do it without using find or grep, I'll try later.

[magicant]

Thank you so much for the pull request. I probably wouldn't have implemented this myself.

[silvanshade]

Maybe I could auto generate the list of allowed OSes by checking what *-signum.h files exist.

This seems like it should suffice, and cheap enough to generate on the fly in the configure script just from the file listings and then checking for the *-signum.h prefixes as substrings in host.

I added a commit that implements this in my local branch: silvanshade/yash:target-os-from-host

I could probably do it without using find or grep, I'll try later.

@Un1q32 Are you still intending to add this?

The configure script already uses find and grep so it seems like it should be fine to use them here.

[Un1q32]

@Un1q32 Are you still intending to add this?

The configure script already uses find and grep so it seems like it should be fine to use them here.

Never got around to it, probably never will, I merged your solution in for now at least.

[magicant]

I made a cross-signum.h that should work on all targets (but less efficiently) on my branch. Does it seem to work for you? cd9ee40

[Un1q32]

Any reason you can't do 4d33fa3? Would improve efficiency of the cross build.

[magicant]

Any reason you can't do 4d33fa3? Would improve efficiency of the cross build.

SIGRTMAX and SIGRTMIN are not guaranteed to be compile-time constants. If those macros expand to something like (__get_current_sigrtmax()), they cannot be used as the size of rttrap_command, which is a statically allocated array.

[Un1q32]

There could be a configure check to see if the value would be constant when cross compiling, I might do that another time. Other than that the commit looks fine except that makesignum should have a 2> /dev/null to not show the error when it fails to run when cross compiling.

[Un1q32]

Wait couldn't you still do the

#if defined SIGRTMIN && defined SIGRTMAX
// do whatever
#else
#define 0
#endif

[Un1q32]

nvm every use of RTSIZE is guarded by that condition anyway so it wouldn't matter.

[Un1q32]

bug with cross compiling detection

[Un1q32]

Unless anyone wants any more features I think this is complete. I've tested cross compiling for mac from linux and everything works. I can't think of anything more this needs.

[Un1q32]

With 0c40a88 it should be possible to make CI automatically test the cross build without needing to setup an emulator or anything, but I don't know enough about your CI setup to set that up myself.

[Un1q32]

Actually why is the default value for RTSIZE 100? Does it work on MIPS Linux where the value generated by makesignum is 102?

[magicant]

Actually why is the default value for RTSIZE 100? Does it work on MIPS Linux where the value generated by makesignum is 102?

Oops, just an oversight.

[Un1q32]

Actually why is the default value for RTSIZE 100? Does it work on MIPS Linux where the value generated by makesignum is 102?

Oops, just an oversight.

What would a better default value be?

[magicant]

What would a better default value be?

I'd go with 255.

[Un1q32]

Anything more you want from this? I would consider this complete now.

[magicant]

I'll merge this tomorrow.