Just how unspecified is repr Rust over extern C?

[CAD97]

Assume a C ABI which is able to be called without a function prototype (is immediately compatible with C's variadic functions without adjustment). That is, the set of assumed, preserved, and clobbered CPU state (e.g. registers) is independent of signature, and any stack space used for arguments is allocated in the caller's stack frame (caller cleanup).

In this case, the following function definition should be valid to call with any list of ffi-safe arguments for the varargs:

#[naked]
unsafe extern "C" fn f(_: i32, _: ...) {
    unsafe { naked_asm!("ret") }
}

In fact, if you define an empty variadic function on x64 Linux, after optimization the assembly is a single ret, as in this naked definition.

The question: given that an extern "C" ABI guarantees that the only difference in call ABI is performing default argument promotion in the caller, is it valid to provide a repr(Rust) as one of the variadic arguments? And if so, is it also valid to call the function through a function pointer transmuted to a non-variadic signature that mentions a repr(Rust) type?

Or, in an alternate phrasing, are all extern "C" fn guaranteed to have a way to write their effective ABI in C? Or are repr(Rust) types in a function signature allowed to arbitrarily modify the calling convention independent of the ABI string specified?

Of course, C variadic support is still unstable currently, but the same scenario as the function pointer transmute can be created on stable by linking to an external symbol.

[RalfJung]

I'm sorry I have no idea what you are asking... but it seems to be a question specific to variadics? But then the last sentence makes it sound like that's not the case. I am lost. Maybe try again but explaining things a bit more slowly and with examples?

[CAD97]

Variadic functions are why the question is meaningful, but not core to the question. The simplest way to ask the question is probably:

What, if anything, is guaranteed about the ABI of extern "C" fn(ReprRustType)?

[RalfJung]

I'd say you can't implement or call such functions from non-Rust code.

[CAD97]

The reason variadics add an interesting wrinkle is that it's possible to implement an extern "C" function which accepts any arguments. If you can't define a function which accepts repr(Rust) types outside of Rust, then such a type can't be processed by such a function, but ignoring the variadic arguments is defined behavior.

So that's the second part of the question: even if you can't predict it, is there a signature that exists by the defined extern "C" ABI for what the ABI of passing a repr(Rust) type, or is that allowed to have a fully arbitrary calling convention?

[RalfJung]

I don't know nearly enough about how variadics work to really say anything with confidence here. So by default my answer would be, you cannot rely on anything. There may not be a C type that corresponds to how a repr(Rust) type is passed.

Maybe in the future we can have proper checks for extern "C" functions to reject types which don't make sense there. We already do that for some vector types, maybe we can slowly expand this.

[CAD97]

I found an answer to the weird part of my question w.r.t. C semantics as well: functions that ignore unused arguments, such as printf, must be called with a prototype in scope (the prototype of such functions necessarily uses the trailing ellipsis parameter) to avoid invoking undefined behavior.

This means that even if it would be valid behavior to e.g. call printf(c"", RustType) (note no formatting specification, thus the extraneous argument shall be evaluated and ignored), this does not imply that it's valid to call (printf as extern "C" fn(*const c_char, ...) as extern "C" fn(*const c_char, RustType))("", RustType). There's still the case where a C extension and/or ABI definition enables you to write a function that accepts any number of arguments and ignores them, but that standard C disallows that is sufficient for me to agree that

• repr(Rust) ABI may differ when passed to extern "C" fn(/* ... */) and extern "C" fn(/* ... */, ...), and
• repr(Rust) ABI may not even have an ABI that can be accepted by C code at all, even with extern "C".

I was thinking about this because I was thinking about how to define "type repr ABI" and "fn extern ABI" again. This allowance makes sense, but does make defining the two fully independently impossible.