icu_capi's Rust-facing API should not lock us into file structure

[sffc]

Currently it is stable icu_capi API that we have things like icu_capi::collections_sets::ffi::CodePointSetBuilder

This is not good because it locks us into a particular file structure.

Remedy: export the types from Rust in a single top-level ffi module, like icu_capi::ffi::CodePointSetBuilder.

@Manishearth

[Manishearth]

Honestly, my opinion is that we should declare the entire icu_capi Rust API to be perma unstable. It's not intended to be used from Rust except for the narrow case of extending the capi (in which case there's a lot of other tricky unstable stuff too; it's similar to provider in that sense).

I'd like the freedom to rename things/etc: Diplomat already constrains us a fair amount here and I don't want to be constrained further.

[sffc]

Honestly, my opinion is that we should declare the entire icu_capi Rust API to be perma unstable.

I'm not opposed to that, but then why did we make a 1.x release of it

[sffc]

It's not intended to be used from Rust except for the narrow case of extending the capi (in which case there's a lot of other tricky unstable stuff too; it's similar to provider in that sense).

I used it in one of my Google integrations to use try_new_unstable with a custom data struct. I exported a C function (not with diplomat) that returned my boxed-up object, which I was then able to hand over to our Diplomat types for lifecycle management.

FFI aims to be near 100% but there will always be some gaps and this is a reasonable way to fill those gaps.

I'd like the freedom to rename things/etc: Diplomat already constrains us a fair amount here and I don't want to be constrained further.

I considered this, but it's already the case that the FFI names are derived from the Rust names. We can't just rename these types willy nilly.

The only thing this would really lock us into is the Rust type corresponding to the FFI type.

[robertbastian]

I'm not opposed to that, but then why did we make a 1.x release of it

Because it's versioned with the rest of ICU4X. Major releases can still contain unstable code.

I also don't consider icu_capi's Rust API to be public. It would be more helpful if the Rustdoc actually listed the extern C functions.

[Manishearth]

I'm not opposed to that, but then why did we make a 1.x release of it

Same reason as provider code? It's useful to expose for niche users. It's fine to release a 1.x crate and declare a section or all of it to have a modified stability guarantee.

FFI aims to be near 100% but there will always be some gaps and this is a reasonable way to fill those gaps.

Same argument for data patching. I don't think these should be private. I think they should be unstable.

It's a specialized power user use case. Fine to be unstable.

The only thing this would really lock us into is the Rust type corresponding to the FFI type.

Right, but it greatly complicates things like renames at times. We have a whole plan for smoothly doing things like versioned structs that to some extent relies on this. Sufficient attribute usage can work around it but it'll get messy.

[Manishearth]

To be clear, I have never assumed this library to have a stable Rust API, and if people have not been on the same page about that, we may need to spend some time cleaning it up, and spend some time revisiting past decisions on how we do renames/etc. I'd rather not do that.

To me "this should be stable" represents a surprising change in our position that needs care to be made, but I understand that my position was never made explicit earlier so it's not really a "change". But if we're deciding this, it's a new piece of nontrivial work blocking 2.0.

At the very least, let's declare it unstable for 2.0 and have a separate broader discussion about grooming that API, it's not just about the module locations.

[sffc]

We consider icu_experimental perma-unstable, and we release it with its own version scheme alongside the main version scheme for the rest of ICU4X. Maybe we should have done something similar for icu_capi.

To be clear, I have never assumed this library to have a stable Rust API, and if people have not been on the same page about that, we may need to spend some time cleaning it up, and spend some time revisiting past decisions on how we do renames/etc. I'd rather not do that.

To me "this should be stable" represents a surprising change in our position that needs care to be made, but I understand that my position was never made explicit earlier so it's not really a "change". But if we're deciding this, it's a new piece of nontrivial work blocking 2.0.

I've been soft-assuming it was stable, perhaps by accident, and it occurred to me that we should probably make an Actual Policy Decision instead of blindly releasing it again as 2.0. If we want it to be Perma Unstable, then we should reflect that in the semver for the crate. It is a bug to not follow semver.

[sffc]

A middle ground might be to declare the ffi modules to be unstable in the same way that the provider modules in component crates are unstable. That should cover nearly 100% of icu_capi.

[sffc]

I suggest we spend a maximum of 15 minutes in the WG meeting tomorrow to reach a conclusion on this.

[Manishearth]

It is a bug to not follow semver.

Then what of the provider modules? They have the same problem.

Releasing it with a similar versioning scheme as icu_experimental is materially worse: we lose the ability to indicate semver for the actual real bindings (which we do guarantee, to some extent), and this is worse off for things like google3 imports where "new major version" is semantically different from "new minor version" from a review and build perspective.

This crate is not intended to be used from Rust, except by power users.

A middle ground might be to declare the ffi modules to be unstable in the same way that the provider modules in component crates are unstable. That should cover nearly 100% of icu_capi.

I'm fine with that. Though I also think we shouldn't have anything that is not in those modules, so this distinction is meaningless to me. I'd rather just blanket make the statement for the crate instead of then having to worry about stuff accidentally exported via non-bridge code.

[sffc]

Then what of the provider modules? They have the same problem.

For the record, I wanted to make a similar change to the provider modules.

#4193 (comment)

[Manishearth]

I'm totally in favor of sticking absolutely everything under icu_capi::rust_api_unstable

(we can use internal private reexports to make this not suck in code)