☂️ [vm/ffi] Native assets feature

[dcharkes]

Native assets support in Dart standalone is available in Dart 3.2 behind an experimental flag: dart --enable-experiment=native-assets run [path/to/script.dart].

This is a high level "feature" issue that we can refer to from various places.

The current way of bundling native code with Dart packages is far from ideal, we will introduce "Native Assets" to Dart.

Problem

As of right now, one can do one (or more) of the following:

1. ship sources in a Dart standalone package and let users run a build by calling dart run <your-package>/setup.dart see webcrypto, or
2. make your bin/setup.dart download the native library, or
3. ship sources a flutter plugins with flutter create --template=plugin_ffi --platform=..., or
4. ship a script that downloads the native library during flutter build see realm_flutter.

This leads to pain for both package developers (options 1-4) and package users (option 1-2). Moreover, there is currently no way to make a package with native code that works both for flutter and Dart standalone.

High level solution

We introduce the concept of "Native Assets" to the Dart eco system.

A native asset is a native library (compiled from native source code).

Three components of the "Native Assets" solution in Dart:

1. A standardized way to build or download native assets. This will be done by a CLI.
2. A transparent way to find native symbols in these native assets in Dart.
3. Integration with all the dart and flutter tooling to link the above two together.

Try now

Native assets support in Dart standalone is available in Dart 3.2 behind an experimental flag : dart --enable-experiment=native-assets run [path/to/script.dart].

Native assets support in Flutter is in development. Progress can be tracked in flutter/flutter#129757.

An example on how to use native assets can be found here.

Checklist

• Basic Dart support: [vm/ffi] Native asset resolution by the Dart VM for FfiNatives #49803
• Flutter support: ☂️ Native assets support flutter/flutter#129757
• Refine core support:
  • [vm/ffi] Make FfiNatives more concise #50097
  • Allow access to global variables in ffi static binding #50551
  • Allow getting address of native symbols when using static ffi binding #50552
  • [ffi] tree-shaking does not remove dependency .so library from the build #52970
  • [vm/ffi] Explore using FfiNative for static linking #49418
  • dart compile exe support static linking of C code #47718
  • [vm/ffi] FfiNative return types treatment inconsistent with asFunction #49518
  • [native_assets][resource_identifiers] Deferred loading and assets #55809
  • [native_assets] Support dart pub global run #56044
  • [native_assets] Hot restart native libs #55850
• Graduate out from experiment flag (--enable-experiment=native-assets)

[dcharkes]

Related issues:

• [vm/ffi] Native asset resolution by the Dart VM for FfiNatives #49803
• [vm/ffi] Make FfiNatives more concise #50097
• Allow access to global variables in ffi static binding #50551
• Allow getting address of native symbols when using static ffi binding #50552
• dart compile exe support static linking of C code #47718
• [vm/ffi] Explore using FfiNative for static linking #49418
• [vm/ffi] FfiNative return types treatment inconsistent with asFunction #49518
• [ffi] tree-shaking does not remove dependency .so library from the build #52970

Related repositories:

• https://github.com/dart-lang/native

[GregoryConrad]

Hi, as a current user of option number 4 above, this sounds really enticing. Right now I am using CMake for Linux/Windows/Android and Ruby (with inline bash) for macOS/iOS to download & extract binaries and it is a significant pain point. A few questions:

1. Would this allow for static linking, dynamic linking, and dynamic loading? Or only a subset of these? It also might be nice to allow users to choose what to do per platform (e.g., static link on iOS/macOS and dynamic link/load on other platforms).
2. Do you envision this being available as a part of Dart 3.0? My guess is no (at least for the initial release), but wanted to ask regardless.
3. It will not always be possible to build native assets on the client-side unfortunately (e.g. Rust, Go, etc.). Will we be able to upload native assets directly to pub.dev, or at least reference the native assets from there (through some artifact repo or GitHub releases for example)?
4. How will web/wasm be handled? It would be good to have some integration to ease development/release, but not too too much, so things like wasmer-js can be supported to use WASM with WASI on web.

[dcharkes]

Hi @GregoryConrad, nice to meet you!

Hi, as a current user of option number 4 above, this sounds really enticing. Right now I am using CMake for Linux/Windows/Android and Ruby (with inline bash) for macOS/iOS to download & extract binaries and it is a significant pain point.

Thanks for bearing with us in the mean time! 🙃

A few questions:

1. Would this allow for static linking, dynamic linking, and dynamic loading? Or only a subset of these? It also might be nice to allow users to choose what to do per platform (e.g., static link on iOS/macOS and dynamic link/load on other platforms).

We will start with dynamic loading: @FfiNative doing dlopen under the hood with the dynamic libraries. In JIT mode (dart ..., flutter test) we will always use dynamic loading.

In AOT (dart compile ..., flutter build) we can think about invoking the native linker and static linking the Dart AOT snapshot with the static libraries, this is tracked in:

• [vm/ffi] Explore using FfiNative for static linking #49418

I have not explored using the native linker to do dynamic loading. I'm not sure if that will be of any benefit in any Dart or Flutter usage scenario. @GregoryConrad do you have a scenario in mind where you would prefer using dynamic loading?

2. Do you envision this being available as a part of Dart 3.0? My guess is no (at least for the initial release), but wanted to ask regardless.

I cannot comment on that. We'd want to ship the feature when it's ready, not earlier.

3. It will not always be possible to build native assets on the client-side unfortunately (e.g. Rust, Go, etc.). Will we be able to upload native assets directly to pub.dev, or at least reference the native assets from there (through some artifact repo or GitHub releases for example)?

We're in discussions about that, but likely yes. We have two options, and might want to support both:

1. The package author precompiles for all targets and uploads the binaries to pub, the package users then download the binaries needed for the architectures they're targetting. (Requirements here is that "A standardized way to build or download native assets. This will be done by a CLI." must have the assets in a standardized location on disk or that the CLI takes an output folder so that pub and the CLI can cooperate on getting binaries in the right place.) Uploading to pub is tracked in:

• Native assets on pub pub#3693

2. A package author can have a build_dependency on package:rust_compiler which is a dart package that downloads a rust compiler. (Requirements here are that we should probably have some kind of "This will install X GB of tools on your computer. Continue: Y/N" in our CLI.)

4. How will web/wasm be handled? It would be good to have some integration to ease development/release, but not too too much, so things like wasmer-js can be supported to use WASM with WASI on web.

Best case scenario we could have a unified API for interacting with the C library on native platforms and a to WASM compiled library on the web:

• [ffi/js-interop/wasm] Unified API for using dart:ffi and JS Interop with WASM #46690

Further discussion can be on that issue.

[GregoryConrad]

I have not explored using the native linker to do dynamic loading. I'm not sure if that will be of any benefit in any Dart or Flutter usage scenario. @GregoryConrad do you have a scenario in mind where you would prefer using dynamic loading?

I don't actually have a case for using dynamic linking over dynamic loading, or vice versa. When I said:

It also might be nice to allow users to choose what to do per platform

I was more concerned with making sure we would be able to choose between static linking and dynamic linking/loading per platform. If someone else has a use case, however, I would be happy to hear it.

2. A package author can have a build_dependency on package:rust_compiler which is a dart package that downloads a rust compiler. (Requirements here are that we should probably have some kind of "This will install X GB of tools on your computer. Continue: Y/N" in our CLI.)

Although I like the idea here a lot, I foresee some issues with this approach. Especially in my project, some custom build steps and quite a few other package dependencies (besides just rustup) are needed to handle cross compilation, especially to android (due to linking with the NDK). I currently have these all done in CI and I think I would just prefer to ship the binaries in a native assets bundle directly. Perhaps having a rust_compiler dependency would work great for many projects, but for some projects with more complex dependencies/needs, this may not be as feasible. Also, if Flutter supports cross-compilation between desktop platforms in the future, this could also be problematic.

And thanks for the rest of your input! Much appreciated.

[eladmaimoni]

Just for the record, as mentioned in my flutter issue, I think the workflow for writing C/C++ plugins should be made simpler.

• User should be able to customize the cmake arguments passed by dart / flutter tools.
• Ideally, when generating the template for a plugin, there should be a single root CMakeLists.txt and not several.
• Also, there should be a place to put source files that are common to all platforms. If i am not mistaken, the plugin template now generates separate files to all platforms. It is very confusing not to have a single 'entry' point for the plugin.

One way this could be achieved is to use CMakePresets.json to configure cmake for each platform.

[stuartmorgan]

@eladmaimoni Plugins are a Flutter-specific concept, as is the template you are referring to.

[dcharkes]

• User should be able to customize the cmake arguments passed by dart / flutter tools.

The build for native assets would be disconnected from the Flutter native build. Flutter would only receive a list of static and dynamic libraries that need to be bundle.

• Ideally, when generating the template for a plugin, there should be a single root CMakeLists.txt and not several.

Even using CMake would be an abstraction in this approach. We haven't settled yet on whether we mandate compilers, but with trying to support Go and Rust as well, constraining C/C++ to a single build system would also be constraining. How various compilers would be supported (for example with Dart packages that wrap them) is still up for discussion.

The way that this is achieved is that we interact through a CLI (likely bin/native.dart or bin/_build.dart) that can run any compiler it wants. It has to output a list of compiled native static and dynamic libraries for Flutter and Dart to bundle in apps. (You don't have have to build from source. You can download precompiled binaries from a CDN or pub.dev as well.)

• Also, there should be a place to put source files that are common to all platforms. If i am not mistaken, the plugin template now generates separate files to all platforms. It is very confusing not to have a single 'entry' point for the plugin.

This is trivial once the build is an arbitrary Dart script. One of my examples is a completely cross platform CMake build.

One way this could be achieved is to use CMakePresets.json to configure cmake for each platform.

Native assets is a completely different approach. Zero Flutter template "boilerplate" will be included. These will be Dart standalone packages that will have their compiled dynamic libraries properly build and bundled with Flutter apps.

[stuartmorgan]

How various compilers would be supported (for example with Dart packages that wrap them) is still up for discussion.

The way that this is achieved is that we interact through a CLI (likely bin/native.dart or bin/_build.dart) that can run any compiler it wants.

I know I've mentioned this in other contexts, but we'll need to be very careful about getting this right; the ecosystem effects if we don't have a robust way to manage the toolchain dependencies automatically could be very unpleasant.

[GregoryConrad]

The way that this is achieved is that we interact through a CLI (likely bin/native.dart or bin/_build.dart) that can run any compiler it wants. It has to output a list of compiled native static and dynamic libraries for Flutter and Dart to bundle in apps. (You don't have have to build from source. You can download precompiled binaries from a CDN or pub.dev as well.)

If you are familiar with Rust, is what you are describing here similar to build.rs in some capacity (in that package authors can create their own build scripts)? That sounds very exciting as:

1. On package development machines, we could throw in some compilation steps to automate compilation and remove the need to manually trigger compilations and sling binaries around everywhere. This is super handy when doing something like dart test so that the build script would handle that compilation step for us to ensure everything is up to date.
2. On users' machines, we simply download or point to precompiled native assets.
3. We can switch between options 1 & 2 based on whether the necessary dev dependencies are installed locally (e.g., running a which cargo fails or not). We will probably want Dart APIs to figure out the build platform (already supported thru Abi.current()) and the target platform, while also supporting cross compilation.

I almost wonder if there should be a standard way to package native assets at all then (other than being able to upload to pub.dev, which would be great); if we have build scripts then we can decide which way works best for our projects based on our own build systems (e.g., target/ could be used for binary locations when using Rust).

However, one obvious drawback is that the build script could access anything on the system. I am not too worried about this issue though, as any Flutter package today can do the same without any warning to the user and other ecosystems such as Rust haven't had many troubles with this--you just simply need to trust the packages you use.