[native_assets_cli] Move `BuildConfig.targetOS` to `BuildConfig.codeConfig.targetOS`, remove it entirely or extend to cover web

[mkustermann]

Currently the CLI protocol mandates BuildConfig.targetOS. Though making builds for the web platform wouldn't have any operating system. So we could either move this to BuildConfig.codeConfig.targetOS or remove it entirely.

We may decide that we introduce a BuildConfig.targetPlatform which is the platform we're going to run Dart code on - but this is different from an operating system concept ("web" is a target platform, but not an operating system).

/cc @dcharkes

[dcharkes]

remove it entirely

This is not an option right? hook/build.dart needs to know what OS a dylib must be built for.

Move [...] or extend to cover web

BuildConfig.codeConfig.targetOS missing while BuildConfig.codeConfig.targetArchitecture is there is slightly annoying, so moving could make sense.

BuildConfig.targetPlatform

That would be native and web?

If users have data assets that are OS dependent, they might start misusing config.codeConfig.targetOS to emit different icons for Android/iOS. How confident are you that we don't need an OS for data assets?

[mkustermann]

Again, the concept of "operating system" does not make sense in general, because for web builds there's no operating system.

The more general concept of "target platform" is a superset, it gives the same information as OS but also allows expressing we compile for the web. So that would allow removing targetOS (as it can be deducted from the more general targetPlatform).

We could structure it slightly differently and have

BuildConfig
  .targetPlatform (if it's native, .nativeConfig will be set, if it's web .webConfig will be set)
  .nativeConfig
     .targetOS
     .codeConfig
         .cCompiler
           .archiver/.linker/....
  .webConfig
     .compiler (JS or Wasm)
     .crossOriginIsolated (or whathever web specific options)

In any case I believe for web builds it doesn't make sense to have OS - except if made some artificial OS.web but that seems very weird because the actual OS the web build runs on a particular operating system, we just don't know at compile-time.

I think we should reduce the top-level scope on BuildConfig to an absolute minimum.

[dcharkes]

.nativeConfig .webConfig

👍
I like this more than .codeConfig and .dataConfig. Because it feels somewhat weird to duplicate info such as targetOS in both .codeConfig and .dataConfig or say .dataConfig does not get access to to OS because it might not exist on the web.

[mkustermann]

I like this more than .codeConfig and .dataConfig. Because it feels somewhat weird to duplicate info such as targetOS in both .codeConfig and .dataConfig or say .dataConfig does not get access to to OS because it might not exist on the web.

I would never introduce .dataConfig. But .codeConfig may still make sense because one can imagine situations where a dart build targets native (e.g. linux) but doesn't support code assets. In which case the bundling tool doesn't have to provide c compiler configuration, target abi levels, etc (basically codeConfig is there or not there and it groups multiple things that are needed when code assets are enabled - otherwise we'd have several fields on .nativeConfig that are either all set or none set depending on whether code assets are enabled)

[dcharkes]

BuildConfig
  .targetPlatform (if it's native, .nativeConfig will be set, if it's web .webConfig will be set)
  .nativeConfig
     .targetOS
     .codeConfig
         .cCompiler
           .archiver/.linker/....
  .webConfig
     .compiler (JS or Wasm)
     .crossOriginIsolated (or whathever web specific options)

I think we should reduce the top-level scope on BuildConfig to an absolute minimum.

This should probably be addressed together with restructuring the JSON:

• [native_assets_cli] JSON protocol nest per asset type #993

[dcharkes]

.codeConfig may still make sense because one can imagine situations where a dart build targets native (e.g. linux) but doesn't support code assets. In which case the bundling tool doesn't have to provide c compiler configuration, target abi levels, etc (basically codeConfig is there or not there and it groups multiple things that are needed when code assets are enabled - otherwise we'd have several fields on .nativeConfig that are either all set or none set depending on whether code assets are enabled)

Thinking a bit further along these lines. Once we add support for WasmAssets, we'll have a wasmConfig (detailing whether we need emscripten ABI, WASI is supported etc.) It might be conceivable that there could be native embedders that would include a wasm runtime, in which case it would make more sense to have config.wasmConfig rather than config.webConfig.wasmConfig.

We might as well adopt the convention that we have toplevel configs for

1. platforms - optional based on if that is the targetPlatform,
2. asset types - optional for if those asset types are in buildAssetTypes, and
3. embedders - optional for if an embedder wants to provide it.

BuildConfig
  // 0. shared
  .outputDirectory
  .outputDirectoryShared
  .packageName
  .packageRoot
  // 1. per target platform
  .targetPlatform // if it's native, .nativeConfig will be set, if it's web .webConfig will be set
  .nativeConfig
    .targetOS
  .webConfig
    .compiler // JS or Wasm
    .crossOriginIsolated // or whatever web specific options
  // 2. per asset
  .buildAssetTypes // every asset type has it's own config, optionally
  .codeConfig
    .targetArchitecture
    .buildMode // This is `config.buildMode` currently, but shouldn't this be for code assets only?
    .linkModePreference
    .targetIOSVersion
    .targetMacOSVersion
    .targetAndroidNdkApi
    .targetIOSSdk
    .cCompiler
      .archiver
      .compiler
      .linker
      ...
  .dataConfig // we haven't found a need yet, so doesn't exist currently
  .wasmConfig
    .abi // emscripten
    .wasiVersion // 'Preview 1' currently
  .javaConfig // or named differently if we'd include Kotlin things?
    .gradlePath // would something like this be useful?
  .iconConfig // in Flutter only, for IconAsset
    .resolution // xxx dpi
  .fontConfig // in Flutter only, for FontAsset
    ... //  Maybe no need for this either
  // 3. per embedder
  .flutterConfig
    .buildMode // release, debug, profile (different from codeConfig.buildMode)

Current target platforms: native and web. dart list flutter list.

(Side note: Using nesting to detail whether things are available is nice. However it doesn't always work. config.codeConfig.targetIOSVersion is only available if config.nativeConfig.os == OS.iOS. It would be tempting to have config.nativeConfig.iOSConfig.targetVersion, but then it would be only have to be if config.buildAssetTypes.contains('code'). So it has to be nullable in either location unfortunately.)

@HosseinYousefi For JarAssets, I'd expect Flutter could provide a config.javaConfig or config.jarConfig and provide a path to Gradle? etc. Any thoughts/ideas on what would be in there?

We could consider whether asset types should be nested under nativeConfig and webConfig. CodeAssets will likely only ever exist on native. WasmAsset, IconAsset, and FontAsset can be supported both on web and native. However, if we nest under platforms the uses of such configs will likely be written as config.nativeConfig.iconConfig ?? config.webConfig.iconConfig by helper libraries. So, I'm leaning towards toplevel for asset types.

@mkustermann Do you have any need for config.buildMode in the data assets or in the fonts/icons exploration in Flutter? If not, we should probably move it inside config.codeConfig.

(Edit: Maybe we should even leave config.targetPlatform out, then new embedders can even add target platforms without having to do a PR. config.isNative => config.nativeConfig != null.)

[mkustermann]

@mkustermann Do you have any need for config.buildMode in the data assets or in the fonts/icons exploration in Flutter? If not, we should probably move it inside config.codeConfig.

I think we should remove it entirely.

(Edit: Maybe we should even leave config.targetPlatform out, then new embedders can even add target platforms without having to do a PR. config.isNative => config.nativeConfig != null.)

I prefer us having config.nativeConfig etc as non-nullables as we already will have config.buildNativeCode getters (based on supportedAssetTypes / buildAssetTypes). So users can write

if (config.buildCodeAssets) {
  print(config.codeAsset.abi); // <- can access as non-nullable, no need for `!` operator
}

  .nativeConfig
   .targetOS
 .webConfig
  .compiler // JS or Wasm
  .crossOriginIsolated // or whatever web specific options
 // 2. per asset
 .buildAssetTypes // every asset type has it's own config, optionally
 .codeConfig
   .targetArchitecture
   .buildMode // This is `config.buildMode` currently, but shouldn't this be for code assets only?
   .linkModePreference
   .targetIOSVersion
   .targetMacOSVersion
   .targetAndroidNdkApi
   .targetIOSSdk
   .cCompiler

I do find it slightly confusing to have config.nativeConfig.targetOS and config.codeConfig.targetArchitecture.
I would prefer if everything a e.g. CLibraryBuilder needs is in one object that can accessed on config and passed to CLibraryBuilder - the information should ideally not been split between several objects.

Yes, theoretically one could have an arm32 interpreter running in the browser and one could theoretically then say a flutter web build should therefore set a C compiler configuration and ask packages to build arm32 C code which it will run in it's arm32 interpreter in the browser. But this is super contrived. I'm not sure we should be considering that use case.

Similarly, yes, theoretically one could have a wasm interpreter in an AOT-compiled android app. Therefore android flutter build apk builds should set a wasm compiler and bundle wasm modules. Though again this is a very contrived example.

What we should remember is that an asset type is usually coupled with a runtime API on how to access those asset. For example a hook emitting code assets is 1 <-> 1 coupled with dart code using dart:ffi. So from that perspective one could even think about having this (assuming we make dart:ffi work on the web)

// config.ffiConfig configures things for hooks
// hooks can produce wasm/native code
// that works with runtime's  `dart:ffi`s `@Native` API
config.ffiConfig.nativeCode.{cCompiler,abi,targetOS,...}
config.ffiConfig.wasmCode.{wasmCompiler,...}

The place where this would not be ideal if there was a need for a cCompiler for non-dart:ffi stuff. That would be a stronger argument for a top-level entry. But it's unclear if that would ever be a usecase. And if so, we would have a new runtime API on how to access them and possibly a new config (which make have overlaps with config.ffiConfig or not).

I'll think a bit more about this. Maybe we can discuss this in the next meeting.