System module dependency leaks through the Subprocess API

[cmcgee1024]

The System module is available as SystemPackage, which is a SwiftPM package, and it supports the major Swift platforms including macOS. Some projects, such as swiftly, use this package instead of using conditional imports for System (macOS only, provided by the SDK) or SystemPackage (every other platform).

When using swift-subprocess in a swift file that makes use of FilePath, introducing an import on Subprocess creates a compile error on macOS because the existing FilePaths become ambiguous because the transitive import on System leaks through because of the references to FileDescriptor. As a result it means that clients must fully qualify FilePath, or any other usages of elements from SystemPackage to satisfy the compiler. Meanwhile, these are very nearly equivalent types.

[cmcgee1024]

This creates friction to allowing different parts of the code to use the run() functions directly because that would require fully qualifying anything that refers to FileDescriptor, or FilePath in the code that uses it. So, for now swiftly would need to provide its own central run function to shield from that.

[iCharlesHu]

@cmcgee1024 do you have any suggestions on how we should resolve this? Are you proposing we always unconditionally use the SystemPackage version?

[iCharlesHu]

Had an offline discussion with @glessard (who manages swift-system): the "correct" way to solve this would be to introduce Subprocess' own public types for FileDescriptor and FilePath that wrap around the system version. We should get this one in after the first tag but before 1.0.

[cmcgee1024]

Thanks @iCharlesHu,

I was originally thinking that since Subprocess exists in the SwiftPM space, and swift-system is offered as a package that it should use SwiftPackage for its API in all OSes and other packages can follow suit without messy conditional imports. It also helps to encourage the use of the standardized FilePath/FileDescriptor instead of the trend of everyone making their own similarly named types that are not compatible.

I'm a bit concerned with swift-subprocess proliferating yet more types for file path and file descriptors here when the goal should be for standardization.

What's the reasoning behind swift-subprocess continuing to conditionally import System? Why not unconditionally import SystemPackage?

[glessard]

If you're running on Darwin, you should import System and use the type from the SDK. The ability to use SystemPackage exists as a way to support development and testing first, and secondarily for access to new functionality that isn't yet in the SDK.

[allevato]

If you're running on Darwin, you should import System and use the type from the SDK. The ability to use SystemPackage exists as a way to support development and testing first, and secondarily for access to new functionality that isn't yet in the SDK.

Out of curiosity, is there a reason to not just unconditionally use SystemPackage even on Darwin? It seems strictly better except for code size and compile time metrics, because as a client I don't have to be concerned about which version of macOS a particular API became available in. Within Google we've been using SystemPackage exclusively for a lot of our tooling and it's nice to have one less axis of compatibility to add to our calculus, since all the machines a tool is executing on might not have the same major version of macOS.

I imagine this distinction could become an issue if a core library like Foundation began exporting APIs that used types from System, but that hasn't happened yet...

[cmcgee1024]

I imagine this distinction could become an issue if a core library like Foundation began exporting APIs that used types from System, but that hasn't happened yet...

I think that FilePath is a good example. We're in desperate need of a single FilePath type to stem the proliferation of them. It would be terrible if FilePath is not FilePath depending on whether it came from SDK or the package. If Foundation re-exports it then it would always be the SDK version, and not the package version.

[jakepetroules]

If Swift System becomes part of the toolchain, it would also solve this problem, as then the expectation would be "you are not allowed to depend on the package", similar to the situation for Foundation.

[cmcgee1024]

If Swift System becomes part of the toolchain, it would also solve this problem, as then the expectation would be "you are not allowed to depend on the package", similar to the situation for Foundation.

For something like FileDescriptor this could be a good way forward since it's unlikely to change in terms of API shape for a long time, possibly just bug fixes.

FilePath can benefit from API enhancements, and adopters of those enhancements. This is where packages with their fine-grained dependencies, and potential for rapid releases can be a big benefit. Relying on the SDK would mean waiting for a new toolchain release, and you get all of the changes at once. Also, for any major changes to the API they would need to wait until the SDK makes a major release, which can be a very long time.

I'm interested to see how to navigate this SDK/Package duality without pushing the complexity onto clients, or their clients by forcing certain conditional import patterns.

Also, I don't see anything obvious in swift-foundation or swift-system repos that says that I'm not allowed to depend on the package, so it's easy to just depend on it and use it.