XCTest requires access to host file system

[kkebo]

Description

I'm not sure which behavior is the expected behavior, but it seems that the test executable requires external file access to run if it is built with the latest (DEVELOPMENT-SNAPSHOT-2024-03-22-a-wasm) toolchain.

main-snapshot-2024-03-13 & DEVELOPMENT-SNAPSHOT-2024-03-20-a-wasm

$ mkdir wasm-test-example
$ cd wasm-test-example
$ swift package init --type library -name WasmTestExample
$ swift build --build-tests --experimental-swift-sdk DEVELOPMENT-SNAPSHOT-2024-03-20-a-wasm
$ wasmtime .build/debug/WasmTestExamplePackageTests.wasm
Test Suite 'All tests' started at 
Test Suite 'testBundle.xctest' started at 
Test Suite 'WasmTestExampleTests' started at 
Test Case 'WasmTestExampleTests.testExample' started at 
Test Case 'WasmTestExampleTests.testExample' passed (0.0 seconds)
Test Suite 'WasmTestExampleTests' passed at 
         Executed 1 test, with 0 failures (0 unexpected) in 0.0 (0.0) seconds
Test Suite 'testBundle.xctest' passed at 
         Executed 1 test, with 0 failures (0 unexpected) in 0.0 (0.0) seconds
Test Suite 'All tests' passed at 
         Executed 1 test, with 0 failures (0 unexpected) in 0.0 (0.0) seconds

main-snapshot-2024-03-20 & DEVELOPMENT-SNAPSHOT-2024-03-22-a-wasm

$ mkdir wasm-test-example
$ cd wasm-test-example
$ swift package init --type library -name WasmTestExample
$ swift build --build-tests --experimental-swift-sdk DEVELOPMENT-SNAPSHOT-2024-03-22-a-wasm
$ wasmtime .build/debug/WasmTestExamplePackageTests.wasm
Error: failed to run main module `.build/debug/WasmTestExamplePackageTests.wasm`

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0x60161a - <unknown>!$s10Foundation6BundleC05_mainB033_6273FB07A96EB35BCC520B3A80F5C3CDLL_WZ
           1: 0x42d00b - <unknown>!swift_once
           2: 0x60167d - <unknown>!$s10Foundation6BundleC4mainACvgZ
           3: 0xc7057b - <unknown>!$s6XCTest11XCTMainMisc_9arguments9observersAA19TestSuiteOrExitCodeOSayAA0A4CaseCm04testK5Class_SaySS_yAHKctG8allTeststG_SaySSGSayAA0A11Observation_pGSgtF
           4: 0xc70378 - <unknown>!$s6XCTest7XCTMain_9arguments9observerss5Int32VSayAA0A4CaseCm04testF5Class_SaySS_yAHKctG8allTeststG_SaySSGSayAA0A11Observation_pGSgtYaFTY0_
           5: 0x4c4830 - <unknown>!swift_task_switchImpl(swift::AsyncContext*, void (swift::AsyncContext* swift_async_context) swiftcall*, swift::SerialExecutorRef)
           6: 0x4c46d9 - <unknown>!swift_task_switch
           7: 0xc712e7 - <unknown>!$s6XCTest7XCTMainys5NeverOSayAA0A4CaseCm04testD5Class_SaySS_yAFKctG8allTeststGYaFTY0_
           8: 0x4c4830 - <unknown>!swift_task_switchImpl(swift::AsyncContext*, void (swift::AsyncContext* swift_async_context) swiftcall*, swift::SerialExecutorRef)
           9: 0x4c46d9 - <unknown>!swift_task_switch
          10: 0xc71267 - <unknown>!$s6XCTest7XCTMainys5NeverOSayAA0A4CaseCm04testD5Class_SaySS_yAFKctG8allTeststGYaF
          11: 0x659f9 - <unknown>!$s27WasmTestExamplePackageTests6RunnerV4mainyyYaFZTY0_
          12: 0x4c4830 - <unknown>!swift_task_switchImpl(swift::AsyncContext*, void (swift::AsyncContext* swift_async_context) swiftcall*, swift::SerialExecutorRef)
          13: 0x4c46d9 - <unknown>!swift_task_switch
          14: 0x65963 - <unknown>!$s27WasmTestExamplePackageTests6RunnerV4mainyyYaFZ
          15: 0x65b0f - <unknown>!$s27WasmTestExamplePackageTests6RunnerV5$mainyyYaFZ
          16: 0x65cc4 - <unknown>!async_Main
          17: 0x65e6e - <unknown>!$sIetH_yts5Error_pIeghHrzo_TR
          18: 0x66194 - <unknown>!$sIetH_yts5Error_pIeghHrzo_TRTA
          19: 0x4c6370 - <unknown>!future_adapter(swift::AsyncContext*)
          20: 0x4c3a44 - <unknown>!swift::runJobInEstablishedExecutorContext(swift::Job*)
          21: 0x4c4473 - <unknown>!swift_job_run
          22: 0x65def - <unknown>!main
          23: 0xc8f258 - <unknown>!__main_void
          24: 0x27a3f - <unknown>!_start
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    2: wasm trap: wasm `unreachable` instruction executed
$ wasmtime --dir / .build/debug/WasmTestExamplePackageTests.wasm
Test Suite 'All tests' started at 2024-03-25 12:38:19.073
Test Suite '/.xctest' started at 2024-03-25 12:38:19.074
Test Suite 'WasmTestExampleTests' started at 2024-03-25 12:38:19.074
Test Case 'WasmTestExampleTests.testExample' started at 2024-03-25 12:38:19.074
Test Case 'WasmTestExampleTests.testExample' passed (0.0 seconds)
Test Suite 'WasmTestExampleTests' passed at 2024-03-25 12:38:19.074
         Executed 1 test, with 0 failures (0 unexpected) in 0.0 (0.0) seconds
Test Suite '/.xctest' passed at 2024-03-25 12:38:19.074
         Executed 1 test, with 0 failures (0 unexpected) in 0.0 (0.0) seconds
Test Suite 'All tests' passed at 2024-03-25 12:38:19.074
         Executed 1 test, with 0 failures (0 unexpected) in 0.0 (0.0) seconds

Environment

• Linux (x86_64, aarch64)
• Swift main snapshot (swift-DEVELOPMENT-SNAPSHOT-2024-03-20-a)
• Swift SDK (swift-wasm-DEVELOPMENT-SNAPSHOT-2024-03-22-a)

[kateinoigakukun]

It might be worth considering making access to Bundle.main optional

[kkebo]

This is not a problem when swift-testing is used instead of XCTest.