A new thread is created for each test even if --test-threads=1

[Taaitaaiger]

With nightly Rust, tests are run on separate threads even if --test-threads=1. It seems that a new thread is created for each test now, while they were previously run on the main thread if --test-threads=1. This breaks testing for jlrs, which depends on all tests running on the same thread. The underlying reason for this is that Julia can only be initialized once during a process's lifetime, and access is limited to the thread that initialized it.

My expectation is that --test-threads runs all tests on the same thread, or that I set use some other flag to achieve this.

Rust version: 1286ee2 2022-11-05

[Taaitaaiger]

This is probably due to #103681

[thomcc]

We don't really guarantee this -- we certainly used to behave the way further in the past (that PR was a revert).

That said, it wasn't really a change intended to break anyone, and it makes sense that it might (in the case of single-threaded C libraries and the like). A PR restoring the previous behavior of --test-threads=1 would be fine, although it's a bummer that will probably bring back a lot of that cruftiness that was removed.

[thomcc]

It might be cleaner to keep a thread around to run them, rather than coordinating use of the main thread for this case. I'd prefer that since previously it had to be duplicated, and it'd be a bummer to keep that forever.

IIUC from your description, it's most important that the tests all happen on some thread, and it's not important that it always be the main thread? (Hmmm... I suppose that's not enough for some cases though 😿)

[Taaitaaiger]

Yeah, it seems like one of those things that's only going to affect a small subset of users. And you understood correctly, rewriting the tests to only use one test per file is a valid approach for me.

The PR does seem like a good change to me, would it be impossible to optionally allow tests to run on the main thread without reintroducing the cruftiness?

[ianks]

This PR can have negative side effects for libs that rely on thread-local state being set before a test run (i.e. using ctor to setup something before the suite). I think it makes sense to keep the old behavior when --test-threads=1. I will take a look when I get some time!

[ianks]

@rustbot claim

[uazu]

I have the same issue. (The one that ehuss closed above.) The change to test on multiple threads one by one instead of just one thread has broken my tests in https://crates.io/crates/stakker.

Stakker can run multi-threaded, but as a single-threaded optimisation I allow a single thread to effectively become "special". This is useful when the app is structured as a main thread plus auxiliary threads. On that "special" main thread, Stakker can safely use global variables internally as an optimisation. However if Stakker is started on any other thread, it panics, because that would make the global variable use unsound, and the crate user needs to select the multi-threaded feature. However now that tests are spread across multiple threads, it immediately panics, and I cannot test the single-threaded functionality using cargo test.

For now I've hacked up a script to use the output of --list to run the tests one by one, but this is quite possibly flakey in case of changes in the output format of --list, so is not a good solution.

[RalfJung]

Funny, when it became a single thread that caused problems (#70492) and now the revert is also causing problems...

FWIW I would say each test getting fresh thread-local state is a feature. Having tests depend on each other is an anti-pattern. The test harness should clean up between tests as much as possible.