How to use multithreading correctly？

[1192724521]

//#[tokio::main(worker_threads = 16)]
//#[tokio::main(worker_threads = 8)]
#[tokio::main(worker_threads = 1)]
async fn main() {
    let app = Router::new().route("/", get(root));
    let addr = SocketAddr::from(([127, 0, 0, 1], 3000));

    axum::Server::bind(&addr)
        .serve(app.into_make_service())
        .await
        .unwrap();
}

async fn root() -> &'static str {
    tokio::spawn(async move {
        tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;
        "success"
    })
    .await
    .unwrap()
}

Complete requests: 100000
Concurrency Level: 10000

work threads	Time taken for tests	Requests per second
1	28	3566
8	28	3545
16	28	3523

Why does the number of worker threads have no effect on concurrency?
I just can't figure it out. I need a little help.please.Thank you.

I want this result

work threads	Time taken for tests	Requests per second
1	28	3566
2	14	7000
4	10	10000
8	10	10000

[Darksonn]

It appears that computational resources are not the bottleneck. To increase the performance, you must determine what the bottleneck is, and fix that issue instead of increasing the number of threads.

For example, maybe the program you are using to connect to the server has some sort of connection limit?

[1192724521]

It's a local connection, and I don't think I set a connection limit. A single thread with 10,000 concurrent sessions can only process 3,500 per second. Then I tried to increase the number of threads, but it didn't seem to work.

[Darksonn]

Connection limit was just a guess. I don't know what is limiting your number of connections/

To be clear, you are using multi-threading correctly - it's just that the number of threads was not the reason behind the limit.

[1192724521]

I'm replacing tokio::time::sleep with std::thread::sleep, and multithreading works.

Working only on a single thread or certain limitations makes my head explode. Ha ha ha ......

[Geo-W]

I use std::thread::sleep in a handler and seems like it blocks all of the incoming request..?

btw how did u measure the requests per second

[Darksonn]

Regarding std::thread::sleep, please read https://ryhl.io/blog/async-what-is-blocking/

[Geo-W]

@Darksonn
Many thanks!!
I got that when running blocking task i should call spawn_blocking.
But there is still something confuses me.

When writing async Rust, the phrase “blocking the thread” means “preventing the runtime from swapping the current task”. This can be a major issue because it means that other tasks on the same runtime will stop running until the thread is no longer being blocked.

As tokio running pool got many threads based on initial setting #[tokio::main(flavor = "multi_thread", worker_threads = 16)] , so blocking any of the thread will cause all other tasks stop running?
As per my testing, it is indeed like so: axum with two handler, one is std::thread::sleep for 10s, and the other one is non-blocking. Calling on blocking handler prevents all of the incoming request for non-blocking handler.

But just found it strange, if only switch task/ switch thread when we reach an await, what is the use of switching thread. When met await it means that current thread is available, so I suppose it shouldn't bother to switch threads already?

Thanks~

[Darksonn]

Await does not make the task move to a different thread. Rather, await makes the thread stop executing the current task, and start executing a different task, if any are available to run.