Help: passing lock objects to spawned tasks leads to deadlock

[zhengtong0898]

This code uses a multi_thread runtime to run.

It is expected that using loop{} in spawn shouldn't cause deadlock due to multi-threading.

However, in the main function's loop, the code never reaches println!, leading to deadlock.

Code-A

use reqwest::Client;
use std::sync::Arc;
use std::thread::current;
use std::time::Duration;
use tokio::sync::RwLock;

struct Site {
    url: String,
}

impl Site {
    async fn print_main_page(&self, client: Client) {
        let _ = client.get(&self.url).send().await.unwrap();
        println!("spawn_print: thread_id: {:?}", current().id());
    }
}

#[tokio::main(flavor = "multi_thread", worker_threads = 10)]
async fn main() {
    let client = Client::builder()
        .proxy(reqwest::Proxy::all("http://127.0.0.1:7890").unwrap())
        .build()
        .unwrap();

    let url = String::from("https://www.google.com");
    let site = Arc::new(RwLock::new(Site { url }));

    let site_clone = site.clone();
    let client_clone = client.clone();
    tokio::spawn(async move {
        let guard = site_clone.read().await;
        guard.print_main_page(client_clone).await;

        // long running
        println!("spawn_loop : thread_id: {:?}", current().id());
        loop {}
    });

    let mut count = 1;
    loop {
        tokio::time::sleep(Duration::from_secs(1)).await;
        println!(
            "main_loop  : thread_id: {:?}: count: {count}",
            current().id()
        );
        count += 1;
    }
}

Code-A output:

spawn_print: thread_id: ThreadId(9)
spawn_loop : thread_id: ThreadId(9)

After some trials, I also found that if I launch another task (tokio::spawn) to run print_main_page, then there will be no blocking at all.

Code-B

use reqwest::Client;
use std::sync::Arc;
use std::thread::current;
use std::time::Duration;
use tokio::sync::RwLock;

struct Site {
    url: String,
}

impl Site {
    async fn print_main_page(&self, client: Client) {
        let _ = client.get(&self.url).send().await.unwrap();
        println!("spawn_print: thread_id: {:?}", current().id());
    }
}

#[tokio::main(flavor = "multi_thread", worker_threads = 10)]
async fn main() {
    let client = Client::builder()
        .proxy(reqwest::Proxy::all("http://127.0.0.1:7890").unwrap())
        .build()
        .unwrap();

    let url = String::from("https://www.google.com");
    let site = Arc::new(RwLock::new(Site { url }));

    let site = site.clone();
    let client_clone = client.clone();
    tokio::spawn(async move {
        tokio::spawn(async move {                                              // launch another task
            let guard = site.read().await;
            guard.print_main_page(client_clone).await;
        })
        .await
        .unwrap();

        println!("spawn_loop : thread_id: {:?}", std::thread::current().id());
        // long running
        loop {}
    });

    let mut count = 1;
    loop {
        tokio::time::sleep(Duration::from_secs(1)).await;
        println!(
            "main_loop  : thread_id: {:?}: count: {count}",
            std::thread::current().id(),
        );
        count += 1;
    }
}

Code-B output:

spawn_print: thread_id: ThreadId(9)
spawn_loop : thread_id: ThreadId(9)
main_loop  : thread_id: ThreadId(1): count: 1
main_loop  : thread_id: ThreadId(1): count: 2
main_loop  : thread_id: ThreadId(1): count: 3

I'm not sure why it behaves this way. Is it due to my misunderstanding or incorrect usage?
Thanks for any help!

[wathenjiang]

The loop{} in your code is a blocking code. Any blocking codes may block the entire tokio runtime.

Do not using a never return( or never yield) blocking code in tokio runtime!

If you are looking for a solution for this kind of use, you can refer to https://ryhl.io/blog/async-what-is-blocking/.

[zhengtong0898]

I've read the links in your answer.
The article discusses that using std::thread::sleep in the sleep_then_print function leads to blocking because it doesn't provided await. The join method generally waits for tasks to finish in sequence. Task switching occurs only after await is provided.

According to the official documentation on the spawn function:

The provided future will start running in the background immediately when `spawn` is called, 
even if you don't `await` the returned `JoinHandle`.

Based on this, I slight modifications to the code example in the article, the tasks are still running concurrently due to the multi-thread model. (note: single-thread model will blocking)

use std::time::Duration;

async fn sleep_then_print(timer: i32) {
    println!("Start timer {}.", timer);

    // No .await here!
    std::thread::sleep(Duration::from_secs(1));

    println!("Timer {} done.", timer);
}

#[tokio::main]
async fn main() {
    // The join! macro lets you run multiple things concurrently.
    tokio::spawn(async { sleep_then_print(1).await });
    tokio::spawn(async { sleep_then_print(2).await });
    tokio::spawn(async { sleep_then_print(3).await });

    loop {
        tokio::time::sleep(Duration::from_secs(1)).await;
    }
}

output:

Start timer 1.
Start timer 2.
Start timer 3.
Timer 2 done.
Timer 1 done.
Timer 3 done.

Thank you for your reply and the information provided.
While it has been somewhat helpful, it seems that it has not fully addressed my concerns.
I will continue to await a resolution to the issue and look forward to further discussion with you. Thank you!

[wathenjiang]

You do not have to know the detail of task switching, but you would better know use tokio::task::spawn_blocking function to run your loop{}. This is one of the key point of https://ryhl.io/blog/async-what-is-blocking/.

Your problem is not dealock issue.

[Darksonn]

You need to use std::future::pending().await instead of loop {}.

[zhengtong0898]

Thanks for the suggestions. I now understand that using tokio::task::spawn_blocking and std::future::pending().await can prevent total blocking.

And I made the code simpler to this version and tried to guess its behavior:

#[tokio::main(flavor = "multi_thread", worker_threads = 10)]
async fn main() {
    tokio::spawn(async move {
        tokio::time::sleep(std::time::Duration::from_secs(1)).await;

        loop {
            println!("----a----: {:?}", std::thread::current().id());
            std::thread::sleep(std::time::Duration::from_secs(1));
        }
    });

    loop {
        println!("----b----: {:?}", std::thread::current().id());
        tokio::time::sleep(std::time::Duration::from_secs(1)).await;
    }
}

output:

----b----: ThreadId(1)
----a----: ThreadId(7)
----b----: ThreadId(1)
----a----: ThreadId(7)
----a----: ThreadId(7)
----a----: ThreadId(7)
----a----: ThreadId(7)
----a----: ThreadId(7)

When the async move {} code block gets to .await, it gives control to the runtime;
after tokio::time::sleep is done, the runtime wakes up theasync move {} Future and gives it control to run the next part.
But once it gets control, it just keeps looping and blocking, making other threads wait to be scheduled."