How do I set a job to State::Failed?

[azambuilds]

I'm using the redis backend on the current main branch

I've got a basic worker and the .build_fn contains a closure that returns an error:

return Err::<(), apalis::prelude::Error>(apalis::prelude::Error::Failed(Arc::new(
    "Failed to process job".to_string().into(),
)));

But when getting a list of jobs:

let failed_jobs = storage_clone.list_jobs(&State::Failed, 1).await;
tracing::info!("Dumping failed jobs: {:#?}", failed_jobs);

I get:

2025-02-15T10:45:57.017897Z ERROR orchestrator::worker: Worker my_worker errored: Failed("Failed to process job")


2025-02-15T10:33:00.999742Z  INFO orchestrator::worker: Dumping failed jobs: Ok(
    [],
)

Is there a way to keep track of these? Or can someone let me know what determines if a job goes to the failed list?

[geofmureithi]

By default, a failed job is returned into the queue. Can you try the same but set max retries to 1?

…

On Sat, 15 Feb 2025 at 13:42, Azam Rahman ***@***.***> wrote: I'm using the redis backend on the current main branch I've got a basic worker and the .build_fn contains a closure that returns an error: return Err::<(), apalis::prelude::Error>(apalis::prelude::Error::Failed(Arc::new( "Failed to process job".to_string().into(), ))); But when getting a list of jobs: let failed_jobs = storage_clone.list_jobs(&State::Failed, 1).await; tracing::info!("Dumping failed jobs: {:#?}", failed_jobs); I get: 2025-02-15T10:33:00.999742Z INFO orchestrator::worker: Dumping failed jobs: Ok( [], ) Is there a way to keep track of these? Or can someone let me know what determines if a job goes to the failed list? — Reply to this email directly, view it on GitHub <#520>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWXVRGWVRAH3KRCP24GTNMT2P4KYBAVCNFSM6AAAAABXGIJWVWVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZXHE3DSMJYGI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[azambuilds]

I tried setting max retries to 1, i tried removing max_retries, etc. but in both cases im getting stuff like the following, which suggests that jobs are being put in queue but not being tracked by the joblist / stats?

2025-02-15T15:14:01.007626Z  INFO orchestrator::worker: Dumping stats: Ok(
    Stat {
        pending: 0,
        running: 0,
        dead: 35,
        failed: 0,
        success: 170,
    },
)
2025-02-15T15:14:23.883596Z  INFO orchestrator::worker: Worker analyzer Uhandled event: Worker { id: WorkerId { name: "analyzer" }, state: Engage(TaskId(Ulid(2103086226378986476020500943023874512))) }
2025-02-15T15:14:23.883689Z  INFO orchestrator::worker: Processing job jobname, attempt: 5
2025-02-15T15:14:23.883739Z ERROR apalis::layers::tracing::on_failure: AbortError: RetryPolicyError: Out of retries after 5 attempts: FailedError: Failed to process job done_in=0 ms
2025-02-15T15:14:23.883830Z ERROR orchestrator::worker: Worker analyzer errored: Abort(OutOfRetries { current_attempt: 5, inner: Failed("Failed to process job") })
2025-02-15T15:14:23.883882Z  INFO orchestrator::worker: Worker analyzer Uhandled event: Worker { id: WorkerId { name: "analyzer" }, state: Engage(TaskId(Ulid(2103086250927071606017741623615257447))) }
2025-02-15T15:14:23.883919Z  INFO orchestrator::worker: Processing job jobname, attempt: 5
2025-02-15T15:14:23.883944Z ERROR apalis::layers::tracing::on_failure: AbortError: RetryPolicyError: Out of retries after 5 attempts: FailedError: Failed to process job done_in=0 ms
2025-02-15T15:14:23.883975Z ERROR orchestrator::worker: Worker analyzer errored: Abort(OutOfRetries { current_attempt: 5, inner: Failed("Failed to process job") })

2025-02-15T15:15:00.999915Z  INFO orchestrator::worker: Dumping pending jobs: Ok(
    [],
)
2025-02-15T15:15:01.000436Z  INFO orchestrator::worker: Dumping failed jobs: Ok(
    [],
)
2025-02-15T15:15:01.001365Z  INFO orchestrator::worker: Dumping running jobs: Ok(
    [],
)
2025-02-15T15:15:01.001834Z  INFO orchestrator::worker: Dumping scheduled jobs: Ok(
    [],
)
2025-02-15T15:15:01.002344Z  INFO orchestrator::worker: Dumping stats: Ok(
    Stat {
        pending: 0,
        running: 0,
        dead: 37,
        failed: 0,
        success: 170,
    },
)

[geofmureithi]

Okay I will write some tests and then we will see where the issue is

…

On Sat, 15 Feb 2025 at 18:19, Azam Rahman ***@***.***> wrote: I tries setting max retries to 1, i tried removing max_retries, etc. but in both cases im getting stuff like the following, which suggests that jobs are being put in queue but not being tracked by the joblist / stats? 2025-02-15T15:14:01.007626Z INFO orchestrator::worker: Dumping stats: Ok( Stat { pending: 0, running: 0, dead: 35, failed: 0, success: 170, }, ) 2025-02-15T15:14:23.883596Z INFO orchestrator::worker: Worker analyzer Uhandled event: Worker { id: WorkerId { name: "analyzer" }, state: Engage(TaskId(Ulid(2103086226378986476020500943023874512))) } 2025-02-15T15:14:23.883689Z INFO orchestrator::worker: Processing job jobname, attempt: 5 2025-02-15T15:14:23.883739Z ERROR apalis::layers::tracing::on_failure: AbortError: RetryPolicyError: Out of retries after 5 attempts: FailedError: Failed to process job done_in=0 ms 2025-02-15T15:14:23.883830Z ERROR orchestrator::worker: Worker analyzer errored: Abort(OutOfRetries { current_attempt: 5, inner: Failed("Failed to process job") }) 2025-02-15T15:14:23.883882Z INFO orchestrator::worker: Worker analyzer Uhandled event: Worker { id: WorkerId { name: "analyzer" }, state: Engage(TaskId(Ulid(2103086250927071606017741623615257447))) } 2025-02-15T15:14:23.883919Z INFO orchestrator::worker: Processing job jobname, attempt: 5 2025-02-15T15:14:23.883944Z ERROR apalis::layers::tracing::on_failure: AbortError: RetryPolicyError: Out of retries after 5 attempts: FailedError: Failed to process job done_in=0 ms 2025-02-15T15:14:23.883975Z ERROR orchestrator::worker: Worker analyzer errored: Abort(OutOfRetries { current_attempt: 5, inner: Failed("Failed to process job") }) — Reply to this email directly, view it on GitHub <#520 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWXVRGVB3M5TKXT3P4VODHT2P5LHTAVCNFSM6AAAAABXGIJWVWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMRRGAZDANA> . You are receiving this because you commented.Message ID: ***@***.***>