Job startup delay

[filex]

The jobs configure the interval. That is the duration between two invocations of the job. The first is "on startup" of Couper.

I had a situation, where I have a kube pod with two containers: Couper and a node-based API server. When the pod is rolled out, both containers start in parallel. Usually, Couper is faster and executes the jobs while the pod-local backend is not available yet. (This results in errors like cannot connect to http://localhost:3000).

The purpose of the execution on startup was, that the interval is not aligned (e.g. with 00:00 midnight). So instead of waiting for one interval before the first execution, we do it immediately. If the first execution is doomed to fail, we risk to maximize the time to the first job execution.

An easy solution could be to mitigate the problem is introduce a startup delay for the job:

job "init_cache" {
  interval = "1h"
  startup_delay = "1m"
  
  // requests  …
}

In this example, Couper would issue the first job 1 minute after startup. And thereafter, every hour.

[filex]

For jobs with longer intervals it doesn't really matter if the first job is issued 1s or 1m after startup. So we could even decide to define the startup_delay with a non-zero default value such as 1m.

I would consider jobs with very short intervals (like 10s or even less) the exception. In that case the user could explicitly disable the startup_delay if that really is an issue.

[filex]

There are a couple of alternative solutions that are all more complex:

• implementing a schedule attribute that accepts cron syntax. This would make the invocations independent from the startup
• a random startup delay within a configurable time span. this would have the additional benefit of unsynchronizing all jobs (currently all jobs with the same interval will always run at the same time. this can cause problems on its own).
• implementing retries when jobs have failed

[johakoch]

Is this related to #726?

[filex]

Kubernetes has a similar concept for readinessProbe and livenessProbe: with inititalDelaySeconds the start of the probe intervals can be delayed.

Hence, "initial_delay" could be an alternative name.