No overlap between jobs

[wuodar]

Currently, one can set no-overlap to true, to avoid two concurrent executions. But is it possible to avoid overlapping between services? E.g.

version: '3.7'

services:
  feature_pipeline:
    image: feature_pipeline
    container_name: feature_pipeline
    restart: always
    user: user
    build:
      context: .
      dockerfile: Dockerfile
    volumes:
      - ${DATA_VOLUME}:/home/user/app/data
    deploy:
        resources:
            limits:
              cpus: ${MAX_CPUS}
              memory: ${MAX_MEMORY}
            reservations:
              cpus: ${MIN_CPUS}
              memory: ${MIN_MEMORY}
    labels:
      ofelia.enabled: "true"
      ofelia.job-exec.feature_pipeline.schedule: ${PIPELINE_SCHEDULE}
      ofelia.job-exec.feature_pipeline.command: "python src/feature_pipeline.py"
      ofelia.job-exec.feature_pipeline.no-overlap: "true"

  inference_pipeline:
    image: inference_pipeline
    container_name: inference_pipeline
    restart: always
    user: user
    build:
      context: .
      dockerfile: Dockerfile
    volumes:
      - ${DATA_VOLUME}:/home/user/app/data
    deploy:
        resources:
            limits:
              cpus: ${MAX_CPUS}
              memory: ${MAX_MEMORY}
            reservations:
              cpus: ${MIN_CPUS}
              memory: ${MIN_MEMORY}
    labels:
      ofelia.enabled: "true"
      ofelia.job-exec.inference_pipeline.schedule: ${PIPELINE_SCHEDULE}
      ofelia.job-exec.inference_pipeline.command: "python src/inference_pipeline.py"
      ofelia.job-exec.inference_pipeline.no-overlap: "true"
      ofelia.job-exec.feature_pipeline.no-overlap: "true"

  ofelia:
    image: mcuadros/ofelia:latest
    restart: always
    depends_on:
      - feature_pipeline
    command: daemon --docker
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro

In this example, I'd like to set ofelia.job-exec.feature_pipeline.no-overlap: "true" but not only for feature_pipeline service, but also avoid running concurrently feature_pipeline and inference_pipeline.

[taraspos]

No, that's not something Ofelia supports at the moment.

[StrangePeanut]

Thanks for submitting the issue. I am also interested in such a feature. Some jobs can be resource intensive or take a long time to complete. In this case it might be preferable to delay the execution of other jobs. There are also scenarios where a job should not be run until another job has been completed, eg one job that fills a queue and one that processes the queue.