Ensure slurm/pbs array jobs are caught asynchronously

To scale on Derecho, we need to launch jobs and allow the main Julia process to pick them up asynchronously, without waiting for the full array to be ready. The key challenge will be to synchronize the workers, ensuring they have the necessary data and functions loaded despite being launched at different times.