Flake error

[EmileSonneveld]

This job failed on Jenkins and has no clear reason of failing.
Job id: j-260312201943483ebec4d60732c5fce8

Re running made it work: j-26031310504249d09dd987a37e4ea7e1 but should be investigated nontheless.

[EmileSonneveld]

An other job that failed once and re-launching worked: j-26031615002145aeaceb28c9cbeb16e3

stack trace

Traceback (most recent call last):
File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 2014, in apply_process
  return process_function(args=ProcessArgs(args, process_id=process_id), env=env)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/openeo-geopyspark-k8s-custom-processes/src/openeo_geopyspark_k8s_custom_processes/custom_processes.py", line 495, in run_cwl_to_stac
  stac_root_new = cwl_common_to_stac(
                  ^^^^^^^^^^^^^^^^^^^
File "/opt/openeo-geopyspark-k8s-custom-processes/src/openeo_geopyspark_k8s_custom_processes/custom_processes.py", line 115, in cwl_common_to_stac
  results = launcher.run_cwl_workflow(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/integrations/calrissian.py", line 685, in run_cwl_workflow
  cwl_job = self.launch_job_and_wait(manifest=cwl_manifest)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/integrations/calrissian.py", line 624, in launch_job_and_wait
  raise RuntimeError(
RuntimeError: CWL Job j-26031615002145aeac-cal-cwl-7f348c62 failed with final_status='failed' after timer.elapsed()=475.59s.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py", line 1177, in start_main
  main(sys.argv)
File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py", line 260, in main
  run_driver()
File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py", line 221, in run_driver
  run_job(
File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/utils.py", line 66, in memory_logging_wrapper
  return function(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeogeotrellis/deploy/batch_job.py", line 351, in run_job
  result = ProcessGraphDeserializer.evaluate(process_graph, env=env, do_dry_run=tracer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 496, in evaluate
  result = convert_node(top_level_node, env=env)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 580, in convert_node
  process_result = apply_process(
                   ^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 1994, in apply_process
  args = {name: convert_node(expr, env=env) for (name, expr) in sorted(args.items())}
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 1994, in <dictcomp>
  args = {name: convert_node(expr, env=env) for (name, expr) in sorted(args.items())}
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 621, in convert_node
  return [convert_node(x, env=env) for x in processGraph]
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 621, in <listcomp>
  return [convert_node(x, env=env) for x in processGraph]
          ^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 580, in convert_node
  process_result = apply_process(
                   ^^^^^^^^^^^^^^
File "/opt/venv/lib64/python3.11/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 2024, in apply_process
  raise OpenEOApiException(f"Unexpected error during {process_id!r}: {detail}. The process had these arguments: {args!r} ") from e
openeo_driver.errors.OpenEOApiException: Unexpected error during 'run_cwl_to_stac': RuntimeError("CWL Job j-26031615002145aeac-cal-cwl-7f348c62 failed with final_status='failed' after timer.elapsed()=475.59s."). The process had these arguments: {'context': {'temporal_extent': ['2018-01-28', '2018-02-03'], 'temporal_baseline': 6, 'burst_id': 329488, 'coherence_window_az': 2, 'coherence_window_rg': 10, 'n_az_looks': 1, 'n_rg_looks': 4, 'polarization': 'vh', 'sub_swath': 'IW2'}, 'cwl_url': 'https://raw.githubusercontent.com/cloudinsar/s1-workflows/refs/heads/main/cwl/sar_coherence.cwl'} 

[EmileSonneveld]

An other job that failed with no clear cause: j-260401095051489a892344a31168ea69
Final application error message: driver container failed with ExitCode: 1, Reason: Error