A proposed ci.yml refactoring splits flux-ubuntu tests into four separate jobs and runs them in parallel on separate nodes. Three jobs cover SPINDLE_DEBUG=[1-3] and the fourth does not set SPINDLE_DEBUG.
The run in question succeeded for SPINDLE_DEBUG=[1,3] and the no-debugging option, but failed on SPINDLE_DEBUG=2 with the error message:
Run docker exec node-1 bash -c 'cd Spindle-build/testsuite && SPINDLE_DEBUG=2 flux alloc --nodes=${workers} ./runTests --nodes=${workers} --tasks-per-node=3'
docker exec node-1 bash -c 'cd Spindle-build/testsuite && SPINDLE_DEBUG=2 flux alloc --nodes=${workers} ./runTests --nodes=${workers} --tasks-per-node=3'
shell: /usr/bin/bash -e {0}
Running: ./run_driver --dependency --push
60.237s: flux-shell[0]: stderr: spindleRunBE failed!
Error: The action 'Run spindle-flux-ubuntu-debug2 testsuite' has timed out after 7 minutes.
At that point the CI script should extract the log files, but based on the ls the script performs, no files were generated. Looking at the likely code path, LOGGING_FINI is not called during any of the error paths, so the log files were likely not flushed to disk before spindle exited.
A proposed
ci.ymlrefactoring splits flux-ubuntu tests into four separate jobs and runs them in parallel on separate nodes. Three jobs cover SPINDLE_DEBUG=[1-3] and the fourth does not set SPINDLE_DEBUG.The run in question succeeded for SPINDLE_DEBUG=[1,3] and the no-debugging option, but failed on SPINDLE_DEBUG=2 with the error message:
At that point the CI script should extract the log files, but based on the
lsthe script performs, no files were generated. Looking at the likely code path,LOGGING_FINIis not called during any of the error paths, so the log files were likely not flushed to disk before spindle exited.