Feat/add procedure api

[d-perl]

Adds functionality to the procedure manager and associated tooling:

• a helper class for Redis interactions, separated for frontend and backend use
• get statuses of running procedures
• abort procedures, reschedule aborted procedures, delete aborted procedures
• new messages for the above
• a builtin procedure for running user scripts
• tests

Helpful for testing:

• run the _dev_runner.py

• Make sure you have a pod created with podman pod create --add-host redis:host-gateway local_bec

• In a bec console:

from bec_server.scan_server.procedures.helper import BackendProcedureHelper
from bec_lib import messages
helper = BackendProcedureHelper(bec.connector)
msg = messages.ProcedureRequestMessage( identifier="sleep", queue="queue1", args_kwargs=((), {"time_s":5}) )
helper.request_procedure(msg)

bec-project/bec_widgets#902 can be used to monitor queues, abort and reschedule procedures, as well as run scripts as procedures

[codecov]

Codecov Report

❌ Patch coverage is 75.62189% with 98 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...server/bec_server/scan_server/procedures/helper.py	72.50%	29 Missing and 4 partials ⚠️
..._server/scan_server/procedures/container_worker.py	66.66%	23 Missing and 2 partials ⚠️
...erver/bec_server/scan_server/procedures/manager.py	77.27%	19 Missing and 6 partials ⚠️
...c_server/scan_server/procedures/container_utils.py	40.00%	8 Missing and 1 partial ⚠️
bec_lib/bec_lib/logger.py	66.66%	1 Missing and 1 partial ⚠️
...r/bec_server/scan_server/procedures/worker_base.py	85.71%	1 Missing and 1 partial ⚠️
bec_lib/bec_lib/redis_connector.py	83.33%	0 Missing and 1 partial ⚠️
...erver/scan_server/procedures/builtin_procedures.py	50.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[d-perl]

@wakonig I just undid the logging commit, we should maybe discuss it in more detail sometime outside of this PR

[wakonig]

@d-perl sorry for the delay. I just tried to run it but it first got stuck and never picked it up. After a restart of the _dev_runner everything was fine.

We should have a quick discussion about the logs tomorrow though. I used the companion branch in BW but I wasn't able actually see the scan table from the procedure.

[wakonig]

works great! 🚀

[wakonig]

just out of curiosity... could you rebase your branch on main and see if the coverage report is fixed? Also fine if you don't want to do this on this branch ;)