job_queue: add logging of long build queue #20
Conversation
| @mutex = Mutex.new | ||
| @queue = Hash.new { |h, k| h[k] = [] } | ||
| @queue_type = queue_type | ||
| @condvar = ConditionVariable.new | ||
| @logger = logger |
There was a problem hiding this comment.
This feels a bit like violence, but I wasn't sure how else to do it. Suggestions welcome.
There was a problem hiding this comment.
Seems fine. Could pull in Rails logging gems if you wanted something better?
Reminds me that Sorbet feels like it would be a really good fit for ci-orchestrator.
There was a problem hiding this comment.
Reminds me that Sorbet feels like it would be a really good fit for ci-orchestrator.
Yeah, orka_api_client supports Sorbet very well but doesn't work with it being a git gem.
However I might be swapping out that for a K8s backed solution now that the Orka 3 changed things, so watch this space.
| job = @queue[:long].shift | ||
| @logger.call("Long build slot available. Scheduling #{job.runner_name} for deployment...") | ||
| break job |
There was a problem hiding this comment.
This is already implied by the existing Deploying VM for job *-long tbh
There was a problem hiding this comment.
I thought about that, but not quite, since other things can happen between a job getting popped and and VM being deployed.
src/job_queue.rb
Outdated
| @logger.call("Running Long Build Jobs: #{running_long_build_count}") | ||
| @logger.call("Available Long Build Slots: #{long_build_slots}") |
There was a problem hiding this comment.
Maybe having a webpage tab with numbers might be more useful here, including total number of jobs deployed and slots available.
There was a problem hiding this comment.
Yea, a dashboard would be useful, but I think logs are also important so I can see how these values evolve over time. Not sure if you also had a log in mind with a separate tab.
There was a problem hiding this comment.
Ok, fair. Thoughts on combining these two into one line so it doesn't fill up the log history limit as much?
e.g. Long builds: x running, x available
This will make it easier to monitor whether the new queueing system is behaving as expected.
carlocab
force-pushed
the
verbose-long-build-logs
branch
from
387f179 to
bdce7b2
Compare
|
Thanks! |
This will make it easier to monitor whether the new queueing system is
behaving as expected.