Asynchronous pruning for RubyThreadPoolExecutor #1082
Conversation
joshuay03
force-pushed
the
better-ruby-thread-pool-executor-pruning
branch
15 times, most recently
from
b6e5656 to
f90d46d
Compare
joshuay03
force-pushed
the
better-ruby-thread-pool-executor-pruning
branch
14 times, most recently
from
c5eca7d to
89b67f4
Compare
joshuay03
force-pushed
the
better-ruby-thread-pool-executor-pruning
branch
6 times, most recently
from
547428b to
a675844
Compare
lib/concurrent-ruby/concurrent/collection/ruby_timeout_queue.rb
Outdated
Show resolved
Hide resolved
joshuay03
force-pushed
the
better-ruby-thread-pool-executor-pruning
branch
7 times, most recently
from
fd02056 to
6cedac3
Compare
lib/concurrent-ruby/concurrent/collection/ruby_timeout_queue.rb
Outdated
Show resolved
Hide resolved
lib/concurrent-ruby/concurrent/collection/ruby_timeout_queue.rb
Outdated
Show resolved
Hide resolved
joshuay03
force-pushed
the
better-ruby-thread-pool-executor-pruning
branch
4 times, most recently
from
defd0f7 to
9739e9a
Compare
lib/concurrent-ruby/concurrent/collection/ruby_timeout_queue.rb
Outdated
Show resolved
Hide resolved
joshuay03
force-pushed
the
better-ruby-thread-pool-executor-pruning
branch
from
9739e9a to
6aa4ee5
Compare
There was a problem hiding this comment.
The Queue stuff looks ready to me, i.e. I reviewed and approve that part.
The pool changes it's hard for me to tell because I haven't had time to understand the implementation in details (and likely won't have time for that soon).
It would be great if another concurrent-ruby maintainer could help review that (and they should feel free to merge this PR after they approve)
| throw :stop | ||
| end | ||
|
|
||
| prunable = false |
There was a problem hiding this comment.
Is it impossible if prunable_capacity is <= 0 that a worker would need to be prunable again?
i.e. what if prunable_capacity is 0 because all workers are busy but later on the pool is no longer running? and then we should prune/finish all workers?
There was a problem hiding this comment.
i.e. what if prunable_capacity is 0 because all workers are busy but later on the pool is no longer running? and then we should prune/finish all workers?
If a worker is busy they will be marked as ready when the task has completed, during which we stop them (in #ns_ready_worker) if the pool is no longer running. Even if the pool is only stopped after this point, we also mark that worker as prune-able again, which means they will eventually time out (cause work is no longer being accepted) and prune themselves.
lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb
Outdated
Show resolved
Hide resolved
lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb
Outdated
Show resolved
Hide resolved
lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb
Outdated
Show resolved
Hide resolved
joshuay03
force-pushed
the
better-ruby-thread-pool-executor-pruning
branch
3 times, most recently
from
e4f5794 to
4d088a0
Compare
joshuay03
force-pushed
the
better-ruby-thread-pool-executor-pruning
branch
from
4d088a0 to
cb37798
Compare
3 participants
Closes #1066
Closes #1075
Alternative to #1079
Implementation is based on the discussion in the linked issues.