feat(taskprocessing): add worker command for synchronous task processing#59015
Open
feat(taskprocessing): add worker command for synchronous task processing#59015
Conversation
Copilot
AI
changed the title
[WIP] Add OCC command for task processing worker
Add Mar 17, 2026
occ taskprocessing:worker dedicated synchronous task processing command
Copilot
AI
changed the title
Add
Add Mar 17, 2026
occ taskprocessing:worker dedicated synchronous task processing command--taskTypes whitelist option to taskprocessing:worker
Copilot
AI
changed the title
Add
taskprocessing:worker: add --taskTypes whitelist option
Mar 17, 2026
--taskTypes whitelist option to taskprocessing:worker
marcelklehr
reviewed
Mar 17, 2026
marcelklehr
reviewed
Mar 17, 2026
Copilot stopped work on behalf of
marcelklehr due to an error
March 17, 2026 20:50
Copilot stopped work on behalf of
marcelklehr due to an error
March 17, 2026 22:04
Copilot
AI
changed the title
Implement occ taskprocessing:worker command
Fix task type starvation in WorkerCommand::processNextTask
Mar 18, 2026
julien-nc
reviewed
Mar 18, 2026
Member
julien-nc
left a comment
There was a problem hiding this comment.
Yay 👍 ❤️
Lgtm (didn't try it yet though). Will approve after trying it.
Contributor
|
Hey, is this the implementation for #52629 or is this something else? |
Copilot stopped work on behalf of
marcelklehr due to an error
March 18, 2026 17:23
Member
|
@szaimen This is something else. We discovered that the way TaskProcessing\SynchronousBackgroundJob works has some flaws that prevent seamless parallelism. So, for taskprocessing we've opted to leave the background job realm entirely and create a new command that runs the tasks in the background. (The alternative, in our view, would be to have one job per task which seems excessive.) #52629 is separate from this effort and I realize that this PR distances us further from a unified solution. I'll pick up #52629 in our team discussion next week. |
Copilot
AI
changed the title
Implement occ taskprocessing:worker command to run tasks
feat(taskprocessing): add worker command for synchronous task processing
Mar 18, 2026
kyteinsky
reviewed
Mar 19, 2026
cdd1d26 to
b6812e1
Compare
Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
…ing:worker command Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
…uffling providers Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
…ick the oldest task Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
…tion-prevention test Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
Signed-off-by: Marcel Klehr <mklehr@gmx.net>
b6812e1 to
ad5e709
Compare
Co-authored-by: Marcel Klehr <mklehr@gmx.net> Signed-off-by: Marcel Klehr <mklehr@gmx.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
taskprocessing:workerCLI command that runs a dedicated loop processing tasks via synchronous (ISynchronousProvider) providers, with starvation prevention via a single cross-typegetNextScheduledTaskquery.Summary
New command:
occ taskprocessing:worker--timeout <s>: exit after N seconds (default: run forever)--interval <s>: sleep between idle polling cycles (default: 1s)--once: process at most one task then exit--taskTypes <id>: restrict to specific task type IDs (repeatable)Starvation prevention
Instead of shuffling providers and issuing one
getNextScheduledTaskcall per provider, all eligible task type IDs are collected first and passed in a single query. This guarantees the globally oldest scheduled task is always picked next, regardless of per-provider queue depth:Test fixes
createTask()helper now accepts an optional task type so task type IDs match provider expectationstestProcessesFirstMatchingProvider→testProcessesCorrectProviderForReturnedTaskType: updated mock expectations to reflect thatgetPreferredProvideris called for every eligible provider andgetNextScheduledTaskreceives all eligible type IDstestPicksOldestTaskAcrossMultipleEligibleProviders: verifies that with two eligible providers the task returned bygetNextScheduledTaskdetermines which provider runsTODO
Checklist
3. to review, feature component)stable32)AI (if applicable)
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.