feat(taskprocessing): add worker command for synchronous task processing#59015
feat(taskprocessing): add worker command for synchronous task processing#59015
Conversation
Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
occ taskprocessing:worker dedicated synchronous task processing command
Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
occ taskprocessing:worker dedicated synchronous task processing command--taskTypes whitelist option to taskprocessing:worker
Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
--taskTypes whitelist option to taskprocessing:worker| declare(strict_types=1); | ||
|
|
||
| /** | ||
| * SPDX-FileCopyrightText: 2024 Nextcloud GmbH and Nextcloud contributors |
There was a problem hiding this comment.
| * SPDX-FileCopyrightText: 2024 Nextcloud GmbH and Nextcloud contributors | |
| * SPDX-FileCopyrightText: 2026 Nextcloud GmbH and Nextcloud contributors |
| declare(strict_types=1); | ||
|
|
||
| /** | ||
| * SPDX-FileCopyrightText: 2024 Nextcloud GmbH and Nextcloud contributors |
There was a problem hiding this comment.
| * SPDX-FileCopyrightText: 2024 Nextcloud GmbH and Nextcloud contributors | |
| * SPDX-FileCopyrightText: 2026 Nextcloud GmbH and Nextcloud contributors |
…ng providers Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
…he oldest task Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
julien-nc
left a comment
There was a problem hiding this comment.
Yay 👍 ❤️
Lgtm (didn't try it yet though). Will approve after trying it.
|
Hey, is this the implementation for #52629 or is this something else? |
|
@szaimen This is something else. We discovered that the way TaskProcessing\SynchronousBackgroundJob works has some flaws that prevent seamless parallelism. So, for taskprocessing we've opted to leave the background job realm entirely and create a new command that runs the tasks in the background. (The alternative, in our view, would be to have one job per task which seems excessive.) #52629 is separate from this effort and I realize that this PR distances us further from a unified solution. I'll pick up #52629 in our team discussion next week. |
…tion-prevention test Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
Adds a
taskprocessing:workerCLI command that runs a dedicated loop processing tasks via synchronous (ISynchronousProvider) providers, with starvation prevention via a single cross-typegetNextScheduledTaskquery.Summary
New command:
occ taskprocessing:worker--timeout <s>: exit after N seconds (default: run forever)--interval <s>: sleep between idle polling cycles (default: 1s)--once: process at most one task then exit--taskTypes <id>: restrict to specific task type IDs (repeatable)Starvation prevention
Instead of shuffling providers and issuing one
getNextScheduledTaskcall per provider, all eligible task type IDs are collected first and passed in a single query. This guarantees the globally oldest scheduled task is always picked next, regardless of per-provider queue depth:Test fixes
createTask()helper now accepts an optional task type so task type IDs match provider expectationstestProcessesFirstMatchingProvider→testProcessesCorrectProviderForReturnedTaskType: updated mock expectations to reflect thatgetPreferredProvideris called for every eligible provider andgetNextScheduledTaskreceives all eligible type IDstestPicksOldestTaskAcrossMultipleEligibleProviders: verifies that with two eligible providers the task returned bygetNextScheduledTaskdetermines which provider runsTODO
Checklist
3. to review, feature component)stable32)AI (if applicable)
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.