feat: UTC-416: Implement 'Ensure all annotators are evaluated' setting for GT tasks #8905

mcanu · 2025-11-27T21:20:46Z

This PR was automatically generated via Follow Merge. Workflow Run
Please ensure that all linked upstream Pull Requests are merged before proceeding with this one.

feat: UTC-416: Implement 'Ensure all annotators are evaluated' setting for GT tasks label-studio-sdk#690

netlify · 2025-11-27T21:20:52Z

✅ Deploy Preview for label-studio-docs-new-theme canceled.

Name	Link
🔨 Latest commit	`c7d1448`
🔍 Latest deploy log	https://app.netlify.com/projects/label-studio-docs-new-theme/deploys/6940116ce4d0020008b16b7e

netlify · 2025-11-27T21:20:53Z

✅ Deploy Preview for label-studio-storybook ready!

Name	Link
🔨 Latest commit	`c7d1448`
🔍 Latest deploy log	https://app.netlify.com/projects/label-studio-storybook/deploys/6940116cc7de4b0008acbb98
😎 Deploy Preview	https://deploy-preview-8905--label-studio-storybook.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

netlify · 2025-11-27T21:20:54Z

✅ Deploy Preview for heartex-docs canceled.

Name	Link
🔨 Latest commit	`c7d1448`
🔍 Latest deploy log	https://app.netlify.com/projects/heartex-docs/deploys/6940116cf558ad00082a92d4

netlify · 2025-11-27T21:21:25Z

✅ Deploy Preview for label-studio-playground ready!

Name	Link
🔨 Latest commit	`c7d1448`
🔍 Latest deploy log	https://app.netlify.com/projects/label-studio-playground/deploys/6940116cbd4b89000839ffe9
😎 Deploy Preview	https://deploy-preview-8905--label-studio-playground.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

codecov · 2025-11-27T21:24:48Z

Codecov Report

❌ Patch coverage is 68.18182% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.65%. Comparing base (ef81f78) to head (c7d1448).
⚠️ Report is 2 commits behind head on develop.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
label_studio/projects/functions/next_task.py	61.11%	7 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8905      +/-   ##
===========================================
+ Coverage    66.57%   66.65%   +0.08%     
===========================================
  Files          829      829              
  Lines        64782    64789       +7     
  Branches     10990    10990              
===========================================
+ Hits         43129    43188      +59     
+ Misses       21649    21597      -52     
  Partials         4        4

Flag	Coverage Δ
lsf-e2e	`52.54% <ø> (+<0.01%)`	⬆️
lsf-integration	`48.46% <ø> (+0.13%)`	⬆️
lsf-unit	`8.81% <ø> (ø)`
pytests	`81.43% <68.18%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

nass600 · 2025-12-04T09:55:46Z

/fm sync

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19924849173

matt-bernstein · 2025-12-05T18:44:15Z

/git merge develop

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19972691403

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19972821374

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19972907652

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19974215346

label_studio/projects/api.py

label_studio/projects/functions/next_task.py

label_studio/projects/models.py

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19974288596

mcanu · 2025-12-08T14:17:36Z

/fm sync

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20044652944

mcanu · 2025-12-08T22:32:05Z

/fm sync

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20045038930

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20045264167

mcanu · 2025-12-09T00:52:46Z

/fm sync

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20048026880

mcanu · 2025-12-09T16:39:36Z

/git merge develop

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20071148448

mcanu · 2025-12-10T13:40:03Z

/fm sync

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20100599882

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20100847806

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20100945769

label_studio/projects/functions/next_task.py

bmartel

Looks like a very precise change, well thought out!

mcanu · 2025-12-11T19:18:49Z

/git merge develop

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20144673543

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20145073906

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20145278941

mcanu · 2025-12-12T21:25:51Z

/fm sync

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20180337188

mcanu · 2025-12-15T13:44:42Z

/fm sync

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20234412494

Added show_ground_truth_always field in project and handle next_task

cf13097

mcanu requested a review from a team as a code owner November 27, 2025 21:20

github-actions bot added the feat label Nov 27, 2025

Modified migration

7148a06

mcanu requested review from hakan458 and matt-bernstein November 27, 2025 21:35

mcanu added 2 commits November 27, 2025 18:53

Fix migration default

11b1697

Fixed migration

cf65c8f

nass600 added 2 commits December 4, 2025 09:59

Sync Follow Merge dependencies

736ddde

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19924849173

Merge branch 'develop' into 'fb-utc-416'

d17a17a

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19924849173

robot-ci-heartex assigned nass600 Dec 4, 2025

robot-ci-heartex added the FM Stale label Dec 4, 2025

nass600 and others added 3 commits December 4, 2025 17:13

added warning badge

655fe5d

added icons

a3fa9b0

Added new fields

d8dc179

matt-bernstein and others added 5 commits December 5, 2025 18:44

Merge branch 'develop' into 'fb-utc-416'

5d38ddf

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19972691403

Sync Follow Merge dependencies

a995df7

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19972821374

Sync Follow Merge dependencies

fc8aabc

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19972907652

Using new fields

cdc89b6

Sync Follow Merge dependencies

2ae8516

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19974215346

matt-bernstein approved these changes Dec 5, 2025

View reviewed changes

label_studio/projects/api.py Show resolved Hide resolved

label_studio/projects/functions/next_task.py Show resolved Hide resolved

label_studio/projects/models.py Outdated Show resolved Hide resolved

Sync Follow Merge dependencies

9788e19

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/19974288596

robot-ci-heartex and others added 2 commits December 8, 2025 22:20

Sync Follow Merge dependencies

da019ac

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20044652944

Fix test

7af4a7e

mcanu and others added 2 commits December 8, 2025 22:34

Sync Follow Merge dependencies

3e5ee9a

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20045038930

Sync Follow Merge dependencies

12275ef

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20045264167

Sync Follow Merge dependencies

256b4b0

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20048026880

mcanu and others added 2 commits December 9, 2025 16:40

Merge branch 'develop' into 'fb-utc-416'

6e80b9c

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20071148448

annotator_evaluation_enabled default to False

719e00e

mcanu requested a review from matt-bernstein December 9, 2025 21:56

Fix columns test

efda0ee

mcanu and others added 4 commits December 10, 2025 13:43

Sync Follow Merge dependencies

5ce366e

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20100599882

Merge branch 'develop' into 'fb-utc-416'

2990a35

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20100599882

Sync Follow Merge dependencies

31f3007

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20100847806

Sync Follow Merge dependencies

798c8d0

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20100945769

bmartel reviewed Dec 10, 2025

View reviewed changes

label_studio/projects/functions/next_task.py Show resolved Hide resolved

bmartel approved these changes Dec 10, 2025

View reviewed changes

mcanu and others added 4 commits December 11, 2025 19:19

Merge branch 'develop' into 'fb-utc-416'

367c9b1

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20144673543

Sync Follow Merge dependencies

254b3af

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20145073906

Merge branch 'develop' into 'fb-utc-416'

a1a7741

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20145073906

Sync Follow Merge dependencies

86598f5

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20145278941

mcanu added 2 commits December 12, 2025 21:29

Sync Follow Merge dependencies

b94d723

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20180337188

Merge branch 'develop' into 'fb-utc-416'

b4c1f2d

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20180337188

mcanu added 2 commits December 15, 2025 13:47

Sync Follow Merge dependencies

862d4b4

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20234412494

Merge branch 'develop' into 'fb-utc-416'

c7d1448

Workflow run: https://github.com/HumanSignal/label-studio/actions/runs/20234412494

feat: UTC-416: Implement 'Ensure all annotators are evaluated' setting for GT tasks #8905

Are you sure you want to change the base?

feat: UTC-416: Implement 'Ensure all annotators are evaluated' setting for GT tasks #8905

Conversation

mcanu commented Nov 27, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for label-studio-docs-new-theme canceled.

Uh oh!

netlify bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for label-studio-storybook ready!

Uh oh!

netlify bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for heartex-docs canceled.

Uh oh!

netlify bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for label-studio-playground ready!

Uh oh!

codecov bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

nass600 commented Dec 4, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matt-bernstein commented Dec 5, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mcanu commented Dec 8, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcanu commented Dec 8, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcanu commented Dec 9, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcanu commented Dec 9, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcanu commented Dec 10, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

bmartel left a comment

Choose a reason for hiding this comment

Uh oh!

mcanu commented Dec 11, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcanu commented Dec 12, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcanu commented Dec 15, 2025 • edited by robot-ci-heartex Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mcanu commented Nov 27, 2025 •

edited by robot-ci-heartex

Loading

netlify bot commented Nov 27, 2025 •

edited

Loading

netlify bot commented Nov 27, 2025 •

edited

Loading

netlify bot commented Nov 27, 2025 •

edited

Loading

netlify bot commented Nov 27, 2025 •

edited

Loading

codecov bot commented Nov 27, 2025 •

edited

Loading

nass600 commented Dec 4, 2025 •

edited by robot-ci-heartex

Loading

matt-bernstein commented Dec 5, 2025 •

edited by robot-ci-heartex

Loading

mcanu commented Dec 8, 2025 •

edited by robot-ci-heartex

Loading

mcanu commented Dec 8, 2025 •

edited by robot-ci-heartex

Loading

mcanu commented Dec 9, 2025 •

edited by robot-ci-heartex

Loading

mcanu commented Dec 9, 2025 •

edited by robot-ci-heartex

Loading

mcanu commented Dec 10, 2025 •

edited by robot-ci-heartex

Loading

mcanu commented Dec 11, 2025 •

edited by robot-ci-heartex

Loading

mcanu commented Dec 12, 2025 •

edited by robot-ci-heartex

Loading

mcanu commented Dec 15, 2025 •

edited by robot-ci-heartex

Loading