Add frame range to Blender command in CueSubmit #1337

n-jay · 2024-01-19T19:35:50Z

Link the Issue(s) this Pull Request is related to.
#1334
#1264

Summarize your change.
Adds frame range to the Blender render command in CueSubmit.
Required for when rendering animations.
Related Blender command linked here.

Extracted from layerRange value in layerData Import regex module

Line too long

n-jay · 2024-01-28T17:44:42Z

Hi all,
it appears that the animation rendering happens successfully as per the RQD logs

However, there appears to be a TypeError popping up from rqdmachine.py

Meanwhile, Cuebot is throwing a thread-related error.

Does this mean the <startframe number> - <endframe number> format is unsupported in the Frame Spec field?

…/OpenCue into blender-framerange

bcipriano

However, there appears to be a TypeError popping up from rqdmachine.py

This looks unrelated to this change. Looks like that part of the code is scanning a file in /proc to collect machine stats. Probably RQD is not familiar with the format of that file's content. Might need a separate PR.

Meanwhile, Cuebot is throwing a thread-related error.

This also looks unrelated to me. I could be wrong but I think it's nothing to worry about -- some race conditions are expected with Cuebot's task management and it should handle this case fine. @DiegoTavares can probably confirm.

bcipriano · 2024-01-29T17:22:50Z

cuesubmit/cuesubmit/Submission.py

+            renderCommand += ' -f {}'.format(frameRange)
+        elif re.match(r'^\d+-\d+$', frameRange):
+            startFrame, endFrame = map(int, frameRange.split("-"))
+            renderCommand += (' -s {startFrame} -e {endFrame} -a'


If I'm reading this correctly, I think this will result in a layer command with the start/end frames hardcoded. I.e., if the frame range is 1-5 the command will contain -s 1 -e 5 -a.

Compare this to the code prior to this change, which instead inserts the #IFRAME# token to act as a placeholder for the frame.

I think instead, we want to use tokens for start frame / end frame:

OpenCue/cuesubmit/cuesubmit/Constants.py

Lines 45 to 54 in 5b9defd

COMMAND_TOKENS = {'#ZFRAME#': 'Current frame with a padding of 4',

'#IFRAME#': 'Current frame',

'#FRAME_START#': 'First frame of chunk',

'#FRAME_END#': 'Last frame of chunk',

'#FRAME_CHUNK#': 'Chunk size',

'#FRAMESPEC#': 'Full frame range',

'#LAYER#': 'Name of the Layer',

'#JOB#': 'Name of the Job',

'#FRAME#': 'Name of the Frame'

}

These tokens should allow Cuebot to swap in the correct frame numbers for each task based on the chunk size.

The thread relate error on cuebot is normal, it's related to a race condition that's handled first come first.

@bcipriano noted.
I was under the impression that the Frame Spec field solely defined the frame range (1-5), hence why I got it directly from that but had no idea about the Chunk Size.

...should allow Cuebot to swap in the correct frame numbers for each task based on the chunk size.

Got it.
To make sure I follow correctly; Frame Spec can be populated with a frame range such as 1-5, and then Chunk Size can be set as an integer value (eg: 2) which defines the number of frames within that 1-5 range that will be processed per RQD node, yes?
And the dedicated #tokens can be used to substitute the respective values.

Sorry, I'm not very familiar with the concept of chunk sizes in render farms, hence the confusion 😅

Yes, you've got that right.

To extend the same example, if the frame range is 1-5 and the chunk size is 2, the set of tasks for that layer should be:

Frames 1-2

Frames 3-4

Frame 5 (don't go past the last frame, despite the chunk size setting)

Cuebot should handle all of this already -- dividing up the frame range into chunks and the replacement of the #FRAME_START# / #FRAME_END# tokens -- so no need to do any of those calculations at this point in the code.

Kind of confusingly, in OpenCue each task is called a "frame". This is because when OpenCue was originally written, it only operated on single frames -- chunk size was always assumed to be 1. We added the chunk size concept later, but the naming stayed the same.

@bcipriano I've resolved this in ce618a0.

Verified on test animation.

animation frame range

bcipriano

Code looks good, just a few minor requests.

cuesubmit/cuesubmit/Constants.py

cuesubmit/cuesubmit/Submission.py

bcipriano · 2024-02-14T20:07:58Z

cuesubmit/cuesubmit/Submission.py

@@ -72,8 +73,14 @@ def buildBlenderCmd(layerData):
        renderCommand += ' -o {}'.format(outputPath)
    if outputFormat:
        renderCommand += ' -F {}'.format(outputFormat)
-    # The render frame must come after the scene and output
-    renderCommand += ' -f {frameToken}'.format(frameToken=Constants.FRAME_TOKEN)
+    if frameRange:


It would be great if you could add unit tests to cover both the single-frame and start/end frame cases.

https://github.com/AcademySoftwareFoundation/OpenCue/blob/master/cuesubmit/tests/Submission_tests.py

Added in 2d4ba88

@bcipriano however, I'm having a bit of trouble with lines #178 and #153 of Submission_tests.py
The test fails when I set it to blender but passes when I set it to shell. This is despite the layerType being set as Blender. Any thoughts on this?

Ping @bcipriano

Update comment explaining render frame range Blender command

For single frame and multi frame Blender renders

bcipriano · 2024-05-09T18:19:23Z

@n-jay Looks like we're getting some python unit test failures here, could you check those out?

n-jay · 2024-05-09T20:04:25Z

@n-jay Looks like we're getting some python unit test failures here, could you check those out?

@bcipriano, yes I caught this failure and was hoping for some feedback: #1337 (comment) 😄

bcipriano

@n-jay Sorry, missed that. See new comment.

bcipriano · 2024-05-09T20:26:02Z

cuesubmit/tests/Submission_tests.py

+            'blenderFile': '/path/to/scene.blend',
+            'outputFormat': 'PNG'},
+    'layerRange': '3-9',
+    'cores': '2'


Do you need to add a services line like the Maya / Nuke tests do?

I think this is causing the job to default to the shell service, so the assertion on line 153 fails.

@bcipriano noted, didn't catch that one. 😄
Resolved in a17be62

A new set of unit test failures seem to be coming from some unrelated changes.
Unsure, but I think the CueGUI fail is caused by this one: 65b74fe
CC: @DiegoTavares

Sorry about that, I'm fixing the pipeline on muster

Pipeline fixed by #1360

Noted @DiegoTavares, thanks for the update!

…/OpenCue into blender-framerange

…ation#1337) * Add frame range to Blender render command Extracted from layerRange value in layerData Import regex module * Fix linting issue Line too long * Add variables for frame range tokens * Update Blender command with tokens for animation frame range * Remove unused import * Add suffix to frame start/end variables * Update comment Update comment explaining render frame range Blender command * Add unit tests For single frame and multi frame Blender renders * Add missing services line to blender unit test layer data

Add frame range to Blender render command

2009e12

Extracted from layerRange value in layerData Import regex module

n-jay force-pushed the blender-framerange branch from 7a29205 to 2009e12 Compare January 19, 2024 19:41

Fix linting issue

2ff33f5

Line too long

Merge branch 'master' of https://github.com/AcademySoftwareFoundation…

29fdbeb

…/OpenCue into blender-framerange

n-jay marked this pull request as ready for review January 28, 2024 17:52

n-jay requested review from bcipriano, gregdenton, jrray, smith1511, larsbijl, DiegoTavares, IdrisMiles and splhack as code owners January 28, 2024 17:52

bcipriano requested changes Jan 29, 2024

View reviewed changes

n-jay added 3 commits February 2, 2024 14:16

Add variables for frame range tokens

507a11a

Update Blender command with tokens for

ce618a0

animation frame range

Remove unused import

3c0f2ff

bcipriano requested changes Feb 14, 2024

View reviewed changes

n-jay added 3 commits February 17, 2024 23:55

Add suffix to frame start/end variables

d3d7c6d

Update comment

2d254c7

Update comment explaining render frame range Blender command

Add unit tests

2d4ba88

For single frame and multi frame Blender renders

bcipriano reviewed May 9, 2024

View reviewed changes

n-jay added 3 commits May 10, 2024 23:00

Add missing services line to blender unit test layer data

a17be62

Merge branch 'master' of https://github.com/AcademySoftwareFoundation…

ee2af1e

…/OpenCue into blender-framerange

Merge branch 'master' of https://github.com/AcademySoftwareFoundation…

41b6e81

…/OpenCue into blender-framerange

n-jay requested a review from ramonfigueiredo as a code owner May 14, 2024 08:37

n-jay requested a review from bcipriano May 14, 2024 08:44

bcipriano approved these changes May 14, 2024

View reviewed changes

DiegoTavares merged commit d6e859e into AcademySoftwareFoundation:master May 21, 2024
12 checks passed

n-jay mentioned this pull request May 26, 2024

[plugin] Draft: Develop Blender plugin #1309

Draft

n-jay mentioned this pull request Jun 19, 2024

Add framespec parameter format check to CueSubmit #1389

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add frame range to Blender command in CueSubmit #1337

Add frame range to Blender command in CueSubmit #1337

n-jay commented Jan 19, 2024 •

edited

Loading

n-jay commented Jan 28, 2024 •

edited

Loading

bcipriano left a comment

bcipriano Jan 29, 2024

DiegoTavares Jan 29, 2024

n-jay Jan 30, 2024

bcipriano Jan 30, 2024

n-jay Feb 2, 2024

bcipriano left a comment

bcipriano Feb 14, 2024

n-jay Feb 26, 2024

n-jay Mar 27, 2024

bcipriano commented May 9, 2024

n-jay commented May 9, 2024

bcipriano left a comment

bcipriano May 9, 2024

n-jay May 10, 2024 •

edited

Loading

DiegoTavares May 10, 2024

DiegoTavares May 13, 2024

n-jay May 14, 2024

	COMMAND_TOKENS = {'#ZFRAME#': 'Current frame with a padding of 4',
	'#IFRAME#': 'Current frame',
	'#FRAME_START#': 'First frame of chunk',
	'#FRAME_END#': 'Last frame of chunk',
	'#FRAME_CHUNK#': 'Chunk size',
	'#FRAMESPEC#': 'Full frame range',
	'#LAYER#': 'Name of the Layer',
	'#JOB#': 'Name of the Job',
	'#FRAME#': 'Name of the Frame'
	}

Add frame range to Blender command in CueSubmit #1337

Add frame range to Blender command in CueSubmit #1337

Conversation

n-jay commented Jan 19, 2024 • edited Loading

n-jay commented Jan 28, 2024 • edited Loading

bcipriano left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bcipriano left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bcipriano commented May 9, 2024

n-jay commented May 9, 2024

bcipriano left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

n-jay May 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

n-jay commented Jan 19, 2024 •

edited

Loading

n-jay commented Jan 28, 2024 •

edited

Loading

n-jay May 10, 2024 •

edited

Loading