Issue on log parsing logic implemented by PR#1335

[DiegoTavares]

The new parsing logic implemented by #1335 has an issue when it comes to consuming lines from stdout and stderr in the following loops:

         with open(self.rqlog.name, 'a') as f:
                # Convert to ASCII while discarding characters that can not be encoded
                for line in frameInfo.forkedCommand.stdout:
                    line = line.encode('ascii', 'ignore')
                    f.write(line.decode('ascii') + '\n')
                for line in frameInfo.forkedCommand.stderr:
                    line = line.encode('ascii', 'ignore')
                    f.write(line.decode('ascii') + '\n')

This logic has to wait until all content of stdout is consumed before consuming content from stderr, which means users monitoring the logs of running jobs will not have a real-time feed of what the job is doing. Besides that, a failure might cause the process to exit before giving it the change to have the content of stderr processed.

OpenCue/rqd/rqd/rqcore.py

Line 342 in 688425c

else:

[DiegoTavares]

FYI @ramonfigueiredo

[ramonfigueiredo]

FYI @ramonfigueiredo

Ok. I'll take a look on that asap.

Thanks!

[ramonfigueiredo]

@DiegoTavares

I created a new PR to fix this problem. See the PR #1427