Skip to content

Streaming from one job to another. #43

@GoogleCodeExporter

Description

@GoogleCodeExporter
Allow job a to specify a "Streams" parameter specifying a job b such that the 
output of job b will be streamed into job a. If job a has multiple tasks then 
the output will be split between them. If job s specifies multiple streams then 
the output will come from all of them and be mixed together by line.

Output will be terminated by a single EOF so job a should wait for this and 
then exit.

Streaming jobs should be submitted to the same node if possible but I'm not 
sure how this will work yet. One way to implement it is by having nodes running 
a receiving job listen to the channel of the dependency jobs until it is 
depleted then return to the main submission channel. 

Original issue reported on code.google.com by ryanbres...@gmail.com on 27 Jul 2012 at 11:35

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions