General Case
The dispatcher currently waits for all parallel function invocations to complete before assembling the output sets and propagating them to the downstream functions. As a result, downstream function invocations are only scheduled once the last parent function has fully completed. The non-blocking each sharding, that creates a new invocation for each input element, does not require the input set to be complete and would allow to create the next invocations immediately.
Keyed Shardings
We can apply the same optimization for keyed shardings if we know that a function's output sets have the same keys as the input sets (which allows us to know if we got back all items of the same key). This is the case for the system provided HTTP communication function. Further, we could add a new attribute that indicates this for user-defined compute functions which would allow the scheduler to apply this optimization for those functions as well.
Implementation
One approach to implement this optimization is to detect the above described composition parts during parsing and scheduling them as separate sub-compositions. Another approach is to change the scheduling granularity from a set-level to an item-level and add some extra information.
What is not quite clear yet is what approach best fits the multi-node system...
General Case
The dispatcher currently waits for all parallel function invocations to complete before assembling the output sets and propagating them to the downstream functions. As a result, downstream function invocations are only scheduled once the last parent function has fully completed. The non-blocking
eachsharding, that creates a new invocation for each input element, does not require the input set to be complete and would allow to create the next invocations immediately.Keyed Shardings
We can apply the same optimization for
keyedshardings if we know that a function's output sets have the same keys as the input sets (which allows us to know if we got back all items of the same key). This is the case for the system providedHTTPcommunication function. Further, we could add a new attribute that indicates this for user-defined compute functions which would allow the scheduler to apply this optimization for those functions as well.Implementation
One approach to implement this optimization is to detect the above described composition parts during parsing and scheduling them as separate sub-compositions. Another approach is to change the scheduling granularity from a set-level to an item-level and add some extra information.
What is not quite clear yet is what approach best fits the multi-node system...