Skip to content

Refactor Composition Scheduling #112

Description

@tostocker

Current State

The current Composition structure holds a vector of function applications (called FunctionDependencies). Each of them holds a set of input and output ids which define the dataflow across function applications. This approach has a couple shortcomings:

  • The composition scheduling in the dispatcher has to do a lot of iterating over all function applications to determine the next runnable functions. This becomes a performance issue when the composition contains many function applications.
  • The structure does not allow for dynamic composition modifications which is something we most likely need if we want to allow for loops and other features.
  • The structure complicates optimizations like scheduling non-blocking invocations immediately (e.g. a function with an each sharding).

We open this issue to collect and discuss further ideas regarding the design / requirements of a future compositions structure...

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestquestionFurther information is requested

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions