diff --git a/sycl/doc/design/CommandGraph.md b/sycl/doc/design/CommandGraph.md new file mode 100644 index 0000000000000..a2005e6e53589 --- /dev/null +++ b/sycl/doc/design/CommandGraph.md @@ -0,0 +1,66 @@ +# Command Graph Extension + +This document describes the implementation design of the +[SYCL Graph Extension](https://github.com/intel/llvm/pull/5626). + +A related presentation can be found +[here](https://www.youtube.com/watch?v=aOTAmyr04rM). + +## Requirements + +An efficient implementation of a lazy command graph execution and its replay +requires extensions to the UR layer. Such an extension is command buffers, +where a command-buffer object represents a series of operations to be enqueued +to the backend device and their dependencies. A single command graph can be +partitioned into more than one command-buffer by the runtime. +We distinguish between backends that support command buffer extensions and +those that do not. Currently command buffer extensions are only supported by +Level Zero. All other backends would fall back to an emulation mode, or not +be reported as supported. + +The emulation mode targets support of functionality only, without potentially +resulting performance improvements, i.e. execution of a closed Level Zero +command list multiple times. + +### Command Buffer extension + +| Function | Description | +| ------------------------- | ------------------------ | +| `piextCommandBufferCreate` | create a command-buffer. | +| `piextCommandBufferRetain` | incrementing reference count of command-buffer. | +| `piextCommandBufferRelease` | decrementing reference count of command-buffer. | +| `piextCommandBufferFinalize` | no more commands can be appended, makes command buffer ready to enqueue on command-queue. | +| `piextCommandBufferNDRangeKernel` | append a kernel execution command to command buffer. | +| `piextEnqueueCommandBuffer` | submit command-buffer to queue for execution | +| `piextCommandBufferMemcpyUSM` | append a USM memcpy command to the command-buffer. | +| `piextCommandBufferMemBufferCopy` | append a mem buffer copy command to the command-buffer. | +| `piextCommandBufferMemBufferCopyRect` | append a rectangular mem buffer copy command to the command-buffer. | + +## Design + +![Basic architecture diagram.](images/SYCL-Graph-Architecture.svg) + +There are two sets of user facing interfaces that can be used to create a +command graph object: +Explicit and Record & Replay API. Within the runtime they share a common +infrastructure. + +## Scheduler integration + +When there are no requirements for accessors in a command graph the scheduler +is bypassed and it is directly enqueued to a command buffer. If +there are requirements, commands need to be enqueued by the scheduler. + +## Memory handling: Buffer and Accessor + +There is no extra support for Graph-specific USM allocations in the current +proposal. Memory operations will be supported subsequently by the current +implementation starting with `memcpy`. + +Buffers and accessors are supported in a command graph. Following restrictions +are required to adapt buffers and their lifetime to a lazy work execution model: + +- Lifetime of a buffer with host data will be extended by copying the underlying +data. +- Host accessor on buffer that are used by a command graph are prohibited. +- Copy-back behavior on destruction of a buffer is prohibited. diff --git a/sycl/doc/design/images/SYCL-Graph-Architecture.svg b/sycl/doc/design/images/SYCL-Graph-Architecture.svg new file mode 100644 index 0000000000000..da0c789c8d560 --- /dev/null +++ b/sycl/doc/design/images/SYCL-Graph-Architecture.svg @@ -0,0 +1 @@ +SYCL Graph ArchitectureCPU, GPU, FPGA, hetero/hybrid/converged architectures …Level ZeroSYCL Graph Extensions APISYCL RuntimeApplicationLegendA BA uses / depends on BSYCL RuntimeImplemented BackendsApplication layerUnified Runtime + Command Buffer ExtensionsCUDA (example)NVIDIA GPUFuture Backend support \ No newline at end of file