[Feature] GPU Direct Storage(GDS) #301
Conversation
ucm/store/device/cuda/cuda_device.cc
Outdated
| CUfileHandle_t cuFileHandle = nullptr; | ||
| auto status = CuFileHandleRecorder::Instance().Get(path, cuFileHandle, | ||
| [&path](CUfileHandle_t& handle, int& fd) -> Status { | ||
| return CreateCuFileHandle(path, O_WRONLY | O_CREAT | O_DIRECT, handle, fd); |
There was a problem hiding this comment.
O_CREAT is unnecessary in D2SSync.The file is already created at the Python level before the dump occurs.
| if (!success) { this->failureSet_->Insert(shard.owner); } | ||
| if (!shard.done) { return; } | ||
| if (device) { | ||
| if (device->Synchronized().Failure()) { this->failureSet_->Insert(shard.owner); } |
There was a problem hiding this comment.
Why Sync GDS need Synchronize() here
There was a problem hiding this comment.
You're right — the current S2DSync/D2SSync operations are already synchronous, so there are no pending operations that need to be waited for.
ucm/store/task/task_queue.h
Outdated
| virtual ~TaskQueue() = default; | ||
| virtual void Push(std::list<Task::Shard>& shards) noexcept = 0; | ||
| virtual Status Setup(const int32_t deviceId, const size_t bufferSize, const size_t bufferNumber, | ||
| class TaskSet* failureSet, const class SpaceLayout* layout, const size_t timeoutMs, bool transferUseDirect) = 0; |
There was a problem hiding this comment.
Can the class keyword be removed here?
There was a problem hiding this comment.
Sure, but the relevant header files need to be included. I’ll make the necessary changes.
ucm/store/device/idevice.h
Outdated
| } | ||
| virtual ~IDevice() = default; | ||
| virtual Status Setup() = 0; | ||
| virtual Status Setup(bool transferUseDirect) = 0; |
There was a problem hiding this comment.
The IDevice base class should not be aware of this config option.
There was a problem hiding this comment.
The device should decide whether to invoke cuFileDriverOpen() during setup based on the transferUseDirect parameter; moreover, cuFileDriverOpen() is CUDA-specific and can not appear in nfsstore.
ucm/store/device/idevice.h
Outdated
| } | ||
| virtual Status D2SSync(const std::string& path, void* address, const size_t length, const size_t file_offset, const size_t dev_offset) { | ||
| return Status::Unsupported(); | ||
| } |
There was a problem hiding this comment.
- Pass a file handle, not the file path, to Device.
- Use lowerCamelCase for parameter names.
- Declare interface methods in the interface class as virtual functions.
There was a problem hiding this comment.
1.Passing a file handle would require using CUfileDescr_t or CUfileHandle_t to obtain the fd inside directstorage_queue.cc, but CUfile is CUDA-specific and can not appear in that file.
2.Received,I will fix it.
3.Received,I will fix it.
ucm/store/task/task_queue.h
Outdated
|
|
||
| #include "task_shard.h" | ||
| #include "task_set.h" | ||
| #include "../nfsstore/cc/domain/space/space_layout.h" |
There was a problem hiding this comment.
StoreTask is base class, should not depend on NfsStore.
There was a problem hiding this comment.
If the Setup function is not declared virtual and does not accept parameters such as SpaceLayout, then the call q->Setup() in trans_manager.h cannot decide whether to invoke cuFileDriverOpen() based on the arguments passed.
3 participants
Purpose
What this PR does / why we need it?
This feature introduces GPU Direct Storage (GDS) support to enable direct data transfer between GPU memory and storage, bypassing CPU memory as an intermediate buffer. This significantly reduces memory bandwidth bottlenecks and improves KV cache loading/offloading performance.
Modifications
Does this PR introduce any user-facing change?
A new parameter, useDirect, has been added to the service-startup command. When set to true, GDS transfer is enabled and the data path is Device Memory <-> Storage (direct). When set to false, the path becomes Device Memory <-> Host Memory <-> Storage, as shown below:
"kv_connector_extra_config": {
"ucm_connector_name": "UcmNfsStore",
"ucm_connector_config": {
"storage_backends": "/home/nfs",
"useDirect" : true
Test
How was this patch tested?
embed without GDS:
embed with GDS:
fetch without GDS:
fetch with GDS:
