Suggestion for fixing a memory leak described in Issues #1325

[ghost]

Problem described in Issues #1325 :

the use of vsg::ref_ptrvsg::Data imageData; as a colorbuffer results in a steadily increasing memory consumption.

Example code used to test:

https://github.com/user-attachments/files/18823235/vsganimation_mod_cpp.txt with 'AnimatedCube.gltf' from the glTF-Sample-Assets. Watching the consumed RAM of the process with top shows it is constantly growing.

My suggestion:

I added _associatedData to track any vsg::ref_ptrvsg::Data objects created from this image.
clearAssociatedData() ensures these references are released when the image is destroyed or reused.

Then I propose to clear associated data when the swapchain is destroyed or recreated to ensure SwapchainImage instances are properly reset between frames.

Made SwapchainImage a subclass of Image to wrap swapchain VkImage handles.

Added std::vector<ref_ptr> _swapchainImages to track swapchain images.
Added resetFrame() to clear associated data each frame.

Modified the constructor to create SwapchainImage instances for each swapchain image and store them in _swapchainImages.

Made modifications to void CollectResourceRequirements::apply(ref_ptr bufferInfo) and
void CollectResourceRequirements::apply(ref_ptr imageInfo):

virtual ~Image()
{
    clearAssociatedData();
    for (auto& vd : _vulkanData)
    {
        vd.release();
    }
}

// defined release() inline to simplify cleanup
void release()
{
    if (image && device)
    {
        vkDestroyImage(*device, image, nullptr);
        image = VK_NULL_HANDLE;
    }
    deviceMemory = nullptr;
    device = nullptr;
}

class VSG_DECLSPEC SwapchainImage : public Inherit<Image, SwapchainImage>
{
public:
    SwapchainImage(VkImage image, Device* device) : Inherit(image, device)
    {
        imageType = VK_IMAGE_TYPE_2D;
        tiling = VK_IMAGE_TILING_OPTIMAL;
        usage = VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT;
        initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
    }
    void clearAssociatedData() { _associatedData.clear(); }
protected:
    virtual ~SwapchainImage() { clearAssociatedData(); for (auto& vd : _vulkanData) { vd.deviceMemory = nullptr; vd.image = VK_NULL_HANDLE; } }
private:
    std::vector<ref_ptr<Data>> _associatedData;
};

void resetFrame()
{
    for (auto& image : _swapchainImages)
    {
        image->clearAssociatedData();
    }
}

void CollectResourceRequirements::apply(ref_ptr<BufferInfo> bufferInfo)
{
    if (bufferInfo && bufferInfo->data && bufferInfo->data->dynamic())
    {
        requirements.dynamicData.bufferInfos.push_back(bufferInfo);
    }
}

void CollectResourceRequirements::apply(ref_ptr<ImageInfo> imageInfo)
{
    if (imageInfo && imageInfo->imageView && imageInfo->imageView->image)
    {
        auto& data = imageInfo->imageView->image->data;
        if (data && data->dynamic())
        {
            if (!dynamic_cast<SwapchainImage*>(imageInfo->imageView->image.get()))
            {
                requirements.dynamicData.imageInfos.push_back(imageInfo);
            }
        }
    }
}

It's not thoroughly tested yet thoughб as I'm having issues with vsgXchange not accepting the AnimatedCube.gltf file (tested only with AnimatedCube.vsgt after conversion). The output confuses me but no memory leak was observed:

Failed to write color buffer to Image_41368.png
info: Image_41368.png
info: supportsBlit = 1
Failed to write color buffer to Image_41369.png
info: Image_41369.png
info: supportsBlit = 1
Failed to write color buffer to Image_41370.png
info: Image_41370.png
info: supportsBlit = 1
(and so on...)

So basically I introduced associateData to tie imageData to the SwapchainImage and resetFrame to clear it per frame.
Will gladly upload or send all the files for testing. Thanks!

[robertosfield]

I haven't had a chance to investigate the reported memory leak yet, once I have I will figure out an appropriate fix.

From a first read through I don't think your suggestion is helpful. There is very specific lifetime resource management in the VSG for C++ and Vulkan objects. The approach you suggested adds a new othogonal scheme that doesn't leverage the existing design. I don't know what level of understanding you have of the existing design and implementation, or the actual reported memory leak.

I won't try to decode your design at this point, it'll be far more productive for me to look specifically at the reported memory leak and step through the issue first hand then study where the issue might be - it could be an error in the usage, or it could be part of the core VSG. I won't know till I've investigated. I don't have the time right now to investigate as I have other pressing tasks to complete first.

The vsgTutorial online book goes into the resource management of the VSG so please have a read through of it:

https://vsg-dev.github.io/vsgTutorial/

If you have specific questions about getting Assimp and vsgXchange to work together then please post a separate thread for this.

[ghost]

ok thank you! i learned a lot, looking forward to the upcoming changes, take care