vsgoffscreenshot.cpp Vulkan validation error when syncing extend and capturing together

[dov]

vsgoffscreenshot.cpp works fine for me after my vsg-dev/vsgExamples#336 and I'm able to resize the window and take a screenshot. However, if I do the following change to ScreenshotHandler::apply(vsg::KeyPressEvent&) (in order to always sync the extent before doing a capture):

         if (keyPress.keyBase == 's')
         {
+           do_sync_extent = true;
            do_image_capture = true;
         }

Then the resulting image capture contains an image with constant RGBA=(0,0,0,0). Further if running the program with the -d option, then I get the following Vulkan validation errors:

info: press 's' to save offscreen render to file
info: press 'e' to set offscreen render extents to same as display
info: Seperate CommandGraph with offscreenCommandGraph first, then main CommandGraph second.
info: replaceChild succeeded

info: offscreen render resized to: 1152x879
VUID-VkRenderPassBeginInfo-None-08996(ERROR / SPEC): msgNum: 1495210966 - Validation Error: [ VUID-VkRenderPassBeginInfo-None-08996 ] Object 0: handle = 0x2b3bf510, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x591f1bd6 | vkCmdBeginRenderPass(): pRenderPassBegin->renderArea.extent.width is zero.
The Vulkan spec states: If the pNext chain does not contain VkDeviceGroupRenderPassBeginInfo or its deviceRenderAreaCount member is equal to 0, renderArea.extent.width must be greater than 0 (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-VkRenderPassBeginInfo-None-08996)
    Objects: 1
        [0] 0x2b3bf510, type: 6, name: NULL
info: writing image to file: screenshot.png
info: image saved.

What happened? When running 'e' for extend and 's' for screenshot separately, then there is no problem.

[dov]

I played with it some more and introduced a delay of four frames between the resizing and the image acquisition, and this fixed the problem. Here is the patch that I introduced which solved the problem:

diff --git a/examples/app/vsgoffscreenshot/vsgoffscreenshot.cpp b/examples/app/vsgoffscreenshot/vsgoffscreenshot.cpp
index f3fa1b80..2bda4f52 100644
--- a/examples/app/vsgoffscreenshot/vsgoffscreenshot.cpp
+++ b/examples/app/vsgoffscreenshot/vsgoffscreenshot.cpp
@@ -524,7 +524,7 @@ class ScreenshotHandler : public vsg::Inherit<vsg::Visitor, ScreenshotHandler>
 {
 public:
     bool do_sync_extent = false;
-    bool do_image_capture = false;
+    int do_image_capture = 0;
 
     ScreenshotHandler()
     {
@@ -540,7 +540,9 @@ public:
         }
         if (keyPress.keyBase == 's')
         {
-            do_image_capture = true;
+            vsg::info("Sync extent and snapshot initiated\n");
+            do_sync_extent = true;
+            do_image_capture = 5;
         }
     }
 };
@@ -742,6 +744,7 @@ int main(int argc, char** argv)
             VkExtent2D displayExtent = displayCamera->getRenderArea().extent;
             if (offscreenExtent.width != displayExtent.width || offscreenExtent.height != displayExtent.height)
             {
+                printf("Syncing the extent\n");
                 auto prevCaptureCommands = captureCommands;
 
                 offscreenExtent = displayExtent;
@@ -758,26 +761,31 @@ int main(int argc, char** argv)
                 vsg::info("offscreen render resized to: ", offscreenExtent.width, "x", offscreenExtent.height);
             }
         }
-        else if (screenshotHandler->do_image_capture && !offscreenEnabled)
+        if (screenshotHandler->do_image_capture>1 && !offscreenEnabled)
         {
-            // set offscreen perspective to same as display except for aspect ratio
-            offscreenPerspective->fieldOfViewY = displayPerspective->fieldOfViewY;
-            offscreenPerspective->nearDistance = displayPerspective->nearDistance;
-            offscreenPerspective->farDistance = displayPerspective->farDistance;
-
-            offscreenEnabled = true;
-            offscreenSwitch->setAllChildren(offscreenEnabled);
+            printf("screenshotHandler->do_image_capture = %d\n", screenshotHandler->do_image_capture);
+            if (--screenshotHandler->do_image_capture == 1)
+            {
+                vsg::info("Taking a snapshot.\n");
+                // set offscreen perspective to same as display except for aspect ratio
+                offscreenPerspective->fieldOfViewY = displayPerspective->fieldOfViewY;
+                offscreenPerspective->nearDistance = displayPerspective->nearDistance;
+                offscreenPerspective->farDistance = displayPerspective->farDistance;
+    
+                offscreenEnabled = true;
+                offscreenSwitch->setAllChildren(offscreenEnabled);
+            }
         }
 
         viewer->update();
         viewer->recordAndSubmit();
         viewer->present();
 
-        if (screenshotHandler->do_image_capture && offscreenEnabled)
+        if (screenshotHandler->do_image_capture==1 && offscreenEnabled)
         {
             // offscreen image was rendered to framebuffer. Disable offscreen
             // rendering and save the capturedImage out to a file.
-            screenshotHandler->do_image_capture = false;
+            screenshotHandler->do_image_capture = 0;
             offscreenEnabled = false;
             offscreenSwitch->setAllChildren(offscreenEnabled);
             saveImage(captureFilename, viewer, device, captureImage, options);

This works around the problem, but I'd be happy to understand a bit better what is happening, and get a less heuristic solution.

[dov]

I tried implementing this when in vsgQt as well, but unfortunately my heuristic work around doesn't work there. In vsgQt I inherited the vsgQt::View and overwrote the update() function to do the functionality of vsgoffscreenshot.png .

However, the image capture and save only work before syncing the extent. After syncing the extent one out of two things happen:

1. The image that I receive back is fully transparent
2. I get the error I described in my initial message

Any hints or debugging ideas would be greatly appreciated.

[dov]

I found the reason for the failures, and it seems to be a bug in RenderGraph::resized(). For whatever reason, resized() sets the RenderGraph::renderArea variable to {{0,0},{0,0}}. Once I reassigned it to the offscreen extent, everything works!