Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ui): canvas image error handling #7632

Merged
merged 6 commits into from
Feb 11, 2025

Conversation

psychedelicious
Copy link
Collaborator

@psychedelicious psychedelicious commented Feb 10, 2025

Summary

  • Render different error state for the two failure modes for canvas image rendering:
    1. Failed to retrieve image DTO = "Unable to find image"
    2. Failed to retrieve image itself = "Unable to load image"
  • Fix two issues that result from canvas image rendering failures:
    1. Canvas image render error placeholder never disappears (when the last generation in a sequence is the failure)
    2. Staging area stuck on progress image when canvas image rendering fails as described (when any generation except the last in a sequence is the failure)
  • Add more data to a couple canvas module repr methods
  • Fix a visual flicker, where the next generation's first progress image shows, then we briefly show the previous generation's final output image, then continue showing the next generation's progress images.

Related Issues / Discussions

offline discussion

QA Instructions

To reproduce the issue, apply this diff to main. It makes the 3rd staged image fail to load.

diff --git a/invokeai/frontend/web/src/features/controlLayers/konva/CanvasStagingAreaModule.ts b/invokeai/frontend/web/src/features/controlLayers/konva/CanvasStagingAreaModule.ts
index c100a9d8a3..6bb96f7fee 100644
--- a/invokeai/frontend/web/src/features/controlLayers/konva/CanvasStagingAreaModule.ts
+++ b/invokeai/frontend/web/src/features/controlLayers/konva/CanvasStagingAreaModule.ts
@@ -83,6 +83,12 @@ export class CanvasStagingAreaModule extends CanvasModuleBase {
 
     if (this.selectedImage) {
       const { imageDTO } = this.selectedImage;
+      const badImage = {
+        image_name: 'asdf',
+        width,
+        height,
+      };
+      const image = stagingArea.selectedStagedImageIndex !== 2 ? imageDTOToImageWithDims(imageDTO) : badImage;
 
       if (!this.image) {
         const { image_name } = imageDTO;
@@ -90,11 +96,7 @@ export class CanvasStagingAreaModule extends CanvasModuleBase {
           {
             id: 'staging-area-image',
             type: 'image',
-            image: {
-              image_name: image_name,
-              width,
-              height,
-            },
+            image,
           },
           this
         );
@@ -102,7 +104,7 @@ export class CanvasStagingAreaModule extends CanvasModuleBase {
       }
 
       if (!this.image.isLoading && !this.image.isError) {
-        await this.image.update({ ...this.image.state, image: imageDTOToImageWithDims(imageDTO) }, true);
+        await this.image.update({ ...this.image.state, image }, true);
         this.manager.progressImage.$lastProgressEvent.set(null);
       }
       this.image.konva.group.visible(shouldShowStagedImage);

Testing:

  • Generate 3 images. The 3rd should display a failure placeholder once generation finishes, which won't disappear when you cycle through the staged images.
  • Discard staging area.
  • Generate 4 images. Once all generations finish, we expect that the progress image will display and cycling through the staged images doesn't make it go away.

Now, check out this PR and generate a few images on canvas. Should work fine.

Then, apply this diff, which reproduces the root cause of the issue, same as the previous diff (just modified to work on this PR):

diff --git a/invokeai/frontend/web/src/features/controlLayers/konva/CanvasStagingAreaModule.ts b/invokeai/frontend/web/src/features/controlLayers/konva/CanvasStagingAreaModule.ts
index 973ae61b33..5c6783dbf8 100644
--- a/invokeai/frontend/web/src/features/controlLayers/konva/CanvasStagingAreaModule.ts
+++ b/invokeai/frontend/web/src/features/controlLayers/konva/CanvasStagingAreaModule.ts
@@ -83,7 +83,12 @@ export class CanvasStagingAreaModule extends CanvasModuleBase {
 
     if (this.selectedImage) {
       const { imageDTO } = this.selectedImage;
-      const image = imageDTOToImageWithDims(imageDTO);
+      const badImage = {
+        image_name: 'asdf',
+        width: 512,
+        height: 512,
+      };
+      const image = stagingArea.selectedStagedImageIndex !== 2 ? imageDTOToImageWithDims(imageDTO) : badImage;
 
       /**
        * When the final output image of a generation is received, we should clear that generation's last progress image.

Run 3 generations, discard staging area, run 4 images. Should all work correctly, no stuck placeholders or progress images.

Merge Plan

n/a

Checklist

  • The PR has a short but descriptive title, suitable for a changelog

@github-actions github-actions bot added the frontend PRs that change frontend files label Feb 10, 2025
@psychedelicious psychedelicious merged commit b224cc8 into main Feb 11, 2025
15 checks passed
@psychedelicious psychedelicious deleted the psyche/fix/ui/canvas-image-error-handling branch February 11, 2025 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
frontend PRs that change frontend files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants