Fix copyImageToCanvas bug and add ReadImageToCanvasOptions

[andremig-bentley]

This PR is to address this issue: https://github.com/iTwin/itwinjs-backlog/issues/1310

This PR has two main goals. First, to address the bug outlined in the above issue. Currently, when there is only a single viewport open and that canvas is copied, such as with copyImageToCanvas, canvas decorations do not show up on the resulting image. This is because the single view is rendered onto an on-screen webgl canvas for higher performance, while the canvas decorations were rendered onto a separate 2d canvas then laid on top. When saving, only the webgl canvas was being copied. This bug is fixed by combining the two canvases before saving them in this single viewport case.

Secondly, the issue above lays out a desire for a way to control whether or not decorations are in the image saved through Viewport.ReadImageToCanvas. This is accomplished through the addition of ReadImageToCanvasOptions. If undefined, or if ReadImageToCanvasOptions.includeCanvasDecorations is false, then the original behavior will persist. If includeCanvasDecorations is set to true, then the saved image will include the canvas decorations.

Also included in this PR is the SaveImageWithCanvasDecorations Tool for Display Test App. This tool gives an example of how to use the new ReadImageToCanvasOptions by allowing you to save an image of the canvas with the decorations either on or off.

[pmconne]

Secondly, the issue above lays out a desire for a way to control whether or not decorations are shown, so that saved images of the canvas can still show no decorations when desired. This is solved with the addition of the canvasDecorations view flag.

Canvas decorations are not the only kinds of decorations. The application should be able to configure what decorations are or not displayed (just as they can control what other content is displayed) in the viewport, separately from capturing an image from the viewport's contents.

ViewFlags are for display styles and they are used to persist data in an iModel, the saved views service, etc. Decorations are mostly a transient viewport thing, not a display style thing, so adding a new view flag is not the correct solution.

[pmconne]

This bug is fixed by combining the two canvases before saving them in this single viewport case.

Arguably, you have introduced a different bug by changing the existing behavior. Do we know if anyone was relying on the existing behavior?

[markschlosseratbentley]

This bug is fixed by combining the two canvases before saving them in this single viewport case.

Arguably, you have introduced a different bug by changing the existing behavior. Do we know if anyone was relying on the existing behavior?

Based on the GitHub issue and discussions: basically the app that raised this issue relies on this bug to be able to disable canvas-decorations-only when reading a viewport. However, this bug-reliance does not work if more than one viewport exists, so it's not a good behavior to rely on anyway. The bug only happens in the single viewport case.

The idea here was to fix the bug so they no longer have to work around the bug by calling internal API and messing around with our direct-to-screen logic from their app. Then, we provide them with a way to truly disable canvas decorations as they desired.

[pmconne]

Then, we provide them with a way to truly disable canvas decorations as they desired.

If they truly only care about canvas decorations (I think there's some confusion about what is a canvas decoration and what is not), a dead-simple solution =

interface ReadImageToCanvasOptions {
  includeCanvasDecorations?: boolean;
}

class Viewport {
  public readImageToCanvas(options?: ReadImageToCanvasOptions): HTMLCanvasElement { ... }
}

If includeCanvasDecorations is undefined then existing behavior is preserved.

I would be reluctant to provide a Viewport.disableCanvasDecorations because canvas decorations include logos (such as Google Maps) that we are required contractually to display. We don't want to make it easy for applications to turn those off indefinitely.

[markschlosseratbentley]

If they truly only care about canvas decorations (I think there's some confusion about what is a canvas decoration and what is not)

@ChrisBAtBentley @jason-crow Can you verify our understanding?

[andremig-bentley]

The changes necessary for the canvasDecoration view flag have been removed.

@jason-crow can you confirm whether the simple solution outlined in this comment above will satisfy what you need? Or, do you think you need something with more control over the decorations akin to the view flags solution (although not the view flags solution as discussed above)? Let me know if you have any other questions.

[jason-crow]

@markschlosseratbentley @pmconne apologies for the delay....

The proposed solution for allowing a prop to specify whether to include decorations:

interface ReadImageToCanvasOptions {
  includeCanvasDecorations?: boolean;
}

class Viewport {
  public readImageToCanvas(options?: ReadImageToCanvasOptions): HTMLCanvasElement { ... }
}

Is a perfect solution for us. We do not actually want to turn off decorations in the viewport, we just want a way to capture a screenshot of the viewport without them using readImageToCanvas. This is preferred for us since a saved view has no mechanism to restore the forms' markers that were possibly active when the screenshot was taken, so ideally the saved view thumbnail would keep them hidden so it doesn't give the user a false expectation about how it will appear when applied.

Since there was a bug that prevented the decorations from showing when a single viewport was active, versus 2, I don't really care if the default is for includeCanvasDecorations to be off or on, since I suppose either way there is a change in behavior; however, the suggested solution of making includeCanvasDecorations off by default probably preserves the most since most consumers likely work with a single viewport.

Thanks for the work on this!

[andremig-bentley]

As of the latest commit, this PR now includes the proposed ReadImageToCanvasOptions solution, as well the SaveImageWithCanvasDecorations Tool in display test app which provides an example of how to use the new options.

[eringram]

core-frontend changes look good to me. Another thing I'd suggest is updating DTA docs for any tool changes

[markschlosseratbentley]

I think this probably warrants a NextVersion.md entry describing the changes to API behavior / flags.

[andremig-bentley]

I think this probably warrants a NextVersion.md entry describing the changes to API behavior / flags.

A NextVersion.md entry has been added.

[markschlosseratbentley]

Can you add a simple test or alter an existing test to use this new flag?

[pmconne]

No. Make the currently inconsistent behavior consistent.

[andremig-bentley]

Do you mean when includeCanvasDecorations is undefined, remove decorations for both single and multi viewport applications in the saved image?

[pmconne]

If I say { includeCanvasDecorations: false }, the canvas decorations should be omitted, regardless of how many viewports I have open. Your @note says they will be included even if I tell you to omit them, if more than one viewport is open.

[pmconne]

Is your note wrong? The comment above it is wrong (it claims any value other than undefined will cause the decorations to be included, but the property is a boolean).

[andremig-bentley]

The comment above it is wrong, should say if true they will be included. The note was correct with the current behavior - basically, if includeCanvasDecorations !== true then current behavior will persist, this meaning if one viewport, no decorations, if multiple viewports, decorations included. You're right that this is inconsistent, working on changing it to:
if includeCanvasDecorations === true, then canvas decorations will be included, regardless of number of viewports. Otherwise, no canvas decorations will be included, regardless of number of viewports.

[pmconne]

My recommendation is to make includeCanvasDecorations a non-optional property. The new options argument to readImageToCanvas is necessarily optional. So if you don't supply any options, you get the current dumb behavior for backwards compatibility. If you supply the options, you must choose whether or not to include canvas decorations. Then you only need to document the dumb "how many viewports" logic in one place (on readImageToCanvas).

[pmconne]

Probably better:

  /** [document dumb behavior here].
   * @deprecated in 5.0 Use the overload accepting a ReadImageToCanvasOptions.
   */
  public readImageToCanvas(): HTMLCanvasElement;
  /** [document sane behavior here] */
  public readImageToCanvas(options: ReadImageToCanvasOptions): HTMLCanvasElement;
  /** @internal */
  public readImageToCanvas(options: ReadImageToCanvasOptions | undefined): HTMLCanvasElement {
    // implementation goes here.
  }

Then you can make includeCanvasDecorations optional and give it a sane default behavior (include the decorations by default, in which case you should probably invert its name to omitCanvasDecorations since undefined is falsy), and we'll be able to remove the dumb behavior eventually.

[andremig-bentley]

This implementation has been added in the latest commit

[markschlosseratbentley]

Is this able to be resolved?

[andremig-bentley]

Still To Do:

• Unit Tests
• Clean up readImageToCanvas declarations/implementations
• Clean up docs/readme/nextversion

[andremig-bentley]

Still To Do:

• Unit Tests
• Clean up readImageToCanvas declarations/implementations
• Clean up docs/readme/nextversion

All pending to-dos have been taken care of.

[ben-polinsky]

Docs look good.

[andremig-bentley]

Latest commit adds two tests to ensure correct behavior when omitCanvasDecorations is undefined.

[markschlosseratbentley]

Thanks for adding the test and NextVersion entry!