Use DwmGetWindowAttribute for Shell#getLocation to avoid shadow margins

[ShahzaibIbrahim]

GetWindowRect returns the bounding rectangle of the entire window including invisible shadows and glass margins. This leads to Shell#getLocation reporting coordinates that do not match the actual visible frame location. For example, when positioning a dialog close to the left edge of a monitor, the reported X coordinate can be shifted by ~9 pixels compared to the visible window frame. The same discrepancy appears when opening dialogs under a maximized parent shell.

This patch changes Shell#getLocation to use:

DwmGetWindowAttribute(hwnd, DWMWA_EXTENDED_FRAME_BOUNDS, ...)

This API returns the extended frame bounds rectangle in screen space, which corresponds to the true visible window frame and excludes the transparent shadow margins. This produces values that align with what the user sees on screen and avoids subtle layout issues in multi-monitor or mixed-DPI setups.

If DWM is not available (e.g., on older Windows versions or when composition is disabled), the implementation falls back to GetWindowRect, preserving backward compatibility.

Benefits:

• Correct window coordinates near monitor edges.
• No more shadow-margin offset (~9px) in reported locations.
• Consistent behavior across normal, minimized, and maximized shells.

Steps to Reproduce

• Run the runtime workspace with monitor specific scaling turned on.
• Move the window to secondary monitor on the right (could be of any zoom)
• Resize the window to smaller size
• Open the "Open Type" window using Ctrl + Shift + T
• Move the Open Type window towards very left of the screen.
• Close Open Type window
• Maximize the size of main window.
• Open the Open Type dialog again
• You will see the dialog appearing on the left monitor instead of where it was last closed (on the right monitor)

Expected result

The sub-shell or dialog should open where it was last closed. There should be no shadow margins that leads to change of coordinates and resulting in window opening in the other monitor.

• requires Introducing DwmGetWindowAttribute to retrieve better window position #2458

[github-actions]

Test Results

  111 files   -  7    111 suites   - 7   11m 42s ⏱️ + 1m 39s
4 375 tests  - 56  4 358 ✅  - 51  14 💤  - 3  3 ❌  - 2 
  242 runs   - 56    241 ✅  - 48   1 💤  - 3  0 ❌  - 5 

For more details on these failures, see this check.

Results for commit 3811c13. ± Comparison against base commit 75c97fd.

This pull request removes 56 tests.

AllWin32Tests ImageWin32Tests ‑ testDisposeDrawnImageBeforeRequestingTargetForOtherZoom
AllWin32Tests ImageWin32Tests ‑ testDrawImageAtDifferentZooms(boolean)[1] true
AllWin32Tests ImageWin32Tests ‑ testDrawImageAtDifferentZooms(boolean)[2] false
AllWin32Tests ImageWin32Tests ‑ testImageDataForDifferentFractionalZoomsShouldBeDifferent
AllWin32Tests ImageWin32Tests ‑ testImageShouldHaveDimesionAsPerZoomLevel
AllWin32Tests ImageWin32Tests ‑ testRetrieveImageDataAtDifferentZooms(boolean)[1] true
AllWin32Tests ImageWin32Tests ‑ testRetrieveImageDataAtDifferentZooms(boolean)[2] false
AllWin32Tests ImageWin32Tests ‑ test_getImageData_fromCopiedImage
AllWin32Tests ImageWin32Tests ‑ test_getImageData_fromImageForImageDataFromImage
AllWin32Tests TestTreeColumn ‑ test_ColumnOrder
…

♻️ This comment has been updated with latest results.

[HeikoKlare]

The change looks good when using monitor-specific UI scaling and significantly improves UX in that case: shells properly open on the correct monitor and not on the monitor to the left if the parent is maximized. The given reproducer is a good way to test it.

For testing I have added some logging of the new and old values to compare where one can see that the x values are off by some pixels:

old: 140, 127; new: 147, 127 old: 88, 67; new: 95, 67 old: -8, -8; new: 0, 0 old: -8, -8; new: 0, 0 old: 44, 52; new: 51, 52 old: 44, 52; new: 51, 52 old: 1501, 1292; new: 1501, 1292 old: 1501, 1292; new: 1501, 1292 old: 37, 52; new: 44, 52 old: 34, 52; new: 41, 52 old: 31, 52; new: 38, 52 old: 28, 52; new: 35, 52 old: 24, 52; new: 31, 52 old: 22, 52; new: 29, 52 old: 18, 53; new: 25, 53 old: 15, 54; new: 22, 54 old: 13, 54; new: 20, 54 old: 10, 54; new: 17, 54 old: 7, 54; new: 14, 54 old: 4, 55; new: 11, 55 old: 3, 55; new: 10, 55 old: 2, 55; new: 9, 55 old: 1, 55; new: 8, 55 old: -1, 55; new: 6, 55 old: -2, 55; new: 5, 55 old: -3, 55; new: 4, 55 old: -4, 55; new: 3, 55 old: -5, 55; new: 2, 55 old: -7, 55; new: 0, 55 old: -8, 54; new: -1, 54

However, the change is incompatible with monitor-specific scaling being disabled. In that case, the offset between the two values is incorporated when reopening a shell, leading it to slightly move to the right on every reopen. See this capture:

Some interesting information on the issue in general can also be found here: https://stackoverflow.com/questions/34139450/getwindowrect-returns-a-size-including-invisible-borders

[ShahzaibIbrahim]

However, the change is incompatible with monitor-specific scaling being disabled. In that case, the offset between the two values is incorporated when reopening a shell, leading it to slightly move to the right on every reopen. See this capture:

I tested this and figured out it happens even with monitor-specific scaling enabled. The stored coordinates when windows is closed and when it's open are identical. However the move event is occurred after opening the shell and when the getLocation is called again and from OS DwmGetWindowAttribute location is requested for the handle, it returns the +7px which I don't understand why since the position hasn't changed. I am unable to figure out this issue and currently I don't have any solution to prevent this right shifting of the shell when it's reopened. @HeikoKlare

[HeikoKlare]

I tested this and figured out it happens even with monitor-specific scaling enabled.

That's interesting. I thought I have tested it and found that to work properly (as the windows use a different logical positioning when running in different DPI awareness), but maybe I did not analyze properly.

What about looking for solutions others may have already found the problem? This seems to be a general "issue" with Win32 API, so many consumers will have already run into the problem (like in the linked Stack Overflow post), so there is a chance to find something in the web (either with or without AI assistance ;-) ).