Vips images rendering offset vs ImageMagick

[fitnycdigitalinitiatives]

Hello,

I've run into a funny issue I wonder if you might have some insight into. We were recently processing a new batch of images when I noticed some of tiles rendered in OpenSeadragon viewer were misaligned. You can see at the bottom of this image how the border is offset.

Once you begin to zoom in, it realigns. I did some troubleshooting tested created the same image using ImageMagick rather than VIPS. In that case the image align correctly initially:

Do you think there might be a bug in VIPs or is this possibly an issue with serverless-iiif?

I will send you the images directly.

Thanks,

Joe Anderson

[mbklein]

That's an interesting find. I'd like to look into it further. Would you be able to share both the ImageMagick-derived TIFF and the VIPS-derived TIFF? You can either attach them to this issue, or if you prefer, you can send them to my gmail address (same username as github).

[fitnycdigitalinitiatives]

Just sent them to you.

[mbklein]

What is the size of the div you're embedding OSD in? I'm seeing some artifactual differences in my rendering, but the closer I can get to the actual OSD options/sizes/framing you're using the better.

Ignore that question. I have more data. Following up soon.

[mbklein]

The main issue seems to be that ImageMagick isn't generating a proper pyramidal TIFF – it's creating a single frame image that the IIIF processor is slicing up as needed. There's not a huge performance hit since the image isn't that big, but it's also smoothing out some rounding issues. I'll keep digging and see if I can be more specific.

[fitnycdigitalinitiatives]

Hmm, that's interesting. And it makes sense because this batch of images were initially digitized back in the day at a lower resolution (max 4200 px) than what we do now, which is probably why I noticed the issue. Because at a lower resolution the rounding makes a bigger difference. But odd that ImageMagick isn't actually making a pyramid!

Additional thing I noticed: I upgraded our OpenSeadragon to the latest version, and with that the misalignment is less pronounced but still there on the bottom right:

[mbklein]

What you're seeing with the latest OpenSeadragon is exactly what I've been seeing looking into this, so that's good.

This is a rounding issue brought on by tiling math. Below is the same image, with red borders drawn at the edges of the IIIF tiles OpenSeadragon is requesting.

For efficiency, iiif-processor figures out the scaling factor of the image first, figures out which page within the pyramid is the next highest resolution, and uses that as its starting point. In this case, we're right on the cusp – the tiles on the left side are working with a scale target of 756x1050, which is exactly the size of pyramid page 2. The scaling calculation for the tiles on the right is rounding the scale target to 757x1051, which is one pixel too large in either direction to use page 2, so those tiles are being generated based on page 1. Since each request is fully self-contained, they can't coordinate and use the same page.

I'm going to have to figure out a solution that won't cause other cascading rounding errors. It's possible that adding a couple pixels' worth of fudge factor to the paging decision will fix it without any noticeable degradation.

[fitnycdigitalinitiatives]

That's quite interesting. So would you essentially add a buffer that says that if it's within like say 10px of the target size, just jump down to the larger page?

[mbklein]

I think a 2px buffer would suffice, because this particular rounding error should only happen 1 pixel on either side of a page size boundary, as in the example above.

[mbklein]

As for the ImageMagick issue, I have learned in the course of researching this issue that ImageMagick does not automatically create all the downsampled levels when creating a pyramidal tiff as I previously thought it did. I'll be updating the docs very soon, and also adding instructions for GraphicsMagick (which uses a slightly different command).

[fitnycdigitalinitiatives]

Great, let me know when the update is merged and I can test it out.

Thanks for your help. It's funny that ImageMagick isn't actually making the ptiff's because when I was initially debugging and saw the ImageMagick image didn't have the misalignment, I had this sinking feeling that I was going to have to rederive all our images with ImageMagick instead of Vips.

[mbklein]

@fitnycdigitalinitiatives I've just released v5.1.3 which includes the pixel buffer fix for this issue. Try it out and let me know how it goes.

[fitnycdigitalinitiatives]

Thanks, I will spin this up tomorrow

[fitnycdigitalinitiatives]

Hello,

I deployed the latest version and I'm not really having any luck so far. I've tried running as default and then tried setting pageThreshold in the environmental variables and doesn't seem to change anything (setting it at 2, 5, 10, 50, 200) and the image keeps showing like below (I've successfully got debugBorder to work).

[mbklein]

The number of pixels used for rounding is configurable (but not exposed by the installer) via an environment variable called pageThreshold. The default is 1. Would you mind bumping it up to 2 or 3 and see if that helps?

[fitnycdigitalinitiatives]

Yes, that's what I was configuring. Here are my environmental variables:

I've changed pageThreshold to all sort of variables, but it doesn't seem to be affecting the misalignment on the bottom.

[mbklein]

This is confusing. Would you mind gathering some information for me to look at?

1. Add DEBUG=iiif-processor:transform to the function's environment variables
2. Refresh the page with the misaligned image
3. Grab the resulting log stream and attach it here (or email it to me if you prefer)

[fitnycdigitalinitiatives]

Ok, I just sent you an email with the logs.

[mbklein]

Sorry, I missed your earlier comment about having tried the pageThreshold.

I assume this is still the result you're seeing on initial load, before you zoom at all. I think I've figured out what's happening here, but I'm having trouble reproducing the exact same effect. Mine is doing this, which is still a misalignment (see the lower right hand tile), but of a different type.

What are the dimensions of the OpenSeadragon div? Even better – is there a URL where I can see what you're seeing, so I can reproduce as much of the same context (size, etc.) in my development environment?

[fitnycdigitalinitiatives]

Here's a link: https://fitnycdigitalinitiatives.github.io/iiif-pixel-debug/

It's definitely more pronounced on my end. Let me know if you see it at the link.

[mbklein]

Perfect, thanks.