"Touching color (white)" behaves inconsistently

[adroitwhiz]

Expected Behavior

"touching color (white)" should always return true over a white backdrop.

Actual Behavior

On the CPU path, "touching color (white)" only returns true when the sprite calling it is within the bounds of another drawable. On the GPU path, it never returns true.

Steps to Reproduce

See this project.

[adroitwhiz]

Can you post the test project as a .zip? I created a simple test project in s2online, and it does return true there:

[fsih]

My mistake, the script wasn't running :p

[adroitwhiz]

The plot thickens:
transparent-background.sb2.zip

Note what happens when the cat touches the edges of the stage.

[fsih]

oh no

[fsih]

So in conclusion:

• Transparent background doesn't count as touching white in Scratch 3 or Scratch 2
• Except transparent in the bounding box of another color in the background does count as white in Scratch 3 but not Scratch 2
• And sprites touching past the edges of the stage count as touching white in Scratch 2 but not Scratch 3

[fsih]

Talked to @cwillisf and we decided that we want to keep the scratch 2 behavior, except that when touching outside of the stage, touching white should not be true. That is

• Transparent background doesn't count as touching white in Scratch 3 or Scratch 2
• White background should count as touching white

[adroitwhiz]

I tried matching that behavior when you first showed what 2.0 did, but got stuck on the case of gradients which fade to transparent:

(this is a .sb2)
transparent gradient backdrop.zip

There's a specific point along the gradient where 2.0 decides it's no longer touching grey, but I have no idea why, because it appears the 2.0 "touching color" code completely ignores alpha.

Perhaps you or @cwillisf might be able to shed some light on how the Flash BitmapData class behaves.

Adobe says it's premultiplied, but if that's the case, there should be erroneous "touching color" results from rounding errors when un-premultiplying the colors, which I don't see.

The BitmapData.threshold function is used to test color values, but it looks like the mask is set to completely ignore alpha values.

The "color () is touching color()" block does mask out the alpha value by 0xF0 instead of ignoring it entirely, but that doesn't explain why the "sprite is touching color ()" block behaves like it does.

EDIT: I've created a branch which includes all my commits that fix "touching color"-related code (#479, #489, and #515). Might help with debugging this.

[fsih]

The gradient background touching color issue feels like a very specific edge case, and we don't have examples of 2.0 projects that depend on the behavior that we've seen yet. Given that, I think for 3.0 we can move ahead with anything that feels right. I think it would be fine to have touching gray be true throughout the gradient. Thoughts?

[adroitwhiz]

Unfortunately, that won't work because while getImageData doesn't return premultiplied values, it does return values that were stored premultiplied and then divided by alpha, resulting in a severe loss of precision as transparency increases. Near the gradient's transparent end, it would take on increasingly divergent shades of grey.

[adroitwhiz]

@cwillisf @fsih

Mystery solved! Scratch 2.0 doesn't count transparent backgrounds as "white" because, as can be seen in this line, it instead counts them as this color (0xF0F080):

You can test this for yourself with this test project (rename extension to .sb2).

Is this worth emulating?

[adroitwhiz]

Also, since colorMatches only tests the top 5 bits (at most) of each color, it appears colors with an alpha of 248 or greater don't suffer from enough precision loss to affect the result.

You can test this with the following code:

(() => {
let threshold = 256;
while (threshold > 0) {
    for (let i = 0; i < 256; i++) {
        let unrounded = Math.round(i * (threshold/256)) / (threshold/256);
        if (((Math.round(unrounded) ^ i) & 0b11111000) !== 0) return threshold + 1;
    }

  threshold--;
}
})()

This means that, if you don't want to emulate the weird yellow color behavior, "touching color" can instead consider colors with an alpha < 248 to be too transparent to test.

[fsih]

I'm dying inside

[fsih]

Since I don't think we should keep the ship-it-yellow behavior, let's decide that touching white should be true when touching the background color. I think that's the most reasonable thing for a beginner to expect, and anyone who is relying on the current behavior is probably really confused by what we're doing

[fsih]

Side note: we should search for how many projects are using touching color F0F080 in jupyter