Fix inaccurate game filter sizes

[ACrazyTown]

Should close #2258!

In short, Flixel used to add the cameras' OpenFL Sprites to the FlxGame instance along with the mouse, debugger, sound tray, etc. Because of this, game filters did not report accurate sizes because they included things outside of the game view.

To resolve this, I added a _cameraContainer Sprite to FlxGame which exclusively holds all of the cameras' Sprites. Game filters are now applied to this container and will no longer affect the debugger, mouse and other OpenFL level Sprites in FlxGame. I guess this is technically a breaking change, but I don't think game filters ever worked properly to begin with.

I tried splitting up each change in a different commit to hopefully make the reviewing process easier

[Geokureli]

What causes this to be a breaking change?

[ACrazyTown]

What causes this to be a breaking change?

What I was trying to say is that game filters used to apply to the whole screen (debugger, mouse, etc.). With this, they'll just apply to all the cameras. This is technically a difference with how they work (breaking change?).

However in this case I think it can slide through. I don't think game filters ever worked properly and this fixes them and also makes them work more like intended (by leaving the debugger and other OpenFL sprites unaffected)

Let me know what you think

[Geokureli]

What I was trying to say is that game filters used to apply to the whole screen (debugger, mouse, etc.). With this, they'll just apply to all the cameras. This is technically a difference with how they work (breaking change?).

I'm also unsure as to whether this is "breaking", but I do think we should continue to apply the filters to the mouse. I know I personally rely on this behavior for pixel-perfect behavior.

However in this case I think it can slide through. I don't think game filters ever worked properly and this fixes them and also makes them work more like intended (by leaving the debugger and other OpenFL sprites unaffected)

Saying it didn't "ever work properly" is a bit misleading, most of the time it worked entirely as expected, and in a few small cases it behaved weird if you needed the size to be exact.

Getting back to the original problem:

In short, Flixel used to add the cameras' OpenFL Sprites to the FlxGame instance along with the mouse, debugger, sound tray, etc. Because of this, game filters did not report accurate sizes because they included things outside of the game view.

Is there a way to force the width and height of the game? is that what openfl/openfl#2797 is trying to do?

I tried adding this to the sample in #2258 and it seems to work.

FlxG.game.scrollRect = new openfl.geom.Rectangle(0, 0, FlxG.stage.stageWidth, FlxG.stage.stageHeight);

I'm not sure whether there is a performance hit to doing this, though

[ACrazyTown]

I'm also unsure as to whether this is "breaking", but I do think we should continue to apply the filters to the mouse. I know I personally rely on this behavior for pixel-perfect behavior.

Fair. We could probably rename _camerasContainer into something more generic, add the mouse sprite to that, and restore the old camera adding logic to add cameras behind the mouse. Alternatively we could split things up into more containers, but I think it's better to avoid nesting hell.

Saying it didn't "ever work properly" is a bit misleading, most of the time it worked entirely as expected, and in a few small cases it behaved weird if you needed the size to be exact.

Sorry, I was going off of personal experiences with them. They didn't work right for me most of the times that I've used them so much so that I've kinda been avoiding them completely.

Is there a way to force the width and height of the game? is that what openfl/openfl#2797 is trying to do?

I tried adding this to the sample in #2258 and it seems to work.

FlxG.game.scrollRect = new openfl.geom.Rectangle(0, 0, FlxG.stage.stageWidth, FlxG.stage.stageHeight);

I'm not sure whether there is a performance hit to doing this, though

The pull request isn't doing that. In short, OpenFL tends to cache filters into a texture to avoid needlessly re-rendering them. Right now there's an issue where that cache texture won't be updated when a sprite is scaled, which lead to #3463.

Not sure about forcing the size, but using the scroll rect might be the right move. Internally it boils down to OpenGL's scissor function, which seems to have a minimal performance hit, if any

[Geokureli]

I tested the scrollRect fix on the FlxBunnyMark demo. I didn't notice any difference, but not sure if more FlxSprites would ever impact this. If you have any ideas of how I should test this, let me know, otherwise I'll make a new PR with that fix, soon.

[ACrazyTown]

BTW, will you also include the changes from this (so that the debugger doesn't get affected by filters) or are you leaving the game filters as-is and just fixing the issue?

[Geokureli]

I'm not against disabling debugger sprites from game filters, but that's a separate discussion, and for now, the mouse and sound tray should be affected by the game. I'm also not against having a cameraContainer that is a child of the game

It would be nice if the dev could have some control over this. Maybe we can have the following display heirarchy:

- FlxGame
    - debugger
    - virtualPad container
    - filtered container
        - mouse
        - sound tray
        - camera container
            - camera 1
            - camera 2

With this, devs can move them around as they please.

[ACrazyTown]

I'm going to close this for now, and revisit sorting things into different containers sometime after your fix