Major perf regression in `d00240d`

[aevyrie]

d00240d introduces a major performance regression causing all (?) entities in the scene to oscillate between being marked visible and not visible. This in turn causes a bimodal performance behavior in the visibility systems, as well as in rendering systems that are now seeing tons of spurious change detection.

This was fixed inadvertently in #22226 by changing the approach and fixing the pathological performance case in the process.

Opening this issue in case the aforementioned PR is not merged.

Details

Before the regression

[crates/bevy_camera/src/visibility/mod.rs:659:5] previous_visible_entities.len() = 0
[crates/bevy_camera/src/visibility/mod.rs:659:5] previous_visible_entities.len() = 0
[crates/bevy_camera/src/visibility/mod.rs:659:5] previous_visible_entities.len() = 0
[crates/bevy_camera/src/visibility/mod.rs:659:5] previous_visible_entities.len() = 0
[crates/bevy_camera/src/visibility/mod.rs:659:5] previous_visible_entities.len() = 0

After the regression

[crates/bevy_camera/src/visibility/mod.rs:700:5] previous_visible_entities.len() = 91146
[crates/bevy_camera/src/visibility/mod.rs:700:5] previous_visible_entities.len() = 0
[crates/bevy_camera/src/visibility/mod.rs:700:5] previous_visible_entities.len() = 91146
[crates/bevy_camera/src/visibility/mod.rs:700:5] previous_visible_entities.len() = 0
[crates/bevy_camera/src/visibility/mod.rs:700:5] previous_visible_entities.len() = 91146

impact on overall frametime between 0.17.3 and the commit in question (thanks to @DGriffin91)

Every other frame is cycling between about 5ms and 9ms.