When calling Collider::set_position or Collider::set_position_wrt_parent it adds the ColliderChanges::PARENT flag to the collider. This causes the step_simulation to do a lot of extra work handling those changes. We call these methods when detecting a change in the Transform of a collider in apply_collider_user_changes, but don't verify that the transform has actually changed based on what is currently set in rapier. This leads to a lot of useless work being done in step_simulation that can lead to huge slowdowns.
This is a very easy fix, and can be done by verifying the value we are setting is different from what is there before setting it.