UC Berkley and others have just developed a gym-like API to multiple physics and rendering engines: https://arxiv.org/abs/2504.18904 . If somebody wants to go through it, I think there might be a few good points for the interfaces in this repo.