The right to left transform is currently not stored (used for stereo differentiable rendering in upstream)