MGS driven SP components left in invalid state should have a way to recover from failed updates

[karencfv]

Specifically in the case of the RoT, it could be left in an un-updateable state. In the case of there being a bad signature check on the alternate RoT image, either the pending-persistent or just the persistent boot preference will need to be set to the good image before proceeding.

We could add a variant to UpdateAttemptStatus called RestoringComponent (or similar) and have apply_update set the status to this new status if the component needs to be set to a different state before an update. This could happen after a precheck.

Needs oxidecomputer/hubris#2050 to fully work

To be able to differentiate whether an RoT has a mismatch with the active version and persistent boot preference or transient/pending boot preference are not empty due to a failed update or an ongoing update we'll need oxidecomputer/hubris#2066 which will be available soon.

[davepacheco]

It seems a little hard to reason about what to do in this situation because it's hard to imagine how we got into this situation without a bug in the planner or executor.

[karencfv]

There's additional context in this comment thread.

Also, it's worth noting that a non-empty transient_boot_preference/pending_persistent_boot_preference or a mismatch between the active slot and persistent_boot_ preference doesn't necessarily mean there is an ongoing update. We need a way to handle the case where it's a failure. The device needs to be reset before proceeding.

There are other ways a device update may fail, @lzrd has more context on those.