Mark the Instance deleted before Admin force-delete cleanup

[chet]

Admin force-delete already commits the Machine ForceDeletion state and current retired IB membership records before it calls UFM or starts direct cleanup. That transaction discovers the associated Instance only after it owns the Machine lock. However, the Instance remains live until cleanup, so a concurrent non-IB configuration update can still commit afterward and change the configuration that force-delete needs to remove.

This Task uses the existing instances.deleted marker as the point after which Instance configuration cannot change. It adds one separate, short Instance transaction after the Machine transaction commits. While holding the Instance row lock, it sets instances.deleted if the row is still live, captures the full cleanup state, and commits. A configuration transaction that commits before the marker is included in that state. If its terminal Instance write runs after the marker commits, the existing deleted IS NULL predicate rejects the write and the configuration transaction rolls back. Machine and Instance locks are never held together.

Scope

• Keep the existing Machine-first transaction and its post-lock Instance discovery and retired IB membership recording unchanged.
• After that transaction commits, lock the discovered Instance in a separate short transaction.
• While holding that lock, set instances.deleted if the row is still live and capture the Instance's current configuration plus both sides of any pending network update in the same transaction.
• Preserve the original deletion timestamp and cleanup behavior on retry.
• Pass the captured state into the existing direct force-delete cleanup.
• Keep UFM, Redfish, DPF, credential, and other external work outside database transactions.
• Reuse the existing stable Instance-address deletion order.

Acceptance criteria

• A configuration transaction committed before the Instance marker is included in the cleanup state.
• If a configuration transaction's terminal Instance write runs after the marker commits, the existing deleted IS NULL predicate rejects the write and the transaction rolls back.
• The instances.deleted marker commits before external cleanup begins.
• No transaction holds both the Machine and Instance locks.
• A failed UFM cleanup leaves the deleted Instance and retired membership records available to an identical retry.
• A crash after either database phase can be completed by retrying the same Admin operation.
• Repeated cleanup remains idempotent and preserves the current Admin force-delete response behavior.
• Focused concurrency coverage proves both commit orderings.

Out of scope

• Changing the existing Machine ForceDeletion, post-lock Instance discovery, or retired IB membership transaction.
• Moving cleanup to a controller, workflow, or new background worker.
• Adding a generic Machine lock helper or another Instance lock helper; reuse the existing find_by_id_for_update.
• Reordering or rereading the final direct-cleanup transaction.
• Broader Machine and DPU lock ordering or state-history hardening, tracked by Audit Machine Lock Ordering for Instance and DPU Writes #5333.
• Tenant prefix overlap admission, routing policy, and UFM API modernization.
• Duplicating the prerequisite allocation, address-ordering, and IB test suites.

Dependencies

The required write guards and cleanup primitives are already merged through #5129, #5136, #5141, #5142, #5287, and #5379. The broader ordering audit in #5333 is a follow-up and does not block this Task.

This Task remains required before tenant prefix reuse can be enabled. Part of #3883.