refactor(server): add backend-neutral IPC process layer

[weicj]

Summary

This stacked PR refactors the native mixed-backend IPC path introduced by Luce-Org#246 into a backend-neutral process layer.

Luce-Org#246 adds the first mixed-backend user: DFlash remote draft placement. This PR keeps that behavior unchanged, but separates the reusable daemon lifecycle from the DFlash-specific feature/proposal protocol so future IPC modes, such as PFlash compression, can share the same process boundary instead of duplicating fork/pipe/work-dir handling.

One reason to do this refactor now is performance work: internal A/B checks did not show the IPC path as an obvious wall-time bottleneck, and some same-backend isolation tests were unexpectedly competitive. A shared IPC process layer gives future latency and transport optimization a single place to improve.

Problem

Before this refactor, the remote draft IPC client owned two different responsibilities:

• generic backend IPC process management: fork/exec, command pipe, binary status stream, ready handshake, scratch directory, and shutdown;
• DFlash draft protocol logic: feature_slice, propose, feature payload files, and hidden-state response handling.

That was acceptable while DFlash draft was the only IPC mode, but it makes the new backend_ipc_daemon harder to extend cleanly.

Architecture Shape

Before:

DFlashDraftIpcClient
  ├─ fork/exec daemon
  ├─ pipe + ready handshake
  ├─ work-dir lifecycle
  └─ DFlash protocol: feature_slice / propose

After:

BackendIpcProcess
  ├─ fork/exec daemon
  ├─ pipe + ready handshake
  └─ work-dir lifecycle

backend_ipc_daemon
  └─ mode: dflash-draft
       └─ DFlash protocol: feature_slice / propose

future modes can reuse the same process layer
  └─ mode: pflash-compress

Changes

1. Generalized the standalone daemon entry point

   • Renamed the primary daemon target to backend_ipc_daemon.
   • The daemon now accepts --backend-ipc-mode=dflash-draft.
   • The daemon entry point now uses the backend-neutral mode form.

2. Added a generic backend IPC process wrapper

   • Added common/backend_ipc.{h,cpp}.
   • Added BackendIpcProcess for daemon lifecycle ownership.
   • Added BackendIpcLaunchConfig for binary path, mode, payload path, mode args, and work dir.
   • Added BackendIpcMode, currently with dflash-draft as the first mode.

3. Kept DFlash draft as a mode on top of the generic process layer

   • DFlashDraftIpcClient now delegates process startup, pipes, ready status, scratch directory, and shutdown to BackendIpcProcess.
   • DFlash-specific payload commands remain in DFlashDraftIpcClient: feature_slice and propose.
   • The DFlash daemon body remains in dflash_draft_ipc_daemon.cpp.

4. Updated docs and build wiring

   • Added backend_ipc.cpp to dflash_common.
   • Updated mixed-backend docs to build/use backend_ipc_daemon.
   • Updated architecture docs to describe the generic backend IPC process layer.

Notes

• The immediate follow-up is to attach PFlash remote compression as another backend IPC mode using the same process layer.
• Checked CUDA sm61 build for dflash_server, backend_ipc_daemon, and test_server_unit; test_server_unit passed. HIP ROCm 6.3.3/gfx906 was checked with fix(hip): map Gemma4 BSA runtime calls Luce-Org/lucebox-hub#258 applied, because current mainline HIP builds are otherwise blocked by the Gemma4 runtime mapping issue fixed there.