Need ability to have server restart after a failure

[NorthernMan54]

mtplx doctor --json

{
"diagnostics": {
"checks": [
{
"command": null,
"docs_url": "https://ml-explore.github.io/mlx/build/html/install.html",
"expected": "macOS >= 14.0 on Apple Silicon",
"fix": "Upgrade to macOS 14+; MLX does not support older macOS.",
"id": "os.macos_version",
"observed": "26.6",
"severity": "error",
"status": "pass"
},
{
"command": "python3 -c "import platform; print(platform.machine(), platform.processor())"",
"docs_url": "https://ml-explore.github.io/mlx/build/html/install.html",
"expected": "native arm64 Python, not Rosetta",
"fix": "Install/use a native arm64 Python. If needed, reinstall via Homebrew arm64 or uv.",
"id": "python.native_arm64",
"observed": {
"machine": "arm64",
"processor": "arm"
},
"severity": "error",
"status": "pass"
},
{
"command": null,
"docs_url": "https://ml-explore.github.io/mlx/build/html/install.html",
"expected": "Python >= 3.10",
"fix": "Install Python 3.10 or newer.",
"id": "python.version",
"observed": "3.14.5",
"severity": "error",
"status": "pass"
},
{
"command": "python3 -m pip install mlx",
"docs_url": "https://ml-explore.github.io/mlx/build/html/install.html",
"expected": "mlx importable",
"fix": "Install MLX into this same native Python environment.",
"id": "mlx.import",
"observed": {
"default_device": "Device(gpu, 0)",
"get_active_memory": 0,
"get_peak_memory": 0,
"mlx": "0.31.2",
"mlx_lm": "0.31.3"
},
"severity": "error",
"status": "pass"
},
{
"command": null,
"docs_url": null,
"expected": "estimated peak <= 80% of unified memory; 48 GiB+ recommended",
"fix": "Close other heavy apps or use a smaller model/profile.",
"id": "resource.memory",
"observed": {
"estimated_peak_gib": 35.3,
"unified_memory_gib": 24.0
},
"severity": "error",
"status": "fail"
},
{
"command": null,
"docs_url": null,
"expected": "free space for model + temp download + safety headroom",
"fix": "Free disk space or set MTPLX_MODEL_DIR to a larger volume.",
"id": "resource.model_cache_disk",
"observed": {
"cache_dir": "/Users/sgracey/.mtplx/models",
"free_gib": 244.12,
"required_gib": 38.25
},
"severity": "warning",
"status": "pass"
},
{
"command": "mtplx pull Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed",
"docs_url": "https://huggingface.co/Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed",
"expected": "default model available in the HF cache or as the verified local startup model",
"fix": "Download the default model before first run.",
"id": "model.cache",
"observed": {
"hf_cache_exists": false,
"hf_cache_path": "/Users/sgracey/.mtplx/models/Youssofal--Qwen3.6-27B-MTPLX-Optimized-Speed",
"hf_cache_validation": null,
"startup_default_model": null
},
"severity": "warning",
"status": "warn"
},
{
"command": "mtplx pull Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed",
"docs_url": "https://huggingface.co/Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed",
"expected": "Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed",
"fix": "Update DEFAULT_HF_MODEL_ID to the published optimized-speed repo.",
"id": "model.default_repo",
"observed": "Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed",
"severity": "error",
"status": "pass"
},
{
"command": null,
"docs_url": "https://docs.docker.com/desktop/setup/install/mac-install/",
"expected": "Docker Desktop installed for Open WebUI Docker path",
"fix": "Install Docker Desktop if you want the Open WebUI Docker integration.",
"id": "docker.binary",
"observed": "/usr/local/bin/docker",
"severity": "warning",
"status": "pass"
},
{
"command": null,
"docs_url": null,
"expected": "port free before starting mtplx serve, or already a healthy MTPLX server",
"fix": "Use --port 8001 or stop the existing process.",
"id": "port.mtplx_server",
"observed": {
"host": "127.0.0.1",
"open": true,
"port": 8000
},
"severity": "warning",
"status": "warn"
},
{
"command": null,
"docs_url": null,
"expected": "port free before starting Open WebUI, or already an Open WebUI container",
"fix": "Use a different Open WebUI host port or stop the process on 3000.",
"id": "port.openwebui",
"observed": {
"host": "127.0.0.1",
"open": false,
"port": 3000
},
"severity": "warning",
"status": "pass"
},
{
"command": null,
"docs_url": null,
"expected": "ThermalForge or TG Pro available for explicit --max only",
"fix": "Install ThermalForge only if you want opt-in fan boost.",
"id": "thermal.control",
"observed": {
"kind": "thermalforge",
"path": "/Users/sgracey/.mtplx/bin/thermalforge",
"version": {
"command": [
"/Users/sgracey/.mtplx/bin/thermalforge",
"--version"
],
"ok": true,
"returncode": 0,
"stderr": "",
"stdout": "0.1.0"
}
},
"severity": "warning",
"status": "pass"
},
{
"command": null,
"docs_url": null,
"expected": "Low Power Mode off for best sustained decode",
"fix": "Turn off Low Power Mode before benchmarking or serving long responses.",
"id": "power.low_power_mode",
"observed": {
"available": true,
"lowpowermode": "0",
"powermode": null,
"thermal": "Note: No thermal warning level has been recorded\nNote: No performance warning level has been recorded\nNote: No CPU power status has been recorded",
"thermal_ok": true
},
"severity": "warning",
"status": "pass"
},
{
"command": null,
"docs_url": null,
"expected": "no recorded thermal or performance warning",
"fix": "Let the Mac cool down or improve airflow before sustained benchmarks.",
"id": "power.thermal_pressure",
"observed": "Note: No thermal warning level has been recorded\nNote: No performance warning level has been recorded\nNote: No CPU power status has been recorded",
"severity": "warning",
"status": "pass"
}
],
"created_at": "2026-07-24T08:28:34-0400",
"host": {
"cache_dir": "/Users/sgracey/.mtplx/models",
"chip": "Apple M4",
"disk_free_bytes": 262125871104,
"disk_free_gib": 244.12,
"mac_model": "Mac16,10",
"machine": "arm64",
"macos_version": "26.6",
"memory_bytes": 25769803776,
"memory_gib": 24.0,
"platform": "macOS-26.6-arm64-arm-64bit-Mach-O",
"processor": "arm",
"python_executable": "/Users/sgracey/Library/Application Support/MTPLX/runtime-venv/bin/python",
"python_version": "3.14.5",
"system": "Darwin"
},
"overall": "fail",
"resources": {
"default_model_size_bytes": 16430000000,
"estimated_runtime_memory_bytes": 37904836480,
"required_download_free_bytes": 41075000000
},
"schema_version": 1,
"support_matrix": {
"preview_test_targets": [
"M3 Max",
"M4 Max",
"M3 Ultra / Mac Studio",
"M5 Max developer machine"
],
"supported": {
"default_model": "Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed",
"default_profile": "sustained",
"docker": "Docker Desktop current plus previous two macOS major releases",
"macos": ">= 14.0",
"platform": "Apple Silicon arm64 Mac",
"python": "native arm64 Python >= 3.10"
}
}
},
"environment": {
"git_branch": "not a git worktree",
"git_status": "not a git worktree",
"hf_path": "/Users/sgracey/Library/Application Support/MTPLX/runtime-venv/bin/hf",
"mlx": {
"default_device": "Device(gpu, 0)",
"get_active_memory": 0,
"get_peak_memory": 0,
"mlx": "0.31.2",
"mlx_lm": "0.31.3"
},
"platform": "macOS-26.6-arm64-arm-64bit-Mach-O",
"project_root": "/Users/sgracey/Library/Application Support/MTPLX/runtime-venv/bin",
"python_executable": "/Users/sgracey/Library/Application Support/MTPLX/runtime-venv/bin/python",
"python_version": "3.14.5 (main, Jun 2 2026, 22:28:56) [Clang 22.1.3 ]",
"uv_path": null
},
"huggingface": {
"cache_dir": "/Users/sgracey/.mtplx/models",
"cache_exists": true,
"cache_writable": true,
"cached_models": 2,
"disk_free_bytes": 262125871104,
"disk_free_gb": 262.126,
"token_present": false,
"token_source": null
},
"policy": {
"benchmark_exactness_smoke_context": 2048,
"fanmax_counts_for_product_gate": false
},
"thermal_control": {
"available": true,
"clock_anchor_enabled": false,
"clock_anchor_policy": "explicit experimental only; never used for product claims",
"instructions": "Install ThermalForge and ensure the thermalforge CLI is on PATH.",
"selected": {
"kind": "thermalforge",
"path": "/Users/sgracey/.mtplx/bin/thermalforge",
"version": {
"command": [
"/Users/sgracey/.mtplx/bin/thermalforge",
"--version"
],
"ok": true,
"returncode": 0,
"stderr": "",
"stdout": "0.1.0"
}
},
"tools": [
{
"kind": "thermalforge",
"path": "/Users/sgracey/.mtplx/bin/thermalforge",
"version": {
"command": [
"/Users/sgracey/.mtplx/bin/thermalforge",
"--version"
],
"ok": true,
"returncode": 0,
"stderr": "",
"stdout": "0.1.0"
}
}
]
},
"tools": {
"powermetrics": "/usr/bin/powermetrics",
"python": "/Users/sgracey/Library/Application Support/MTPLX/runtime-venv/bin/python",
"smc_atlas": null,
"smc_atlas_exists": false,
"sovereign": null,
"sovereign_exists": false,
"sudo": "/usr/bin/sudo"
}
}

Exact command

None

Model path or repo id

Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed

Chip, RAM, macOS version

M4 Mac mini, 24Gb, Tahoe

[NorthernMan54]

After fixing my earlier tuning issues, found that if the server crashed it does not have the ability to restart.

My instance was crashing on a regular basis and I found this in the logs when crashed

libc++abi: terminating due to uncaught exception of type std::runtime_error: [METAL] Command buffer execution failed: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)

Adding the ability to automatically restart, and update the app to track or be aware of restarts is needed.

Secondly, have the log file actually live on the file system so it is viewable is needed

Third, have the ability to log memory statistics would enable easier tuning.

[davidtai]

Until restart support lands in the app itself, launchd handles this well — it's the macOS-native supervisor, and it also covers your second
point because launchd redirects stdout/stderr to a file you choose. The abort you're hitting can't be recovered in-process (libc++abi calls std::terminate, the process is gone), so an external supervisor is the reliable mechanism regardless. This is the setup I run for a long-lived MTPLX server.

1. Wrapper script, e.g. ~/mtplx/run-server.sh. launchd runs this; the exec at the end replaces the shell with the server, so launchd supervises the real server PID:

#!/bin/zsh
echo "[mtplx] start $(date '+%Y-%m-%d %H:%M:%S')"   # restart marker in the log
export PATH="/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin"
exec <your usual mtplx server command, exactly as you run it today>

Then chmod +x ~/mtplx/run-server.sh and mkdir -p ~/mtplx/logs.

2. LaunchAgent plist at ~/Library/LaunchAgents/com.mtplx.server.plist (absolute paths everywhere — launchd doesn't expand ~):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
  <key>Label</key><string>com.mtplx.server</string>
  <key>ProgramArguments</key>
  <array><string>/Users/YOU/mtplx/run-server.sh</string></array>
  <key>EnvironmentVariables</key>
  <dict>
    <key>HOME</key><string>/Users/YOU</string>
    <key>PATH</key><string>/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
  </dict>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
  <key>ThrottleInterval</key><integer>15</integer>
  <key>StandardOutPath</key><string>/Users/YOU/mtplx/logs/server.log</string>
  <key>StandardErrorPath</key><string>/Users/YOU/mtplx/logs/server.log</string>
</dict></plist>

3. Load it:

launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.mtplx.server.plist

Day-to-day:

launchctl kickstart -k gui/$(id -u)/com.mtplx.server   # force a restart now
launchctl bootout gui/$(id -u)/com.mtplx.server        # stop + unload
launchctl print gui/$(id -u)/com.mtplx.server          # status, PID, last exit reason
tail -f ~/mtplx/logs/server.log

Behavior notes:

• KeepAlive respawns the server on any exit, including the libc++abi abort. ThrottleInterval 15 keeps a crash loop from spinning — launchd waits 15 s between respawns.
• That also means kill/Ctrl-C won't stop it; it just comes back. bootout is how you actually stop it.
• Restart tracking: every respawn re-runs the wrapper, so the timestamped start line gives you a restart history in the log (grep -c '\[mtplx\] start' server.log), and launchctl print shows the last exit status.
• The gui (per-user) domain starts the server at login and runs it inside your login session, which is the domain you want for Metal work. On a headless box, enable auto-login so the session exists after a reboot.

On the crash itself: kIOGPUCommandBufferCallbackErrorOutOfMemory is GPU wired-memory exhaustion, so the daemon only treats the symptom. Crash frequency drops a lot once model weights + KV cache + peak activations sit comfortably under the wired limit — worth re-checking max context / concurrency against physical RAM. For interim memory stats while tuning (your third point), a sidecar loop like

while :; do date; footprint $(pgrep -fn mtplx); sleep 60; done >> ~/mtplx/logs/mem.log

gives you a per-minute footprint trace that lines up with the server log timestamps.

Hope this helps!

[NorthernMan54]

@davidtai Thanks for sharing this, I was going to do something similar.

[youssofal]

two things here. the crash itself: can you attach the full log around the libc++abi abort (the lines above it carry the actual throw)? that's a bug we want dead regardless of restarts. the restart ask: accepted direction, app-managed supervision is queued for the next cycle. davidtai's launchd unit is the right workaround meanwhile.

[PhilipJohnBasile]

I can take the narrow app-managed supervision slice that was accepted for the next cycle. I am working from current main on bounded abnormal-exit retries, backoff/circuit breaking, explicit-stop suppression, restart state/count visibility, and hermetic DaemonSupervisor tests. This does not claim the underlying Metal OOM investigation, persistent log-file work, or memory telemetry. I will keep those separate and open a draft PR with CPU-only proof for the supervision behavior.