Distributed cluster serving across Macs (tensor + pipeline parallel)

[ashhart]

Distributed cluster serving across Macs

Serve one model across two or more Macs from the admin dashboard: pick the
cluster, pick the model, choose tensor or pipeline parallelism, activate.
Planning, staging, memory safety and rollback are handled by the server; the
dashboard exposes one /v1 endpoint when the cluster is ready.

Verified on hardware

• Tensor parallel — Qwen3.6-27B across 2 Macs over Thunderbolt RDMA
  (jaccl): 28.6 tok/s vs 16.1 single-node (1.78x), byte-identical output.
• Pipeline — MiniMax-M3-4bit (225 GB, more than either Mac can serve with
  usable context alone) split across a 128 GB MacBook Pro (workstation role,
  18-layer stage pinned exactly as planned) and a 256 GB Mac Studio: one-click
  activation from the dashboard reached ready on both ranks and passed the
  one-token canary over jaccl. An earlier run served 212 tokens before losing
  its remote rank to an idle ssh control-channel drop; keepalives were added
  since, and a long sustained-serving run is the remaining verification.

What's in here

• Capability-aware planning: a model that mlx-lm cannot split (no
  pipeline()/shard()) is refused at plan time, before staging — not after
  a hundred gigabytes have moved. The catalogue answers "which of my models
  fit this cluster, and at how much context".
• Node roles: a workstation Mac (someone works on it) admits less than a
  headless one, enforced by the memory guard at load, not just planning.
• Memory safety: per-rank admission against the GPU's real ceiling, a
  watchdog sampling during the load, and a rank-0 prefill guard per request.
• MiniMax-M3 support for mlx-lm ranks: registered over the vendored
  mlx-vlm implementation with an assigned-stage pipeline() so a rank
  materialises only the layers the plan gave it.
• Selective staging: a rank stages only the shards its stage needs, with
  disk checks and size verification.
• Liveness: idle heartbeats, peer watchdogs, and clean teardown — a
  vanished Mac becomes a stated failure instead of a hung collective.

Honest caveats (things a reviewer would find anyway)

1. memory_guard_tier is not yet carried per-rank (role is); ranks default
   to balanced.
2. The pre-load even-split guard is deliberately conservative: it refuses an
   uneven plan whose even split would not fit, even for architectures that
   honour the assigned-stage pin. Fails closed; costs some availability on
   workstation splits.
3. SSH keepalives mitigate idle control-channel drops; fully SIGHUP-immune
   remote rank supervision is future work.
4. tests/test_glm_mtp_patch.py has 2 tolerance failures that pre-exist this
   branch (file unchanged from origin/HEAD).
5. MiniMax tests importorskip("mlx_vlm") on runners without it.

Tests

7725 passed, 52 skipped locally (Apple Silicon). Commits are sequenced for
review: models → planning → guards → lifecycle → API → GUI → docs. Happy to
split into stacked PRs if that's easier to review.

[sdiamanEXUS]

Highly interested in this!

[jundot]

Thank you for this impressive PR. Distributed cluster serving has been on my long-term roadmap, and I did not expect such a substantial implementation to arrive this quickly. Since this is a large change that will need an extended testing period, I plan to manage and validate it through a separate development release. I’ll revisit the PR after the 0.5.4 release and share my review then. Thanks again for the excellent contribution.

[ashhart]

No problem at all, if you need me to make any changes, let me know.

What you've done with oMLX is amazing and I'd love to contribute more to the development 🙌🏻

[smithcoin]

I have a feeling that many people (myself included) are looking for something like this given the start/stop nature of exo's development and communication.

[ashhart]

Exo is phenomenal and I did take inspiration from some of their work.

However, every time I tried it with Exo, I was thwart with overcoming issues in terminal and in the end I just gave up as even Claude/Codex could not get it to work.

My goal was to make clustering as easy as clicking a button.

[monroewilliams]

This looks excellent, thanks for doing it!

I've been working on my own version of this for a while, with some different design decisions. Mine's nowhere near PR-ready, but I do have inference basically working across at least two machines for Qwen3.6-27B working using an mlx ring in raw TCP mode.
My focus has been solely on the use case of running larger models than will fit on one machine, and one of the two higher-RAM machines I have is an M1 that doesn't support rdma, so, I'm only using pipeline mode over tcp.
I didn't cover transmitting the model data, I just only attempt to distribute inference to peers that already have a model with a matching model ID available (I'm using hfdownloader manually beforehand to sync models across machines, and having oMLX load them from the huggingface cache).
I also used the exo implementation as reference/inspiration, although I didn't directly copy any of their code (unless the llm I was using did so without my noticing).

Potentially interesting bits of my implementation:

• each oMLX instance that has distributed switched on (in a new section in settings) advertises itself using mDNS/bonjour using an oMLX-specific service type (looking through your notes, it looks like I may have picked the same service name you did. Oops... :D)
• any node with distributed mode enabled that receives an inference request and tries to load a model will potentially make itself the coordinator and set up a ring. There's a free-memory threshold in settings, and it only loads in distributed mode if loading the model would take free memory below that threshold (so it doesn't try to distribute smaller models, only ones that need the extra memory)
• there's a new endpoint on the http interface that the coordinator uses to get peer details, communicate the ring setup parameters to workers, and start the ring. Each worker allocates a new ephemeral port when setting up the ring, and communicates the port number back to the coordinator over the endpoint so it can set up the full ring list and push it back out to all ring members.
• the settings include an optional shared key for the cluster, and if it's present the coordinator and worker prove to each other that they have matching keys using something like HMAC-CHAP
• each worker loads the model on a background thread and then loops on that thread waiting for tensors to arrive over the ring. The cycle is for it to wait for a known-size integer "header" tensor that communicates a tensor shape and other parameters, followed by a recv with the shape gleaned from the header to match the actual tensor. It uses the same codepath for prefill and generation tensors, they just get different tensor shapes, and the final rank's send back to the coordinator is conditional on a flag in the header tensor.
• model sharding gives each node a number of layers proportional to the amount of physical RAM it has
• probably some other things I've forgotten. :)

My experimental branch has a lot more work left to do before it would actually be usable, and a fair bit of the code was authored by Qwen-3.6-27B, so there's likely a lot of slop in there I haven't noticed. I've pushed the squashed branch to my fork as a single commit, if anything in there looks useful feel free to pick ideas out of it and incorporate them. :)

monroewilliams@54561e8

I'm going to pull your PR branch and give it a go. Thanks again for your work on this! :)

[paalolav]

Impressive! I have 2x Mac Studio M2 ultra 64gb and a Mac Studio M2 max 64gb. This I’m going to test!

[smithcoin]

Would it help if I ran this on two 512 gb M3 max studios for testing or should I was for a release candidate?

[ashhart]

@smithcoin I'd love to see what you can run across both your Studios with this, post feedback!

[jundot]

@ashhart Thank you for this, and sorry it took me a while to come back with a proper review. The scope, and the fact that you verified it on real hardware, both stand out, and distributed serving is a feature I have wanted in oMLX for a long time. I read through the whole branch. I agree with the direction and I want to land it, so here is how I would like to manage it rather than asking you to polish everything in one pass.

I am splitting the feedback into three groups.

Before I merge (A). These are the only hard blockers, and they are small:

• Remote code execution on the coordinator (SSH argument injection). The peer hostname reaches ssh unvalidated in transport.py (_run_ssh, _rdma_devices) through the /transports, /link-status, and /peer-health endpoints. A hostname that starts with - is parsed by ssh as an option, so GET /admin/api/cluster/link-status?hosts=-oProxyCommand=touch /tmp/pwned reaches subprocess.run(["ssh", "-oProxyCommand=touch /tmp/pwned", "ibv_devices"]), and ssh runs the ProxyCommand on the coordinator before it ever connects. The launch and deployment paths already guard this with validate_ssh_target, so routing these three endpoints through the same check closes it.
• The key-exchange and Bonjour pairing tokens are signed with a plain SHA-256 of the payload, so anyone can recompute a valid signature. Since a verified token installs an SSH key into authorized_keys and trusts a host key, this needs a keyed MAC or an out-of-band shared secret.
• The SSH keepalive options landed on _strict_ssh_argv, which is only used for the short probe commands. The long-lived rank sessions run through _install_strict_ssh_wrapper, which does not carry those options, so the idle control-channel drop that ended the MiniMax runs is still unguarded. Adding the same three -o lines to the wrapper script should cover it.

Merge to dev, then follow-up PRs (B). The happy path works, and I would rather iterate on these in the open than block the merge on them:

• _config_int's 4096 ceiling is applied to hidden_size, so a model without an explicit head_dim in its config reserves 0 bytes for KV. I reproduced this with Qwen2.5-72B and Llama-13B shapes. The plan passes and the rank dies on the first long prompt.
• The prefill guard raises inside fetch_nearest_cache, which runs after the request has already been shared to the peer ranks, so a rejection stops rank 0 while the others stay in the collective. It also only checks rank 0's slice, so a non-zero rank has no memory guard once it is loaded. I think this one wants a small redesign.
• benchmarks/tp_identity_probe.py imports _make_tensor_parallel_group and _shard_stage, which are not in the branch, so the identity check cannot run as shipped. Restoring it would make the byte-identical result reproducible.
• I will open issues for the rest: orphaned ranks when the coordinator process dies, activation holding the engine-pool lock during a load, and the heartbeat comparing two machines' wall clocks.

Ongoing, with tests (C). The cluster tab strings are not localized yet, the idle polling re-runs peer probes and drops the in-progress plan, and there are staging cleanup and lint items. I am happy to take some of these on alongside you.

For the dev builds I would also like to put distributed inference behind an explicit toggle in Settings > Advanced, so the cluster tab, the cluster routes, and the Bonjour advertising only appear once someone turns it on. That keeps the surface off by default while the feature matures, and it also stops a normal install from advertising itself over mDNS. The toggle is about exposure, not safety, so the A items above still need fixing regardless.

I will send exact repro steps for the other A items so you are fixing rather than hunting. This is a strong first contribution and I would be glad to have you keep building on it.

[jundot]

Repro and fix pointers for the A items, as promised. Line numbers are against the PR head.

1. RCE (SSH argument injection). The three GET handlers split the hosts query and pass it straight to the ssh helpers without validation, unlike the launch and deployment paths:

• cluster_transports (routes.py:1602) -> detect_cluster_transports -> transport.py:_rdma_devices (ssh <host> ibv_devices)
• cluster_link_status (routes.py:1651) -> assess_link
• cluster_peer_health (routes.py:1626) -> check_peers

Fix is to run each host through the same validate_ssh_target the launch path already uses, and return 400 on a bad target:

from .deployment import validate_ssh_target

try:
    host_list = [validate_ssh_target(h.strip()) for h in hosts.split(",") if h.strip()]
except ValueError as exc:
    raise HTTPException(status_code=400, detail=str(exc)) from exc

validate_ssh_target already accepts the optional user@ form, so peer-health can validate the full item before splitting on @. Repro before the fix: GET /admin/api/cluster/link-status?hosts=-oProxyCommand=touch /tmp/pwned writes /tmp/pwned on the coordinator.

2. Forgeable pairing and key-exchange tokens. sign_key_exchange_token / verify_key_exchange_token (ssh_keys.py:190,237) and the pairing token (discovery.py:279,308) compute the signature as hashlib.sha256(payload_json).hexdigest(), with no secret, so verify just recomputes a public hash. Anyone can mint a token that passes. Since a verified token appends a key to authorized_keys and keyscans the host into known_hosts, the check needs to authenticate the sender: either an hmac over the payload with a persistent per-node secret, or drop the signature idea and require a short shared secret entered on both Macs during pairing.

3. Keepalive on the wrong channel. The keepalive options are on _strict_ssh_argv (launch.py:430), which only runs the short probe commands. The long-lived rank sessions use the PATH wrapper _install_strict_ssh_wrapper (launch.py:453), which does not carry them, so the idle drop is still there. Adding the same three options to the wrapper script fixes it:

"-o BatchMode=yes "
"-o ConnectTimeout=5 "
"-o StrictHostKeyChecking=yes "
"-o ServerAliveInterval=15 "
"-o ServerAliveCountMax=4 "
"-o TCPKeepAlive=yes "

test_cluster_launch.py:339 asserts the wrapper has exactly three options, so that assertion updates with it.

---

Not an A item and not blocking the merge, but the KV-zero reservation reproduces in three lines, so here it is:

from omlx.cluster.planner import _kv_bytes_per_token_per_layer as kv
print(kv({"hidden_size": 8192, "num_attention_heads": 64, "num_key_value_heads": 8}))  # 0, should be 4096
print(kv({"hidden_size": 5120, "num_attention_heads": 40, "num_key_value_heads": 40}))  # 0, should be 20480
print(kv({"hidden_size": 8192, "num_attention_heads": 64, "num_key_value_heads": 8, "head_dim": 128}))  # 4096

The cause is the 0 < value <= 4096 bound in _config_int (planner.py:557) being applied to hidden_size. _activation_bytes_per_token already reads hidden_size with a 1_000_000 bound (planner.py:526), so lifting the ceiling for the hidden-size lookups lines the two up.

[ashhart]

Thank you for the detailed review and repros.

I’ve confirmed all three A items and will address them in focused commits with regression tests.
For the token flow, I’ll use a short-lived one-time shared pairing secret and authenticated key exchange rather than an unkeyed digest.

I’ll leave the B/C items and the Advanced toggle for follow-up work against the development branch so this update stays narrowly scoped

[tannerdsilva]

Testing this on two Mac Studio M3 Ultra 256GB. I self-participated the "MLX Distributed" WWDC session earlier this summer, so these machines have already done this successfully in limited capacity.

The most immediate issue I'm finding (from the server output) is that interactive ssh authentication messages are disrupting the setup process.

Messages mostly about "known hosts from different IP addresses".

Will continue tweaking the experiment here and report my results.

Attempting to run Deepseek-V4-Flash-0731 distributed, which I'm already running successfully with oMLX on a single node.

[ashhart]

@tannerdsilva Thanks, this is very useful.

Some setup probes can still inherit OpenSSH's interactive host-key policy when the same Mac is discovered through a different address. I will improve that flow around the paired node identity and verified host key.

If possible, please retain the exact SSH stderr and note whether the target shown is
a. local hostname or an IP address.

That will help verify the fix against your topology.

[smithcoin]

I got this error and it wasn't clear the command to run:

I'll keep playing with it and try to post a comparison. Here was my benchmark running that model on exo:

Update - I am having trouble figuring out how to pair them in the GUI. Here are the diagnostic logs from each machine:

omlx-cluster-diagnostics-2026-08-08T20-35-22.940Z.json

omlx-cluster-diagnostics-2026-08-08T20-34-21.744Z.json

[ashhart]

@smithcoin Thanks for the report, the screenshot was very helpful.

I've pushed an update that makes this setup clearer.

The error means the peer Mac's SSH host key has not been trusted yet. The dashboard now provides an Open SSH setup in oMLX action, a copyable Terminal command, and key-generation instructions if the managed oMLX SSH key does not exist yet.

After pulling the latest PR branch, please retry the pairing flow. If it still fails, sharing fresh cluster diagnostics from both Macs would be very useful.

l've also improved the remote-code check so models requiring trust
_remote_code should now fail before opening large weight files, avoiding the misleading memory spike after a failed load.

[ashhart]

@jundot

Quick status update following your review:

All three A merge blockers are now resolved with regression tests:

• SSH targets are validated before reaching any transport helpers.
• Pairing and key-exchange tokens now use authenticated HMAC signatures with a shared pairing secret.
• The long-lived rank SSH wrapper now carries the keepalive options.

I originally planned to leave the B items for follow-ups, but after reproducing them, each had a contained fix with test coverage, so these are now resolved too:

• Models without an explicit head_dim reserve KV memory correctly.
• Prefill admission is now a collective, rank-agreed decision using each rank’s local memory guard.
• The tensor-parallel identity probe runs using the shipped tensor strategy API.

The Advanced distributed-inference toggle is also implemented and defaults to off. Subsequent tester reports prompted improvements to prompt-free SSH setup, first-time pairing guidance, and fail-fast handling for models requiring remote code.

The relevant test suites are passing. Unless you would like further changes, I propose holding the branch stable now for hardware retesting and your re-review.

I’m happy to take ownership of follow-up issues for orphaned ranks, engine-pool locking, cross-machine heartbeat timing, cluster localization, dashboard polling state, and staging cleanup after this lands in the development branch.

Thanks.

[tannerdsilva]

@ashhart thanks for your hard work on this! Its funny you mention topology - that seems to be the core of my issue for sure.

I am not a python dev but I have extensive experience and am happy to contribute any amount "test effort" or tokens to help the cause alongside you! I will test anything you need, just tell me what you want! And apologies for the delayed response.....

I had trouble with this stage during the WWDC session. I got it working at the time with some basic hacks....that aren't getting me by here. So I am currently stuck with oMLX in a way I didn't get stuck with mlx-lm distributed.

Topology:

I have two hosts: dario-doomer.local and sam-slayer.local. Both are "headless" and identically spec'd. These hosts have TWO shared networks:

1. The traditional gigabit ethernet route (IPv4 and IPv6)
2. The Thunderbolt route (JACCL - yes I enabled that obscure TB feature in recovery mode too)

During WWDC session, I never had any luck with the thunderbolt bridge before configuring it with static networking. With a static private network (10.200.215.x/24 and fdXX:XXXX:XXXX:XXXX/64) on both hosts, I can get them talking over Thunderbolt. At the time with mlx-lm, I believe I had it working through Thunderbolt over ONLY IPv6 specifically. For oMLX I'm not even trying to touch IPv6 for obvious reasons. So I'm trying with IPv4 addresses.

OBSERVATION:

When I am driving from machine dario-doomer, this host will SSH into itself using the hardcoded string 127.0.0.1 (obviously IPv4) but it will SSH into all remote machines by their mDNS names. which may resolve to IPv4 or IPv6 depending on the stack. I observed this with mlx-lm but see it as a particular hazard for oMLX's implementation (mixed stacks and mDNS possibly taking you down the wrong path). You're using mDNS names for SSH and then for other stuff.........with no guarantee what route that takes you on.

OBSERVATION:

As of 0.5.8-beta1 - I can observe that the "auto networking" features do not get me into a working configuration. I notice in the networking section of the UI - both of my hosts have an IP address under 10.1.x.x (if I recall) which is not an address space I currently have routed on these hosts. The addresses dont ping, I'm not sure where oMLX got these values.

I attempted to correct the IP addresses of both of my hosts, to use the STATIC IPv4 assigned to Thunderbolt bridge0.

I thought this would be the winning formula since this got me working on mlx-lm, however, when I do this on oMLX, I get UI warnings about "bridge0 does not support rdma_ctl" which I was surprised to see.

BETA2

As of 0.5.8-beta2 - I see certain regressions in the networking page that will make this impossible for me to test again without another release. Amongst the issues I'm having with beta2, the biggest is that I have an error that says "model is 0.3 GB larger".

TL:DR

• 0.5.8-beta1 had issues with network topology
• 0.5.8-beta2 appears to have other regressions beyond network topology
• If you're offering to lead this PR to completion, I'm offering to match your effort as a tester. I'm happy to help!
• My biggest concerns with networking relate to mDNS, IPv4, IPv6. Between these three terms, it seems like Apple or oMLX (probably both) made too many assumptions and things are not as reliable as they should be.

[tannerdsilva]

@tannerdsilva Thanks, this is very useful.

Some setup probes can still inherit OpenSSH's interactive host-key policy when the same Mac is discovered through a different address. I will improve that flow around the paired node identity and verified host key.

If possible, please retain the exact SSH stderr and note whether the target shown is
a. local hostname or an IP address.

That will help verify the fix against your topology.

To answer this directly: the target causing the error was an mDNS name and NOT an IP address.

I was riding in the back of an Uber at the time and didn't have sufficient capacity to capture the output - apologies.

[smithcoin]

Perhaps unrelated but I did want to bring this to attention (I could open another PR). I pulled HEAD from this PR and attempted to install using OMLX_WITH_CUSTOM_KERNEL=1 pip install -e . and it failed (output below). I am just running it with pip install -e .. I do have the xcode tools installed so I don't know what's happening there.

Obtaining file:///Users/office/omlx
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: mlx==0.32.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.32.0)
Collecting mlx-lm @ git+https://github.com/ml-explore/mlx-lm@ab1806e8f5d6aa035973af194a1b9198ab4754dc (from omlx==0.5.8.dev2)
  Using cached mlx_lm-0.31.3-py3-none-any.whl
Requirement already satisfied: regex in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (2026.7.19)
Collecting mlx-embeddings @ git+https://github.com/Blaizzy/mlx-embeddings@32981fa4e8064ed664b52071789dd18271fe4206 (from omlx==0.5.8.dev2)
  Using cached mlx_embeddings-0.1.0-py2.py3-none-any.whl
Requirement already satisfied: transformers<5.13,>=5.12.1 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (5.12.1)
Requirement already satisfied: mistral-common>=1.10 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (1.11.7)
Requirement already satisfied: tokenizers>=0.19.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.22.2)
Requirement already satisfied: huggingface-hub>=1.19.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (1.27.0)
Requirement already satisfied: numpy<2.4,>=1.24.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (2.3.5)
Requirement already satisfied: tqdm>=4.66.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (4.70.0)
Requirement already satisfied: pyyaml>=6.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (6.0.3)
Requirement already satisfied: itsdangerous>=2.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (2.2.0)
Requirement already satisfied: jinja2>=3.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (3.1.6)
Requirement already satisfied: rich>=13.0.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (15.0.0)
Requirement already satisfied: sentencepiece in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.2.2)
Requirement already satisfied: tiktoken in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.13.0)
Requirement already satisfied: protobuf in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (7.35.1)
Requirement already satisfied: requests>=2.28.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (2.34.2)
Requirement already satisfied: httpx<1,>=0.27.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.28.1)
Requirement already satisfied: socksio>=1.0.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (1.0.0)
Requirement already satisfied: tabulate>=0.9.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.10.0)
Requirement already satisfied: psutil>=5.9.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (7.2.2)
Requirement already satisfied: setproctitle>=1.3.3 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (1.3.7)
Requirement already satisfied: fastapi>=0.108.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.141.1)
Requirement already satisfied: uvicorn>=0.23.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.52.1)
Requirement already satisfied: python-multipart>=0.0.5 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.0.32)
Requirement already satisfied: jsonschema>=4.0.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (4.26.0)
Requirement already satisfied: openai-harmony in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.0.8)
Requirement already satisfied: cohere_melody>=0.9.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (0.13.0)
Collecting mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd (from omlx==0.5.8.dev2)
  Using cached mlx_vlm-0.6.3-py3-none-any.whl
Requirement already satisfied: Pillow>=9.0.0 in ./venv/lib/python3.11/site-packages (from omlx==0.5.8.dev2) (12.3.0)
Collecting dflash-mlx @ git+https://github.com/jundot/dflash-mlx@474f8e1ba95864f5bb0759cd1b9a13c80abc5ce3 (from omlx==0.5.8.dev2)
  Using cached dflash_mlx-0.1.10+omlx.3-py3-none-any.whl
Requirement already satisfied: markitdown==0.1.7 in ./venv/lib/python3.11/site-packages (from markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (0.1.7)
Requirement already satisfied: datasets>=2.19.1 in ./venv/lib/python3.11/site-packages (from mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (5.0.1)
Requirement already satisfied: miniaudio>=1.59 in ./venv/lib/python3.11/site-packages (from mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (1.71)
Requirement already satisfied: llguidance>=1.7.0 in ./venv/lib/python3.11/site-packages (from mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (1.7.6)
Requirement already satisfied: mlx-audio>=0.4.3 in ./venv/lib/python3.11/site-packages (from mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (0.4.6)
Requirement already satisfied: opencv-python>=4.12.0.88 in ./venv/lib/python3.11/site-packages (from mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (5.0.0.93)
Requirement already satisfied: starlette>=1.0.1 in ./venv/lib/python3.11/site-packages (from mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (1.6.0)
Requirement already satisfied: beautifulsoup4 in ./venv/lib/python3.11/site-packages (from markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (4.15.0)
Requirement already satisfied: charset-normalizer in ./venv/lib/python3.11/site-packages (from markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (3.4.9)
Requirement already satisfied: defusedxml in ./venv/lib/python3.11/site-packages (from markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (0.7.1)
Requirement already satisfied: magika~=0.6.1 in ./venv/lib/python3.11/site-packages (from markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (0.6.3)
Requirement already satisfied: markdownify in ./venv/lib/python3.11/site-packages (from markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (1.2.3)
Requirement already satisfied: lxml in ./venv/lib/python3.11/site-packages (from markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (6.1.1)
Requirement already satisfied: mammoth~=1.11.0 in ./venv/lib/python3.11/site-packages (from markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (1.11.0)
Requirement already satisfied: pdfminer-six>=20251230 in ./venv/lib/python3.11/site-packages (from markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (20260107)
Requirement already satisfied: pdfplumber>=0.11.9 in ./venv/lib/python3.11/site-packages (from markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (0.11.10)
Requirement already satisfied: python-pptx in ./venv/lib/python3.11/site-packages (from markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (1.0.2)
Requirement already satisfied: mlx-metal==0.32.0 in ./venv/lib/python3.11/site-packages (from mlx==0.32.0->omlx==0.5.8.dev2) (0.32.0)
Requirement already satisfied: anyio in ./venv/lib/python3.11/site-packages (from httpx<1,>=0.27.0->omlx==0.5.8.dev2) (4.14.2)
Requirement already satisfied: certifi in ./venv/lib/python3.11/site-packages (from httpx<1,>=0.27.0->omlx==0.5.8.dev2) (2026.7.22)
Requirement already satisfied: httpcore==1.* in ./venv/lib/python3.11/site-packages (from httpx<1,>=0.27.0->omlx==0.5.8.dev2) (1.0.9)
Requirement already satisfied: idna in ./venv/lib/python3.11/site-packages (from httpx<1,>=0.27.0->omlx==0.5.8.dev2) (3.18)
Requirement already satisfied: h11>=0.16 in ./venv/lib/python3.11/site-packages (from httpcore==1.*->httpx<1,>=0.27.0->omlx==0.5.8.dev2) (0.16.0)
Requirement already satisfied: click>=8.1.7 in ./venv/lib/python3.11/site-packages (from magika~=0.6.1->markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (8.4.2)
Requirement already satisfied: onnxruntime>=1.17.0 in ./venv/lib/python3.11/site-packages (from magika~=0.6.1->markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (1.28.0)
Requirement already satisfied: python-dotenv>=1.0.1 in ./venv/lib/python3.11/site-packages (from magika~=0.6.1->markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (1.2.2)
Requirement already satisfied: cobble<0.2,>=0.1.3 in ./venv/lib/python3.11/site-packages (from mammoth~=1.11.0->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (0.1.4)
Requirement already satisfied: packaging>=20.0 in ./venv/lib/python3.11/site-packages (from transformers<5.13,>=5.12.1->omlx==0.5.8.dev2) (26.3)
Requirement already satisfied: typer in ./venv/lib/python3.11/site-packages (from transformers<5.13,>=5.12.1->omlx==0.5.8.dev2) (0.27.1)
Requirement already satisfied: safetensors>=0.4.3 in ./venv/lib/python3.11/site-packages (from transformers<5.13,>=5.12.1->omlx==0.5.8.dev2) (0.8.0)
Requirement already satisfied: filelock>=3.10.0 in ./venv/lib/python3.11/site-packages (from huggingface-hub>=1.19.0->omlx==0.5.8.dev2) (3.32.2)
Requirement already satisfied: fsspec>=2023.5.0 in ./venv/lib/python3.11/site-packages (from huggingface-hub>=1.19.0->omlx==0.5.8.dev2) (2026.6.0)
Requirement already satisfied: hf-xet<2.0.0,>=1.5.2 in ./venv/lib/python3.11/site-packages (from huggingface-hub>=1.19.0->omlx==0.5.8.dev2) (1.6.0)
Requirement already satisfied: typing-extensions>=4.1.0 in ./venv/lib/python3.11/site-packages (from huggingface-hub>=1.19.0->omlx==0.5.8.dev2) (4.16.0)
Requirement already satisfied: pyarrow>=21.0.0 in ./venv/lib/python3.11/site-packages (from datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (25.0.0)
Requirement already satisfied: dill<0.4.2,>=0.3.0 in ./venv/lib/python3.11/site-packages (from datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (0.4.1)
Requirement already satisfied: pandas in ./venv/lib/python3.11/site-packages (from datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (3.0.5)
Requirement already satisfied: xxhash in ./venv/lib/python3.11/site-packages (from datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (3.8.1)
Requirement already satisfied: multiprocess<0.70.20 in ./venv/lib/python3.11/site-packages (from datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (0.70.19)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in ./venv/lib/python3.11/site-packages (from fsspec[http]<=2026.6.0,>=2023.1.0->datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (3.14.3)
Requirement already satisfied: aiohappyeyeballs>=2.5.0 in ./venv/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (2.7.1)
Requirement already satisfied: aiosignal>=1.4.0 in ./venv/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (1.4.0)
Requirement already satisfied: attrs>=17.3.0 in ./venv/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (26.1.0)
Requirement already satisfied: frozenlist>=1.1.1 in ./venv/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (1.8.0)
Requirement already satisfied: multidict<7.0,>=4.5 in ./venv/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (6.7.1)
Requirement already satisfied: propcache>=0.2.0 in ./venv/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (0.5.2)
Requirement already satisfied: yarl<2.0,>=1.17.0 in ./venv/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (1.24.5)
Requirement already satisfied: pydantic>=2.9.0 in ./venv/lib/python3.11/site-packages (from fastapi>=0.108.0->omlx==0.5.8.dev2) (2.13.4)
Requirement already satisfied: typing-inspection>=0.4.2 in ./venv/lib/python3.11/site-packages (from fastapi>=0.108.0->omlx==0.5.8.dev2) (0.4.2)
Requirement already satisfied: annotated-doc>=0.0.2 in ./venv/lib/python3.11/site-packages (from fastapi>=0.108.0->omlx==0.5.8.dev2) (0.0.5)
Requirement already satisfied: MarkupSafe>=2.0 in ./venv/lib/python3.11/site-packages (from jinja2>=3.0->omlx==0.5.8.dev2) (3.0.3)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in ./venv/lib/python3.11/site-packages (from jsonschema>=4.0.0->omlx==0.5.8.dev2) (2025.9.1)
Requirement already satisfied: referencing>=0.28.4 in ./venv/lib/python3.11/site-packages (from jsonschema>=4.0.0->omlx==0.5.8.dev2) (0.37.0)
Requirement already satisfied: rpds-py>=0.25.0 in ./venv/lib/python3.11/site-packages (from jsonschema>=4.0.0->omlx==0.5.8.dev2) (2026.6.3)
Requirement already satisfied: cffi>=1.12.0 in ./venv/lib/python3.11/site-packages (from miniaudio>=1.59->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (2.1.1)
Requirement already satisfied: pycparser in ./venv/lib/python3.11/site-packages (from cffi>=1.12.0->miniaudio>=1.59->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (3.0)
Requirement already satisfied: pydantic-extra-types>=2.10.5 in ./venv/lib/python3.11/site-packages (from pydantic-extra-types[pycountry]>=2.10.5->mistral-common>=1.10->omlx==0.5.8.dev2) (2.11.1)
Requirement already satisfied: annotated-types>=0.6.0 in ./venv/lib/python3.11/site-packages (from pydantic>=2.9.0->fastapi>=0.108.0->omlx==0.5.8.dev2) (0.8.0)
Requirement already satisfied: pydantic-core==2.46.4 in ./venv/lib/python3.11/site-packages (from pydantic>=2.9.0->fastapi>=0.108.0->omlx==0.5.8.dev2) (2.46.4)
Requirement already satisfied: scipy>=1.10.0 in ./venv/lib/python3.11/site-packages (from mlx-audio>=0.4.3->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (1.17.1)
Requirement already satisfied: sounddevice>=0.5.3 in ./venv/lib/python3.11/site-packages (from mlx-audio>=0.4.3->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (0.5.5)
Requirement already satisfied: flatbuffers in ./venv/lib/python3.11/site-packages (from onnxruntime>=1.17.0->magika~=0.6.1->markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (25.12.19)
Requirement already satisfied: cryptography>=36.0.0 in ./venv/lib/python3.11/site-packages (from pdfminer-six>=20251230->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (50.0.0)
Requirement already satisfied: pypdfium2>=5.9.0 in ./venv/lib/python3.11/site-packages (from pdfplumber>=0.11.9->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (5.12.1)
Requirement already satisfied: pycountry>=23 in ./venv/lib/python3.11/site-packages (from pydantic-extra-types[pycountry]>=2.10.5->mistral-common>=1.10->omlx==0.5.8.dev2) (26.2.16)
Requirement already satisfied: urllib3<3,>=1.26 in ./venv/lib/python3.11/site-packages (from requests>=2.28.0->omlx==0.5.8.dev2) (2.7.0)
Requirement already satisfied: markdown-it-py>=2.2.0 in ./venv/lib/python3.11/site-packages (from rich>=13.0.0->omlx==0.5.8.dev2) (4.2.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in ./venv/lib/python3.11/site-packages (from rich>=13.0.0->omlx==0.5.8.dev2) (2.20.0)
Requirement already satisfied: mdurl~=0.1 in ./venv/lib/python3.11/site-packages (from markdown-it-py>=2.2.0->rich>=13.0.0->omlx==0.5.8.dev2) (0.1.2)
Requirement already satisfied: soupsieve>=1.6.1 in ./venv/lib/python3.11/site-packages (from beautifulsoup4->markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (2.9.2)
Requirement already satisfied: six<2,>=1.15 in ./venv/lib/python3.11/site-packages (from markdownify->markitdown==0.1.7->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (1.17.0)
Requirement already satisfied: python-dateutil>=2.8.2 in ./venv/lib/python3.11/site-packages (from pandas->datasets>=2.19.1->mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm@78b96eb5462141447b9a6b4943ef553891da56dd->omlx==0.5.8.dev2) (2.9.0.post0)
Requirement already satisfied: XlsxWriter>=0.5.7 in ./venv/lib/python3.11/site-packages (from python-pptx->markitdown[docx,pdf,pptx]==0.1.7->omlx==0.5.8.dev2) (3.2.9)
Requirement already satisfied: shellingham>=1.3.0 in ./venv/lib/python3.11/site-packages (from typer->transformers<5.13,>=5.12.1->omlx==0.5.8.dev2) (1.5.4)
Building wheels for collected packages: omlx
  Building editable for omlx (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building editable for omlx (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [154 lines of output]
      /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated
      !!
      
              ********************************************************************************
              Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0).
      
              By 2027-Feb-18, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.
      
              See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
              ********************************************************************************
      
      !!
        corresp(dist, value, root_dir)
      /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/config/_apply_pyprojecttoml.py:61: SetuptoolsDeprecationWarning: License classifiers are deprecated.
      !!
      
              ********************************************************************************
              Please consider removing the following classifiers in favor of a SPDX license expression:
      
              License :: OSI Approved :: Apache Software License
      
              See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
              ********************************************************************************
      
      !!
        dist._finalize_license_expression()
      /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/dist.py:765: SetuptoolsDeprecationWarning: License classifiers are deprecated.
      !!
      
              ********************************************************************************
              Please consider removing the following classifiers in favor of a SPDX license expression:
      
              License :: OSI Approved :: Apache Software License
      
              See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
              ********************************************************************************
      
      !!
        self._finalize_license_expression()
      running editable_wheel
      creating /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx.egg-info
      writing /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx.egg-info/PKG-INFO
      writing dependency_links to /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx.egg-info/dependency_links.txt
      writing entry points to /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx.egg-info/entry_points.txt
      writing requirements to /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx.egg-info/requires.txt
      writing top-level names to /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx.egg-info/top_level.txt
      writing manifest file '/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx.egg-info/SOURCES.txt'
      reading manifest file '/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx.egg-info/SOURCES.txt'
      adding license file 'LICENSE'
      writing manifest file '/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx.egg-info/SOURCES.txt'
      creating '/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx-0.5.8.dev2.dist-info'
      [WARNING] MACOSX_DEPLOYMENT_TARGET is set to a lower value (15.0) than the version on which the Python interpreter was compiled (26.0), and will be ignored.
      creating /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-ephem-wheel-cache-zc5gearz/wheels/8d/b1/08/a56ceafc4b4781de6d262e6f44ddeb3e79350155554933e760/tmpkf7po8l_/.tmp-cvh8aewz/omlx-0.5.8.dev2.dist-info/WHEEL
      [WARNING] MACOSX_DEPLOYMENT_TARGET is set to a lower value (15.0) than the version on which the Python interpreter was compiled (26.0), and will be ignored.
      running build_py
      running build_ext
      -- The CXX compiler identification is AppleClang 21.0.0.21000101
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found Python: /Users/office/omlx/venv/bin/python3.11 (found suitable version "3.11.15", minimum required is "3.8") found components: Interpreter Development.Module
      -- Found MLX: /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/mlx/lib/libmlx.dylib
      -- Configuring done (1.7s)
      -- Generating done (0.0s)
      -- Build files have been written to: /private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/tmp2p8be0ly.build-temp/omlx.custom_kernels.bonsai._ext
      [  5%] Building spec_decode.air
      xcrun: error: unable to find utility "metal", not a developer tool or in PATH
      make[2]: *** [spec_decode.air] Error 72
      make[2]: *** Waiting for unfinished jobs....
      [ 11%] Building bonsai_quantized.air
      xcrun: error: unable to find utility "metal", not a developer tool or in PATH
      make[2]: *** [bonsai_quantized.air] Error 72
      make[1]: *** [CMakeFiles/omlx_bonsai_metallib.dir/all] Error 2
      make[1]: *** Waiting for unfinished jobs....
      [ 16%] Building CXX object CMakeFiles/nanobind-static.dir/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/nanobind/src/nb_func.cpp.o
      [ 22%] Building CXX object CMakeFiles/nanobind-static.dir/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/nanobind/src/nb_internals.cpp.o
      [ 27%] Building CXX object CMakeFiles/nanobind-static.dir/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/nanobind/src/trampoline.cpp.o
      [ 33%] Building CXX object CMakeFiles/nanobind-static.dir/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/nanobind/src/nb_ndarray.cpp.o
      [ 38%] Building CXX object CMakeFiles/nanobind-static.dir/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/nanobind/src/nb_enum.cpp.o
      [ 44%] Building CXX object CMakeFiles/nanobind-static.dir/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/nanobind/src/nb_static_property.cpp.o
      [ 50%] Building CXX object CMakeFiles/nanobind-static.dir/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/nanobind/src/implicit.cpp.o
      [ 55%] Building CXX object CMakeFiles/nanobind-static.dir/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/nanobind/src/common.cpp.o
      [ 61%] Building CXX object CMakeFiles/nanobind-static.dir/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/nanobind/src/error.cpp.o
      [ 66%] Building CXX object CMakeFiles/nanobind-static.dir/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/nanobind/src/nb_type.cpp.o
      [ 72%] Linking CXX static library libnanobind-static.a
      [ 72%] Built target nanobind-static
      make: *** [all] Error 2
      Traceback (most recent call last):
        File "/Users/office/omlx/venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
          main()
        File "/Users/office/omlx/venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
          json_out["return_val"] = hook(**hook_input["kwargs"])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/office/omlx/venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 303, in build_editable
          return hook(wheel_directory, config_settings, metadata_directory)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 474, in build_editable
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 410, in _build_with_temp_dir
          self.run_setup()
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 317, in run_setup
          exec(code, locals())  # noqa: S102 # exec is intentional here
          ^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 62, in <module>
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)  # type: ignore[return-value]
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 168, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 184, in run_commands
          dist.run_commands()
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1028, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1106, in run_command
          super().run_command(command)
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1047, in run_command
          cmd_obj.run()
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 142, in run
          self._create_wheel_file(bdist_wheel)
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 352, in _create_wheel_file
          files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 275, in _run_build_commands
          self._run_build_subcommands()
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 302, in _run_build_subcommands
          self.run_command(name)
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 342, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1106, in run_command
          super().run_command(command)
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1047, in run_command
          cmd_obj.run()
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/mlx/extension.py", line 73, in run
          super().run()
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 95, in run
          _build_ext.run(self)
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 382, in run
          self.build_extensions()
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 498, in build_extensions
          self._build_extensions_serial()
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 524, in _build_extensions_serial
          self.build_extension(ext)
        File "/private/var/folders/38/lkswt6ss2pj3c4nddbh6jd_80000gp/T/pip-build-env-mkwmtf3w/overlay/lib/python3.11/site-packages/mlx/extension.py", line 68, in build_extension
          subprocess.run(
        File "/opt/homebrew/Cellar/python@3.11/3.11.15_4/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '--build', '.', '-j32']' returned non-zero exit status 2.
      An error occurred when building editable wheel for omlx.
      See debugging tips in: https://setuptools.pypa.io/en/latest/userguide/development_mode.html#debugging-tips
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building editable for omlx
Failed to build omlx
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> omlx

[smithcoin]

The first thing that I noticed in the GUI is that each machine thinks the other machine doesn't have enough memory:

Another thing that is slightly 'frustrating' is that when I am trying to work on the cluster this element keeps toggling between this state:

and this state:

which causes a layout shift in the whole GUI making it hard to edit anything.

In the GUI it states that RDMA isn't enabled for both machines, but it is:

Here are my diagnostic logs from the two machines again:
omlx-cluster-diagnostics-2026-08-09T18-57-32.886Z.json

omlx-cluster-diagnostics-2026-08-09T18-58-12.956Z.json

Also I noticed this javascript error in the admin section:

[smithcoin]

I'd be willing to download another model and give an apples to apples comparison to exo (if that is the issue), but that was the largest model I got working with that. If there is anything else I should do let me know. Happy to get this working!

[ashhart]

@smithcoin

Thanks for the detailed testing and diagnostics. I’ve pushed another update addressing the reported memory, topology, RDMA, and dashboard issues:

• Memory capacity now comes from each Mac’s exact live oMLX admission ceiling.
• Removed the misleading 64 GiB placeholder that appeared as “56 GiB usable” before a peer had answered.
• Memory measurement no longer requires collective IP addresses to have already been configured.
• Cached or failed peer probes can no longer make the model catalogue appear ready.
• Peer memory is now associated with its SSH identity, so identically named Macs cannot inherit one another’s measurements.
• Memory cards remain aligned with the correct Mac in clusters containing three or more nodes.
• Automatic SSH connections now explicitly use IPv4, preventing a .local hostname from unexpectedly selecting an unrelated IPv6 route.
• Fabric discovery checks the route selected by macOS and verifies connectivity in both directions.
• If the preferred Thunderbolt/RDMA address is unreachable, oMLX tries the next viable shared route and can fall back to verified Ethernet/TCP.
• Manually entered static IPv4 addresses are preserved and use the TCP ring rather than being overwritten by mDNS discovery.
• RDMA probe failures caused by SSH or tooling errors are now reported as “unable to verify” rather than incorrectly claiming RDMA is disabled.
• For clusters with more than two Macs, every Mac-to-Mac pair is verified instead of checking only coordinator-to-worker links.
• A fabric matrix is re-read whenever topology placement changes rank order, preventing addresses or RDMA devices from being assigned to the wrong rank.
• An unverified fabric now blocks performance probing, model staging, and activation before any distributed process starts.
• Automatic retry polling keeps the actionable error visible, eliminating the repeated dashboard layout shift.
• Late catalogue and planning responses are discarded when their inputs have changed.
• Structured API validation errors now render as readable messages instead of [object Object].
• SSH destinations are validated before fabric discovery, autoconfiguration, link setup, or memory probes can invoke SSH.
• First-time SSH setup now includes a copyable Terminal command and managed-key generation guidance.
• Models requiring trust_remote_code are detected before opening large weight files, avoiding the misleading memory spike after a failed load.

The complete cluster test suite passes: 743 tests. JavaScript syntax, Python compilation, lint, diff, and privacy checks are also clean.

The separate xcrun: unable to find utility "metal" error indicates that the full Xcode Metal toolchain is unavailable or not selected on that Mac; it is not caused by cluster topology or model sizing.

Please pull the latest PR branch and retry both automatic discovery and your static IPv4 Thunderbolt configuration. Fresh diagnostics would be very useful if either route still fails.

[smithcoin]

Okay I installed Xcode and the Metal toolchain and pulled the latest version.

I am still getting the layout shift from this component which hurts the usability of the whole page:

It doesn't seem like I way to have the short lived key between the two devices. I am able to ssh from each machine to the other FYI.

The machines are still not reading the correct size of the other (or detecting RDMA)

I still see this front end error:

Here are the logs:
omlx-cluster-diagnostics-2026-08-09T21-18-47.164Z.json
omlx-cluster-diagnostics-2026-08-09T21-18-34.055Z.json

Let me know if there in another manner in which you'd like me to test.

[ashhart]

@smithcoin Thanks, I’ve pushed a fix for this.

SSH errors now remain stable during polling, and the pairing UI explains the dedicated oMLX key with clear steps for both Macs. Please update and restart oMLX on both machines, hard-refresh the browser, then try pairing again.

Your diagnostics show ~495 GiB and RDMA enabled on both Macs, so the earlier values appeared to be stale UI state.