Discussion: enterprise desktop fleets as distributed AI infrastructure

[fatdragonGordon]

Hi all — really impressed by the direction of mesh-llm and the broader idea of decentralized inference across heterogeneous edge devices.

I am currently with Lenovo’s desktop business, where we ship roughly 14M desktop systems annually across enterprise, education, SMB, gaming, and consumer segments. A meaningful portion of the installed base includes GPU-capable systems with substantial local compute resources.

Internally, we’ve been exploring a broader question:

Can large-scale endpoint fleets evolve into part of a distributed AI infrastructure layer for local inference, agentic workloads, and idle compute participation?

What’s interesting to us about mesh-llm is the emphasis on:

• heterogeneous devices,
• distributed orchestration,
• local-first AI,
• and open ecosystem participation.

Would love to connect with maintainers or ecosystem folks interested in discussing:

• enterprise endpoint participation,
• orchestration/runtime integration,
• edge inference experimentation,
• or broader distributed compute models.

Feels like there may be interesting alignment between large enterprise desktop fleets and emerging decentralized AI infrastructure.

Happy to continue the discussion here or connect directly.

Gordon

[i386]

@fatdragonGordon very relevant to some things we are discussing and we'd love to chat. Is there a good email to contact you on?

Or reach out via LinkedIn https://www.linkedin.com/in/jameswdumay

[fatdragonGordon]

Hi James, my email is: chenchang@lenovo.com, happy to chat.

[michaelneale]

@fatdragonGordon yeah a lot of the ideas for this came when I looked around the house and saw a bunch of idle compute which was pretty closely interlinked, so why not, but extending this to enterprise is absolutely more valuable, I agree. Would love to talk more.

Given standard operating environments, there is an opportunity to make this available and not interrupt the user - so when it is unneeded can contribute some compute, models or stages as needed to the mesh (and via private networks)

[ndizazzo]

@fatdragonGordon This PR might be interesting to you: #560. It's just a flag-gated mockup right now with the potential direction, but I run a few machines like Mic pointed out that I don't want running at all times.

One of the things I thought would be useful is using AWS / GCP / DO APIs to wake virtual machines, to eliminate billing when the node is not needed. Otherwise, "suspending" the mesh connection on self-owned baremetal machines would be the mechanism for non-cloud infra.

[fatdragonGordon]

Thanks for the feedback and great mock up!

This actually makes whole lot of sense. I like the idea of maintaining a fleet of mesh-capable nodes and selectively bringing them into the active mesh based on demand. The reserved capacity approach seems like a reasonable path toward supporting more predictable and enterprise-oriented deployments.

I also agree with your point that a coordinator/control plane will probably be necessary to manage this effectively.

Really appreciate you putting together the mockup and helping evolve the idea.

[michaelneale]

yeah that is pretty cool.

I have been playing with vast.ai and also wondered if similar things could be used where you can bring things on line based on low cost (signal for unused I guess?)>

A controller is elected now at least when doing multi stage hosting, and ideally that one would be the most stable node.

In the case of multi model (mixture of agents style) then as long as there is enough stability for things to converge, then not a whole lot is needed for a control plane (compared to cost of hosting the weights, control plane logic is very very lightweight!).

All fun ideas worth exploring.