Model Blending (Mixture-of-Agents + LLM Routers)

[TomLucidor]

There are three approaches to mixing multiple models (with different sources and architectures):

• Mixture of Agents (aggregating answers) instead of routing https://github.com/togethercomputer/MoA
• Difficulty based (or "easy vs hard") routing like RouteLLM https://github.com/lm-sys/RouteLLM
• Task-based routing like Wilmer https://github.com/SomeOddCodeGuy/WilmerAI

Key considerations would be:

• self-hosting cost relative to accuracy needs
• mitigating possible biases in some of the bigger models

[rmusser01]

Ties in with semantic caching and having a request proxy sit in front of all requests.