Add Graph of Tiered Experts Architecture environment

[signabuilder]

Graph of Tiered Experts Architecture Environment

Trains a language model to act as a routing policy over frozen Hermes-series expert tiers.

What it does

• 7 frozen Hermes-series experts distributed across 3B, 8B, and 70B tiers
• 120 routing scenarios (8 query templates x 15 topics) covering triage, synthesis, validation, execution, simulation, classification, and research
• 3-component reward: ideal match (Jaccard similarity, 50%), capability alignment (30%), cost efficiency (10%)
• Expert selection via JSON output, e.g. ["a0", "v0"]

Why it matters

Standard MoE trains experts and gates jointly inside a single model architecture. This environment instead learns post-hoc routing across frozen Hermes-series tiers, making the routing problem trainable without retraining the experts themselves.

Architecture

Expert	Tier	Role
g0	DeepHermes 3 3B	Triage
g1	DeepHermes 3 3B	Classification
a0	DeepHermes 3 8B	Synthesis
a1	DeepHermes 3 8B	Adversarial challenge
v0	Hermes 3 70B	Validation
b0	Hermes 3 70B	Execution
q0	DeepHermes 3 8B	Quorum simulation

Quick start

vllm serve NousResearch/DeepHermes-3-Llama-3-8B-Preview --port 8378
python environments/community/moe_routing/moe_routing_env.py serve --port 8332
python environments/community/moe_routing/moe_routing_env.py process --num_trajectories 100

Research applications

• Graph of Tiered Experts Architecture routing
• LM-as-router via RL
• Cost-aware expert selection
• Federated routing policy optimization via Psyche/DisTrO

[signabuilder]

n

[dmahan93]

This being called MoE conflicts with the common nomenclature of referring to a specific model architecture, usually i see this referred to as agent routing or similar