|
| 1 | +# Greybox LLM Strategy — Validator Setup Guide |
| 2 | + |
| 3 | +Switch your GenLayer node from random LLM provider selection to deterministic ordered fallback via OpenRouter. |
| 4 | + |
| 5 | +## What is Greybox? |
| 6 | + |
| 7 | +By default, the node picks a random LLM provider for each call. With **greybox**, the node uses a fixed priority chain configured via `meta.greybox` fields in the YAML config. |
| 8 | + |
| 9 | +**Default text chain:** deepseek-v3.2 → qwen3-235b → claude-haiku-4.5 → kimi-k2 → glm-5 → llama-3.3 (heurist) → llama-3.3 (ionet) |
| 10 | + |
| 11 | +**Default image chain:** gpt-5.1-mini → gemini-3-flash → claude-haiku-4.5 |
| 12 | + |
| 13 | +Chain order is determined by the `meta.greybox` priority numbers on each model in the YAML. Lower number = higher priority. You can change the order by editing these numbers — no Lua changes needed. |
| 14 | + |
| 15 | +OpenRouter is the primary aggregator. If it fails, the node falls back to direct provider APIs (heurist, ionet). |
| 16 | + |
| 17 | +## Prerequisites |
| 18 | + |
| 19 | +- GenLayer node **v0.5.7+** (tarball must include `genvm-modules-llm-release.yaml` and `genvm-llm-greybox.lua`) |
| 20 | +- An **OpenRouter API key** — get one at https://openrouter.ai/keys |
| 21 | + |
| 22 | +## Step-by-Step Setup |
| 23 | + |
| 24 | +### 1. Stop the node |
| 25 | + |
| 26 | +```bash |
| 27 | +sudo systemctl stop genlayer-node |
| 28 | +``` |
| 29 | + |
| 30 | +### 2. Add OpenRouter API key to .env |
| 31 | + |
| 32 | +```bash |
| 33 | +# Find your active .env |
| 34 | +ENV_FILE="/opt/genlayer-node/.env" |
| 35 | + |
| 36 | +# Add the key (or edit the file manually) |
| 37 | +echo "OPENROUTERKEY=sk-or-v1-your-key-here" >> ${ENV_FILE} |
| 38 | +``` |
| 39 | + |
| 40 | +### 3. Apply the release LLM config |
| 41 | + |
| 42 | +The tarball ships with a unified config that includes all backends (openrouter, morpheus, heurist, ionet, etc.). |
| 43 | + |
| 44 | +```bash |
| 45 | +VERSION=$(readlink /opt/genlayer-node/bin | sed 's|/bin||; s|.*/||') |
| 46 | + |
| 47 | +cp /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-modules-llm-release.yaml \ |
| 48 | + /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml |
| 49 | +``` |
| 50 | + |
| 51 | +### 4. Switch to greybox strategy |
| 52 | + |
| 53 | +```bash |
| 54 | +sed -i 's/genvm-llm-default\.lua/genvm-llm-greybox.lua/' \ |
| 55 | + /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml |
| 56 | +``` |
| 57 | + |
| 58 | +### 5. Verify the config |
| 59 | + |
| 60 | +```bash |
| 61 | +# Check lua script path |
| 62 | +grep lua_script_path /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml |
| 63 | +# Expected: lua_script_path: ${exeDir}/../config/genvm-llm-greybox.lua |
| 64 | + |
| 65 | +# Check openrouter is present |
| 66 | +grep -A2 'openrouter:' /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml |
| 67 | +# Expected: enabled: true |
| 68 | + |
| 69 | +# Check the greybox Lua file exists |
| 70 | +ls -la /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-llm-greybox.lua |
| 71 | +``` |
| 72 | + |
| 73 | +### 6. Start the node |
| 74 | + |
| 75 | +```bash |
| 76 | +sudo systemctl start genlayer-node |
| 77 | +``` |
| 78 | + |
| 79 | +### 7. Verify greybox is active |
| 80 | + |
| 81 | +Wait for an LLM transaction to be processed, then check the logs: |
| 82 | + |
| 83 | +```bash |
| 84 | +sudo journalctl -u genlayer-node --no-hostname | grep "greybox" |
| 85 | +``` |
| 86 | + |
| 87 | +You should see entries like: |
| 88 | +``` |
| 89 | +greybox: using text chain count: 5 |
| 90 | +greybox: success provider: openrouter model: deepseek/deepseek-v3.2 is_primary: true |
| 91 | +``` |
| 92 | + |
| 93 | +## Switching Back to Default |
| 94 | + |
| 95 | +To revert to random provider selection: |
| 96 | + |
| 97 | +```bash |
| 98 | +sudo systemctl stop genlayer-node |
| 99 | + |
| 100 | +sed -i 's/genvm-llm-greybox\.lua/genvm-llm-default.lua/' \ |
| 101 | + /opt/genlayer-node/${VERSION}/third_party/genvm/config/genvm-module-llm.yaml |
| 102 | + |
| 103 | +sudo systemctl start genlayer-node |
| 104 | +``` |
| 105 | + |
| 106 | +## Updating Greybox on a Running Node (No Full Restart) |
| 107 | + |
| 108 | +If you need to update the Lua script or model config without stopping the whole node, |
| 109 | +you can restart just the LLM module on each GenVM instance: |
| 110 | + |
| 111 | +```bash |
| 112 | +# Find the GenVM manager port (check your config or active ports) |
| 113 | +PORT=3999 |
| 114 | + |
| 115 | +# Stop the LLM module |
| 116 | +curl -X POST "http://127.0.0.1:${PORT}/module/stop" \ |
| 117 | + -H 'Content-Type: application/json' \ |
| 118 | + -d '{"module_type": "Llm"}' |
| 119 | + |
| 120 | +# Start the LLM module (reloads Lua script and config) |
| 121 | +curl -X POST "http://127.0.0.1:${PORT}/module/start" \ |
| 122 | + -H 'Content-Type: application/json' \ |
| 123 | + -d '{"module_type": "Llm", "config": null}' |
| 124 | +``` |
| 125 | + |
| 126 | +Repeat for each GenVM instance port. There is no atomic restart — each instance |
| 127 | +restarts independently. |
| 128 | + |
| 129 | +## Customizing the Chain Order |
| 130 | + |
| 131 | +The greybox Lua script reads chain membership and priority from `meta.greybox` on each model in the YAML config. Example: |
| 132 | + |
| 133 | +```yaml |
| 134 | +models: |
| 135 | + deepseek/deepseek-v3.2: |
| 136 | + supports_json: true |
| 137 | + meta: |
| 138 | + greybox: { text: 1 } # text chain, priority 1 (primary) |
| 139 | + openai/gpt-5.1-mini: |
| 140 | + supports_json: true |
| 141 | + supports_image: true |
| 142 | + meta: |
| 143 | + greybox: { image: 1 } # image chain, priority 1 |
| 144 | + anthropic/claude-haiku-4.5: |
| 145 | + supports_json: true |
| 146 | + supports_image: true |
| 147 | + meta: |
| 148 | + greybox: { text: 3, image: 3 } # both chains |
| 149 | +``` |
| 150 | +
|
| 151 | +**Change model order:** Edit the priority numbers. Lower number = tried first. |
| 152 | +
|
| 153 | +**Add a model to the chain:** Add `meta: { greybox: { text: N } }` to any model in any enabled backend. |
| 154 | + |
| 155 | +**Remove a model from the chain:** Remove its `meta.greybox` field. |
| 156 | + |
| 157 | +**Disable an entire provider:** Remove its API key from `.env` — all its models drop out automatically. |
| 158 | + |
| 159 | +The YAML config **must** have `meta.greybox` fields on at least some models. If none are found, the LLM module will fail to start with an error. |
| 160 | + |
| 161 | +After editing the YAML, restart the LLM module (see "Updating Greybox on a Running Node" above) or restart the node. |
| 162 | + |
| 163 | +## Troubleshooting |
| 164 | + |
| 165 | +**"module_failed_to_start" error:** |
| 166 | +- Check that `genvm-llm-greybox.lua` exists in the config directory |
| 167 | +- Check that `OPENROUTERKEY` is set in `.env` and not empty |
| 168 | +- Check that the openrouter backend shows `enabled: true` in the YAML |
| 169 | + |
| 170 | +**No "greybox:" entries in logs:** |
| 171 | +- The greybox Lua only logs when an LLM call happens. Run a transaction that uses an intelligent contract with LLM calls. |
| 172 | +- Verify `lua_script_path` points to `genvm-llm-greybox.lua` (not `default`) |
| 173 | + |
| 174 | +**All models exhausted error:** |
| 175 | +- OpenRouter may be down or your key is invalid |
| 176 | +- Check your key at https://openrouter.ai/settings/keys |
| 177 | +- Fallback providers (heurist, ionet) also need valid keys if you want fallback to work |
0 commit comments