Skip to content

Commit bf9aba8

Browse files
MCERQUAMCERQUA
committed
✨ Implement automatic model template selection system
- Added comprehensive ModelTemplates class with 10+ model templates - Integrated auto-template selection in admin panel - Templates automatically selected based on model ID/name - Added client-side model-templates.js for UI integration - Enhanced admin panel with template preview and management - Added test suite for template selection verification - Templates persist with model selection in localStorage - Supports TinyLlama, Phi, Mistral, Llama, CodeLlama, Alpaca, Vicuna, Wizard, Orca, StableLM formats - Default fallback for unknown models - Context size auto-adjustment based on model capabilities
1 parent 3b063f8 commit bf9aba8

File tree

8 files changed

+1020
-11
lines changed

8 files changed

+1020
-11
lines changed

‎docs/TEMPLATE-SYSTEM.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Model Template Auto-Population System
2+
3+
## Overview
4+
The LLM Router now includes an intelligent template auto-selection system that automatically configures the correct chat template format for each model type.
5+
6+
## Features
7+
8+
### 🎯 Auto-Detection
9+
- Automatically detects model type from ID and name
10+
- Selects appropriate chat template without manual configuration
11+
- Supports 10+ model architectures out of the box
12+
13+
### 📋 Supported Model Templates
14+
15+
1. **TinyLlama** - `<|system|>`, `<|user|>`, `<|assistant|>` format
16+
2. **Phi** - Microsoft Phi instruction format
17+
3. **Mistral/Mixtral** - `[INST]` format
18+
4. **Llama 2** - Meta's Llama instruction format
19+
5. **CodeLlama** - Specialized coding assistant format
20+
6. **Alpaca/Vicuna** - Stanford's instruction format
21+
7. **Wizard** - WizardLM format
22+
8. **Orca** - Microsoft Orca format
23+
9. **StableLM** - Stability AI format
24+
10. **ChatML** - OpenAI compatible format
25+
11. **Default** - Generic fallback format
26+
27+
### 🔧 How It Works
28+
29+
1. **Model Selection**: When a user selects a model in the admin panel
30+
2. **Pattern Matching**: System analyzes model ID and name
31+
3. **Template Selection**: Appropriate template is automatically selected
32+
4. **Configuration Update**: Template settings are applied and saved
33+
5. **Persistence**: Settings persist across sessions
34+
35+
### 📁 Implementation Files
36+
37+
- `/src/config/ModelTemplates.js` - Server-side template definitions
38+
- `/public/chat/model-templates.js` - Client-side template mappings
39+
- `/public/chat/admin.js` - Admin panel integration
40+
41+
### 🧪 Testing
42+
43+
Run the template selection test:
44+
```bash
45+
node test-template-selection.js
46+
```
47+
48+
Access the integration test page:
49+
```
50+
http://localhost:3000/chat/test-admin-integration.html
51+
```
52+
53+
### 💡 Usage Example
54+
55+
```javascript
56+
// Automatic template selection
57+
const modelId = 'tinyllama-1.1b-chat';
58+
const template = ModelTemplates.getTemplateForModel(modelId);
59+
// Returns TinyLlama template with correct prefixes
60+
61+
// Format a conversation
62+
const messages = [
63+
{ role: 'system', content: 'You are helpful.' },
64+
{ role: 'user', content: 'Hello!' },
65+
{ role: 'assistant', content: 'Hi there!' }
66+
];
67+
const formatted = ModelTemplates.formatConversation(messages, modelId);
68+
```
69+
70+
### 🎛️ Admin Panel Integration
71+
72+
The admin panel now:
73+
- Shows all available templates in a dropdown
74+
- Auto-selects the correct template when switching models
75+
- Displays template preview with actual formatting
76+
- Saves template configuration with model selection
77+
- Allows custom template creation and management
78+
79+
### ✅ Benefits
80+
81+
1. **Zero Configuration** - Works automatically for known models
82+
2. **Better Accuracy** - Correct formatting improves model responses
83+
3. **Time Saving** - No manual template configuration needed
84+
4. **Extensible** - Easy to add new model templates
85+
5. **User Friendly** - Seamless experience in admin panel
86+
87+
## Adding New Templates
88+
89+
To add support for a new model type:
90+
91+
1. Add template definition to `/src/config/ModelTemplates.js`
92+
2. Add client-side mapping to `/public/chat/model-templates.js`
93+
3. Add model ID pattern matching if needed
94+
95+
Example:
96+
```javascript
97+
newmodel: {
98+
name: 'NewModel',
99+
systemPrefix: 'System: ',
100+
userPrefix: 'User: ',
101+
assistantPrefix: 'Assistant: ',
102+
messageSeparator: '\n',
103+
stopSequences: ['User:', 'System:']
104+
}
105+
```
106+
107+
## Troubleshooting
108+
109+
- If a model doesn't auto-select the correct template, check the model ID pattern matching
110+
- Custom templates are saved in localStorage and persist across sessions
111+
- The default template is used as fallback for unknown models

‎models/registry.json‎

Lines changed: 31 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,11 @@
1919
"completion": true
2020
},
2121
"metrics": {
22-
"loadTime": 1755209001275,
22+
"loadTime": 1755211708696,
2323
"inferenceCount": 2,
2424
"totalTokens": 2,
25-
"avgLatency": 1504,
26-
"lastUsed": "2025-08-14T22:03:27.310Z"
25+
"avgLatency": 1498.5,
26+
"lastUsed": "2025-08-14T22:48:31.124Z"
2727
},
2828
"source": "./models/tinyllama/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf",
2929
"path": "./models/tinyllama/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf"
@@ -44,14 +44,41 @@
4444
"completion": true
4545
},
4646
"metrics": {
47-
"loadTime": 1755210612191,
47+
"loadTime": 1755211711029,
4848
"inferenceCount": 0,
4949
"totalTokens": 0,
5050
"avgLatency": 0,
5151
"lastUsed": null
5252
},
5353
"source": "models/phi-2/phi-2.Q4_K_M.gguf",
5454
"path": "models/phi-2/phi-2.Q4_K_M.gguf"
55+
},
56+
{
57+
"id": "llama-2-7b-chat",
58+
"name": "llama-2-7b-chat.Q4_K_M",
59+
"version": "1.0.0",
60+
"format": "gguf",
61+
"architecture": {
62+
"type": "llama"
63+
},
64+
"parameters": {},
65+
"metadata": {},
66+
"capabilities": {
67+
"streaming": false,
68+
"batching": false,
69+
"quantization": false,
70+
"embedding": false,
71+
"completion": true
72+
},
73+
"metrics": {
74+
"loadTime": 1755211785508,
75+
"inferenceCount": 0,
76+
"totalTokens": 0,
77+
"avgLatency": 0,
78+
"lastUsed": null
79+
},
80+
"source": "models/llama-2-7b-chat/llama-2-7b-chat.Q4_K_M.gguf",
81+
"path": "models/llama-2-7b-chat/llama-2-7b-chat.Q4_K_M.gguf"
5582
}
5683
]
5784
}

‎public/chat/admin.html‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -383,6 +383,7 @@ <h3>System Logs</h3>
383383
<span id="notificationText"></span>
384384
</div>
385385

386+
<script src="/chat/model-templates.js"></script>
386387
<script src="/chat/admin.js"></script>
387388
</body>
388389
</html>

‎public/chat/admin.js‎

Lines changed: 118 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,9 @@ class AdminPanel {
2525
this.setupMonitoring();
2626
this.setupImportExport();
2727

28+
// Initialize template selector with all available templates
29+
this.updateTemplateSelector();
30+
2831
// Load config first
2932
this.loadCurrentConfig();
3033
// Then load models, which will restore the saved selection
@@ -109,6 +112,8 @@ class AdminPanel {
109112
this.saveCurrentTemplate();
110113
});
111114

115+
// Initialize the template selector and list
116+
this.updateTemplateSelector();
112117
this.loadTemplateList();
113118
}
114119

@@ -246,20 +251,110 @@ class AdminPanel {
246251
<p><strong>Context:</strong> ${model.contextSize || 'Unknown'}</p>
247252
<p><strong>Status:</strong> ${model.loaded ? 'Loaded' : 'Not loaded'}</p>
248253
`;
254+
255+
// Auto-select the appropriate template for this model
256+
this.autoSelectTemplate(modelId, model.name);
257+
258+
// Also apply any model-specific parameters if available
259+
if (model.contextSize) {
260+
document.getElementById('contextSize').value = model.contextSize;
261+
document.getElementById('contextSizeValue').textContent = model.contextSize;
262+
this.config.parameters.contextSize = model.contextSize;
263+
}
249264
}
250265
}
251266
this.config.activeModel = modelId;
252267
}
253268

269+
autoSelectTemplate(modelId, modelName) {
270+
// Get the appropriate template for this model
271+
const template = window.ModelTemplates.getTemplateForModel(modelId, modelName);
272+
const templateId = window.ModelTemplates.getTemplateIdForModel(modelId, modelName);
273+
274+
// Update the template dropdown if we're on the templates section
275+
const templateSelect = document.getElementById('activeTemplate');
276+
if (templateSelect) {
277+
// Check if this template exists in our saved templates
278+
if (!this.templates[templateId]) {
279+
// Add the template to our saved templates
280+
this.templates[templateId] = template;
281+
this.saveTemplates();
282+
283+
// Update the dropdown
284+
this.updateTemplateSelector();
285+
}
286+
287+
// Select the template
288+
templateSelect.value = templateId;
289+
this.loadTemplate(templateId);
290+
291+
// Show notification
292+
this.showNotification(`Auto-selected ${template.name} template for ${modelName || modelId}`, 'info');
293+
}
294+
295+
// Also update the template in config
296+
this.config.activeTemplate = templateId;
297+
this.config.templateSettings = template;
298+
}
299+
300+
updateTemplateSelector() {
301+
const selector = document.getElementById('activeTemplate');
302+
if (!selector) return;
303+
304+
const currentValue = selector.value;
305+
selector.innerHTML = '';
306+
307+
// Add all predefined templates from ModelTemplates
308+
const allTemplates = window.ModelTemplates.getAllTemplates();
309+
allTemplates.forEach(template => {
310+
const option = document.createElement('option');
311+
option.value = template.id;
312+
option.textContent = template.name;
313+
selector.appendChild(option);
314+
});
315+
316+
// Add any custom saved templates
317+
Object.keys(this.templates).forEach(name => {
318+
if (!allTemplates.find(t => t.id === name)) {
319+
const option = document.createElement('option');
320+
option.value = name;
321+
option.textContent = `Custom: ${name.charAt(0).toUpperCase() + name.slice(1)}`;
322+
selector.appendChild(option);
323+
}
324+
});
325+
326+
// Restore selection if it exists
327+
if (currentValue && Array.from(selector.options).some(opt => opt.value === currentValue)) {
328+
selector.value = currentValue;
329+
}
330+
}
331+
254332
// Template Management
255333
loadTemplate(templateName) {
256-
const template = this.templates[templateName] || this.templates.default;
334+
// First check if it's a predefined template from ModelTemplates
335+
const predefinedTemplate = window.ModelTemplates.templates[templateName];
336+
const template = predefinedTemplate || this.templates[templateName] || this.templates.default;
337+
257338
document.getElementById('systemPrefix').value = template.systemPrefix || '';
258339
document.getElementById('userPrefix').value = template.userPrefix || '';
259340
document.getElementById('assistantPrefix').value = template.assistantPrefix || '';
260-
document.getElementById('messageSeparator').value = template.separator || '\\n\\n';
261-
document.getElementById('stopSequences').value = (template.stopSequences || []).join(', ');
341+
document.getElementById('messageSeparator').value = template.messageSeparator || template.separator || '\\n\\n';
342+
343+
// Handle stop sequences - could be array or string
344+
const stopSeq = template.stopSequences;
345+
if (Array.isArray(stopSeq)) {
346+
document.getElementById('stopSequences').value = stopSeq.join(', ');
347+
} else if (typeof stopSeq === 'string') {
348+
document.getElementById('stopSequences').value = stopSeq;
349+
} else {
350+
document.getElementById('stopSequences').value = '';
351+
}
352+
262353
this.updateTemplatePreview();
354+
355+
// Update config
356+
this.config.activeTemplate = templateName;
357+
this.config.templateSettings = template;
263358
}
264359

265360
updateTemplatePreview() {
@@ -292,16 +387,32 @@ class AdminPanel {
292387
const container = document.getElementById('templateList');
293388
container.innerHTML = '';
294389

295-
Object.keys(this.templates).forEach(name => {
390+
// First add all predefined templates from ModelTemplates
391+
const predefinedTemplates = window.ModelTemplates.getAllTemplates();
392+
predefinedTemplates.forEach(template => {
296393
const div = document.createElement('div');
297394
div.className = 'template-item';
298395
div.innerHTML = `
299-
<span>${name}</span>
300-
<button class="btn btn-sm" onclick="adminPanel.loadTemplate('${name}')">Load</button>
301-
<button class="btn btn-sm" onclick="adminPanel.deleteTemplate('${name}')">Delete</button>
396+
<span>${template.name} (Built-in)</span>
397+
<button class="btn btn-sm" onclick="adminPanel.loadTemplate('${template.id}')">Load</button>
302398
`;
303399
container.appendChild(div);
304400
});
401+
402+
// Then add any custom saved templates
403+
Object.keys(this.templates).forEach(name => {
404+
// Skip if it's already in predefined templates
405+
if (!predefinedTemplates.find(t => t.id === name)) {
406+
const div = document.createElement('div');
407+
div.className = 'template-item';
408+
div.innerHTML = `
409+
<span>${name}</span>
410+
<button class="btn btn-sm" onclick="adminPanel.loadTemplate('${name}')">Load</button>
411+
<button class="btn btn-sm" onclick="adminPanel.deleteTemplate('${name}')">Delete</button>
412+
`;
413+
container.appendChild(div);
414+
}
415+
});
305416
}
306417

307418
deleteTemplate(name) {

0 commit comments

Comments
 (0)