Skip to content

Commit 86ba9bc

Browse files
impoilerimpoiler
committed
[Maxim] Varibles mapping docs
1 parent a829894 commit 86ba9bc

File tree

3 files changed

+216
-0
lines changed

3 files changed

+216
-0
lines changed

‎docs.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,6 +293,7 @@
293293
"library/evaluators/custom-evaluators",
294294
"library/evaluators/third-party-evaluators",
295295
"library/evaluators/folders-in-evaluators",
296+
"library/evaluators/variables-mapping",
296297
"library/evaluators/sessions-in-evaluators"
297298
]
298299
},

‎library/evaluators/meta.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
"custom-evaluators",
66
"third-party-evaluators",
77
"sessions-in-evaluators",
8+
"variables-mapping",
89
"folders-in-evaluators"
910
]
1011
}

‎library/evaluators/variables-mapping.mdx‎

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
---
2+
title: Map Variables to Evaluators
3+
description: Map variables from your prompts, workflows, or datasets to evaluator inputs using our flexible mapping system
4+
---
5+
6+
import { MaximPlayer } from '/snippets/maximPlayer.mdx';
7+
import { Plus } from 'lucide-react';
8+
9+
Variable mapping defines where evaluators should access data from during evaluation. You can map variables from prompts, workflows, or datasets to any pre-built or custom evaluator.
10+
11+
When you select an evaluator, Maxim automatically fills in variable mappings based on common patterns. You can modify these mappings to fit your specific needs. If a variable isn't found in Maxim's built-in options, it falls back to matching dataset column names.
12+
13+
## Dataset Variables
14+
15+
Access any dataset column by its type or column name. These variables are available for all entity types:
16+
17+
- `dataset.input` - Input column (same as `run.input`)
18+
- `dataset.output` - Output column
19+
- `dataset.expectedOutput` - Expected output column
20+
- `dataset.scenario` - Scenario column
21+
- `dataset.expectedSteps` - Expected steps column
22+
- `dataset.expectedToolCalls` - Expected tool calls column
23+
- `dataset.columns["column name"]` - Any custom column by name
24+
25+
## Run Variables
26+
27+
Access run-specific data using the `run.` prefix. Available options depend on your entity type (prompt or workflow).
28+
29+
## Prompt Variable Mapping
30+
31+
### Single-Turn Prompts
32+
33+
Access prompt version data using the `version.` prefix:
34+
35+
- `version.messages` (`JSON`) - All messages in the prompt version, including saved messages
36+
- `version.systemMessage` (`String`) - System message from the saved version
37+
- `version.userMessages` (`JSON`) - User messages combining saved version messages and run input
38+
39+
Access run data:
40+
41+
- `run.input` (`String`) - Input variable
42+
- `run.output` (`String`) - Output variable
43+
- `run.retrieval` (`String`) - Retrieved context based on your selection
44+
- `run.toolCalls` (`JSON`) - Tool calls with name, arguments, and results
45+
46+
### Multi-Turn Prompts
47+
48+
Variable mapping works the same as single-turn prompts:
49+
50+
**Version variables:**
51+
- `version.messages` (`JSON`) - All messages in the prompt version
52+
- `version.systemMessage` (`String`) - System message
53+
- `version.userMessages` (`JSON`) - Combined user messages
54+
55+
**Run variables:**
56+
- `run.input` (`String`) - Input variable
57+
- `run.output` (`String`) - Output variable
58+
- `run.retrieval` (`String`) - Retrieved context
59+
- `run.toolCalls` (`JSON`) - Tool calls with details
60+
61+
## Workflow Variable Mapping
62+
63+
### Single-Turn Workflows
64+
65+
Access workflow response data using the `response.` prefix:
66+
67+
- `run.input` (`String`) - Input variable
68+
- `run.output` (`String`) - Output variable
69+
- `run.retrieval` (`String`) - Retrieved context
70+
- `run.response` (`JSON/any`) - Workflow response with nested field access via dropdown or dot notation for objects, index notation for arrays
71+
72+
### Multi-Turn Workflows
73+
74+
For simulation runs, map variables from the conversation:
75+
76+
- `run.session` (`JSON`) - Complete conversation session
77+
78+
**Example session structure:**
79+
80+
```json
81+
[
82+
{
83+
"request": {
84+
"type": "text",
85+
"payload": { "query": "Can you please book a hotel for me?" }
86+
},
87+
"response": {
88+
"type": "text",
89+
"payload": {
90+
"query": "Sure — may I know the city, check-in date, and number of guests?"
91+
}
92+
},
93+
"turn": 0
94+
},
95+
{
96+
"request": {
97+
"type": "text",
98+
"payload": {
99+
"query": "In Bangalore, 2 guests, check-in on 5th Nov, budget around $80 per night."
100+
}
101+
},
102+
"response": {
103+
"type": "text",
104+
"payload": {
105+
"query": "Got it. Do you prefer a hotel near city center or closer to airport?"
106+
}
107+
},
108+
"turn": 1
109+
},
110+
{
111+
"request": {
112+
"type": "text",
113+
"payload": { "query": "City center. Also free breakfast would be great." }
114+
},
115+
"response": {
116+
"type": "text",
117+
"payload": {
118+
"query": "Understood. Should I auto-book the best matching option or show options first?"
119+
}
120+
},
121+
"turn": 2
122+
},
123+
{
124+
"request": {
125+
"type": "text",
126+
"payload": { "query": "Show options first." }
127+
},
128+
"response": {
129+
"type": "text",
130+
"payload": {
131+
"query": "Here are 3 hotels under $80 with free breakfast near city center. Would you like me to proceed with Hotel Blu Orchid?"
132+
}
133+
},
134+
"turn": 3
135+
}
136+
]
137+
```
138+
139+
## Voice agent Variable Mapping
140+
141+
For voice agent simulation runs, map variables from the conversation:
142+
143+
- `run.session` (`JSON`) - Complete conversation session
144+
- `run.recordingUrl` (`String`) - Call recording url of the call
145+
146+
**Example session structure:**
147+
148+
```json
149+
[
150+
{
151+
"request": {
152+
"type": "text",
153+
"payload": { "query": "Can you please book a hotel for me?" }
154+
},
155+
"response": {
156+
"type": "text",
157+
"payload": {
158+
"query": "Sure — may I know the city, check-in date, and number of guests?"
159+
}
160+
},
161+
"turn": 0
162+
},
163+
{
164+
"request": {
165+
"type": "text",
166+
"payload": {
167+
"query": "In Bangalore, 2 guests, check-in on 5th Nov, budget around $80 per night."
168+
}
169+
},
170+
"response": {
171+
"type": "text",
172+
"payload": {
173+
"query": "Got it. Do you prefer a hotel near city center or closer to airport?"
174+
}
175+
},
176+
"turn": 1
177+
},
178+
{
179+
"request": {
180+
"type": "text",
181+
"payload": { "query": "City center. Also free breakfast would be great." }
182+
},
183+
"response": {
184+
"type": "text",
185+
"payload": {
186+
"query": "Understood. Should I auto-book the best matching option or show options first?"
187+
}
188+
},
189+
"turn": 2
190+
},
191+
{
192+
"request": {
193+
"type": "text",
194+
"payload": { "query": "Show options first." }
195+
},
196+
"response": {
197+
"type": "text",
198+
"payload": {
199+
"query": "Here are 3 hotels under $80 with free breakfast near city center. Would you like me to proceed with Hotel Blu Orchid?"
200+
}
201+
},
202+
"turn": 3
203+
}
204+
]
205+
```
206+
207+
## No-code agent Variable Mapping
208+
209+
Access no-code agent run data:
210+
211+
- `run.input` (`String`) - Input variable
212+
- `run.output` (`String`) - Output variable
213+
- `run.retrieval` (`String`) - Retrieved context based on your selection
214+
- `run.toolCalls` (`JSON`) - Tool calls with name, arguments, and results

0 commit comments

Comments
 (0)