Conversation
for more information, see https://pre-commit.ci
environments/gsm8k_server.py
Outdated
| ) | ||
|
|
||
| async with self.server.managed_server(tokenizer=self.tokenizer) as managed: | ||
| logger.warning( |
environments/gsm8k_server.py
Outdated
| max_tokens=self.config.max_token_length, | ||
| temperature=1.0, | ||
| ) | ||
| logger.warning( |
environments/gsm8k_server.py
Outdated
|
|
||
| state = managed.get_state() | ||
| nodes = state["nodes"] | ||
| logger.warning( |
example_trainer/api.py
Outdated
| "X-Atropos-Client": "trainer", | ||
| "X-Atropos-Pid": str(os.getpid()), |
There was a problem hiding this comment.
This was a sanity check - it has been removed
|
|
||
| if isinstance(default_server_configs, APIServerConfig): | ||
| server_configs = final_openai_config | ||
| server_configs = [final_openai_config] |
There was a problem hiding this comment.
I mean, you're not supposed to pass this in like that
| teacher_server: Optional[APIServerConfig] = Field( | ||
| default=None, | ||
| description="Teacher inference server configuration.", | ||
| ) |
There was a problem hiding this comment.
ah, i probably commented poorly, it should be the same as how we setup the server_manager, so we may need to pass in a new thing to init
There was a problem hiding this comment.
Updated this to follow this pattern. I removed teacher_server from TeacherDistillationConfig, so the env config now only carries env-level knobs like teacher_enabled and teacher_top_k. Teacher server wiring is now passed separately via teacher_server_configs at init
| "teacher_top_k", self.config.teacher_top_k | ||
| ) | ||
| ) | ||
| top_k = max(1, top_k) |
There was a problem hiding this comment.
max should be 0, because prompt logprobs are (selected token + topk), disabled would be setting it to -1 or lower. I would also be amenable to a group override that's skip_teacher_top_k
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
2 participants
No description provided.