Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot use name field in Chat Completion messages (CoreMessage, LanguageModelV1Message) #2198

Open
AaronFriel opened this issue Jul 8, 2024 · 0 comments · May be fixed by #2199
Open

Cannot use name field in Chat Completion messages (CoreMessage, LanguageModelV1Message) #2198

AaronFriel opened this issue Jul 8, 2024 · 0 comments · May be fixed by #2199
Labels
ai/core enhancement New feature or request

Comments

@AaronFriel
Copy link
Contributor

Description

In the previous Message type, it was possible to set both role and name fields. The name field was erroneously removed, though it serves an important function in identifying multiple users, or assistant messages from different contexts. The tokenizers for chat and instruction tuned models may contain a special token for the separator between a role and the name.

For our use case, we use the "name" field to label messages from the assistant that contain guardrails. We then prompt the model - obviously this is not 100% reliable, but it is at least somewhat effective - to not reveal these messages. We also use the name field to label assistant messages that may be the result of other agents.

This change has broken compatibility with previous versions of the AI SDK.

Related: most OpenAI models (since GPT-3.5-0613) will tokenize both the role and name, though the exact format is not specified. Many other models accept arbitrary "role" fields, though instruct/chat tuning may strongly benefit from certain roles being used, or roles that contain the token for system, assistant, or user, such as "user (Friel)" or "user-friel".

Code example

No response

Additional context

No response
AaronFriel added a commit to AaronFriel/ai that referenced this issue Jul 8, 2024
@AaronFriel
fix (ai): Accept name on CoreMessage, pass to providers
79ecc0f 
Adds an optional `name` field to the relevant message types in the core package,
provider SDK, and the OpenAI provider. A review of the current providers found
no others that accept a `name` field, so no other providers are modified.

OpenAI's API is leading the way here, whether for multi-user or multi-agent
interactions, LLM APIs are likely to accept named participants beyond 'user',
'assistant', etc.

As described in vercel#2198, the `name` field may be used with OpenAI's API to provide
additional context for system and assistant messages, e.g.: for guardrails and
reducing the risks of prompt injection.

Fixes vercel#2198
@AaronFriel AaronFriel linked a pull request Jul 8, 2024 that will close this issue
Open
@lgrammel lgrammel added enhancement New feature or request ai/core labels Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ai/core enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix (ai): Accept name on CoreMessage, pass to providers AaronFriel/ai
2 participants
@lgrammel @AaronFriel