Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions docs/prompt-compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -283,11 +283,11 @@ OpenAI 的文件上传现在不再是“只传文件本体”的通用路径,

兼容层现在只保留 `current_input_file` 这一种拆分方式;旧的 `history_split` 配置字段已移除,读取旧配置时会忽略它且不会再写回。

- `current_input_file` 默认开启;它在统一 completion runtime 入口全局生效,并采用 inline-first 策略。runtime 会先按拆分前的完整 `PromptTokenText` 估算整体上下文 token;当整体上下文不超过 `current_input_file.inline_max_tokens`(默认 `30000`)时,请求保持完整 inline prompt,不上传生成的上下文或工具文件。只有最新 user turn 的纯文本长度达到 `current_input_file.min_chars`(默认 `0`)且整体上下文超过 inline 阈值时,runtime 才会把“完整上下文”合并进 `DS2API_HISTORY.txt` 上下文文件。文件内容会先经过各协议入口的标准化,再序列化成按轮次编号的 `DS2API_HISTORY.txt` 风格 transcript,带有 `# DS2API_HISTORY.txt` 标题和 `=== N. ROLE ===` 分段;如果当前请求带有可用 tools 且 tool choice 不是 `none`,runtime 还会上传 `DS2API_TOOLS.txt`,只承载本次请求的工具名称、描述和参数 schema。live prompt 中则会给出一个中性的 continuation user 消息,要求模型使用附加 conversation context 作为当前工作状态并直接回答最新请求;如果有工具文件,live prompt 会额外说明附加 tool reference 是本轮可用工具和 schema 的权威来源,避免把任务拉回起点,也避免把大段 schema 再次内联到 prompt。
- `current_input_file.filename_policy` 控制生成文件的可见文件名策略,合法值为 `legacy`、`neutral`、`neutral_random`,默认 `neutral_random`。`legacy` 保留旧文件名用于显式回滚;`neutral` / `neutral_random` 用于降低模型可见的实现细节。
- `current_input_file` 默认开启;它在统一 completion runtime 入口全局生效,并采用 inline-first 策略。runtime 会先按拆分前的完整 `PromptTokenText` 估算整体上下文 token;当整体上下文不超过 `current_input_file.inline_max_tokens`(默认 `30000`)时,请求保持完整 inline prompt,不上传生成的上下文或工具文件。只有最新 user turn 的纯文本长度达到 `current_input_file.min_chars`(默认 `0`)且整体上下文超过 inline 阈值时,runtime 才会把“完整上下文”合并进生成的 conversation context 文件。文件内容会先经过各协议入口的标准化,再序列化成按轮次编号的 transcript,当前正文标题仍为 `# DS2API_HISTORY.txt`;如果当前请求带有可用 tools 且 tool choice 不是 `none`,runtime 还会上传 tool reference 文件,只承载本次请求的工具名称、描述和参数 schema。live prompt 中则会给出一个中性的 continuation user 消息,要求模型使用附加 conversation context 作为当前工作状态并直接回答最新请求;如果有工具文件,live prompt 会额外说明附加 tool reference 是本轮可用工具和 schema 的权威来源,避免把任务拉回起点,也避免把大段 schema 再次内联到 prompt。
- `current_input_file.filename_policy` 控制生成文件的可见文件名策略,合法值为 `legacy`、`neutral`、`neutral_random`,默认 `neutral_random`。`legacy` 上传 `DS2API_HISTORY.txt` / `DS2API_TOOLS.txt` 用于显式回滚;`neutral` 上传 `conversation-notes.txt` / `tool-reference.txt`;`neutral_random` 上传 `conversation-notes-xxxxxx.txt` / `tool-reference-xxxxxx.txt`,用于降低模型可见的实现细节。
- 如果 `current_input_file.enabled=false`,请求会直接透传,不上传任何拆分上下文文件。
- 对客户端回包里的上下文 token 统计,短上下文 inline 路径直接按完整 prompt 计数;触发 `current_input_file` 后 live prompt 被缩短时,仍会沿用**拆分前的完整 prompt 语义**做计数,而不是按缩短后的占位 prompt 计算;否则会把真实上下文显著算小。
- 如果空输出重试最终需要切换托管账号,runtime 会在新账号下重新上传已生成的 `DS2API_HISTORY.txt` 和可选 `DS2API_TOOLS.txt`,再用新的 generated `file_id` 重建 completion payload;客户端原有文件引用会保留在 generated 文件之后。这样可以避免把旧账号不可见的 generated file_id 带到新账号请求里。
- 如果空输出重试最终需要切换托管账号,runtime 会在新账号下重新上传已生成的 conversation context 和可选 tool reference,再用新的 generated `file_id` 重建 completion payload;客户端原有文件引用会保留在 generated 文件之后。这样可以避免把旧账号不可见的 generated file_id 带到新账号请求里。
- runtime 会为 `DS2API_HISTORY.txt`、`DS2API_TOOLS.txt` 和 live prompt 计算稳定 SHA-256 内容 hash,并写入 completion request 日志字段;`DS2API_TOOLS.txt` 还会使用账号隔离、`model_type` 隔离、内容 hash 隔离的 5 分钟内存缓存。同一账号在短时间内复用完全相同的工具 schema 时,会复用已上传的 tools `file_id`;`DS2API_HISTORY.txt` 只记录 hash,不做 file_id 缓存,避免把长对话历史大量留存在本地内存。

相关实现:
Expand All @@ -299,10 +299,10 @@ OpenAI 的文件上传现在不再是“只传文件本体”的通用路径,
- 全局 completion runtime 应用点:
[internal/completionruntime/nonstream.go](../internal/completionruntime/nonstream.go)

当前输入转文件启用并触发时,上传的历史文件真实文件名是 `DS2API_HISTORY.txt`,文件内容是完整 `messages` 上下文;它会使用 OpenAI-compatible 的消息/transcript 序列化规则和 DeepSeek 角色标记,再按轮次编号成 `DS2API_HISTORY.txt` 风格的 transcript(不再注入文件边界标签):
当前输入转文件启用并触发时,上传的历史文件名由 `filename_policy` 决定,文件内容是完整 `messages` 上下文;它会使用 OpenAI-compatible 的消息/transcript 序列化规则和 DeepSeek 角色标记,再按轮次编号成 transcript(不再注入文件边界标签):

```text
[uploaded filename]: DS2API_HISTORY.txt
[uploaded filename]: conversation-notes-a1b2c3.txt
# DS2API_HISTORY.txt
Prior conversation history and tool progress.

Expand All @@ -319,10 +319,10 @@ Prior conversation history and tool progress.
...
```

如果当前请求带有可用 tools,还会额外上传 `DS2API_TOOLS.txt`
如果当前请求带有可用 tools,还会额外上传 tool reference

```text
[uploaded filename]: DS2API_TOOLS.txt
[uploaded filename]: tool-reference-d4e5f6.txt
# DS2API_TOOLS.txt
Available tool descriptions and parameter schemas for this request.

Expand All @@ -333,13 +333,13 @@ Description: Search docs
Parameters: {"type":"object",...}
```

开启后,请求的 live prompt 不再直接内联完整上下文和大段工具 schema,而是保留一个 user role 的短提示,提示模型基于已提供上下文直接回答最新请求;system prompt 中仍保留 DSML tool call 格式约束。上传后的生成文件 `file_id` 会排在 `ref_file_ids` 前部,顺序是 `DS2API_HISTORY.txt`、可选的 `DS2API_TOOLS.txt`,再接客户端原有文件引用。
开启后,请求的 live prompt 不再直接内联完整上下文和大段工具 schema,而是保留一个 user role 的短提示,提示模型基于已提供上下文直接回答最新请求;system prompt 中仍保留 DSML tool call 格式约束。上传后的生成文件 `file_id` 会排在 `ref_file_ids` 前部,顺序是 conversation context、可选的 tool reference,再接客户端原有文件引用。

缓存边界:

- 缓存 key 包含账号 scope、`model_type`、文件名和内容 hash,不跨账号复用 generated `file_id`
- 只缓存 `DS2API_TOOLS.txt` 的 `file_id`,因为工具 schema 在写代码 / agent 场景中高度重复
- 不缓存 `DS2API_HISTORY.txt` 的 `file_id`,因为历史内容通常随每轮请求变化且可能很大
- 缓存 key 包含账号 scope、`model_type`、逻辑文件类型和内容 hash,不跨账号复用 generated `file_id`
- 只缓存 tool reference 的 `file_id`,因为工具 schema 在写代码 / agent 场景中高度重复
- 不缓存 conversation context 的 `file_id`,因为历史内容通常随每轮请求变化且可能很大
- 日志字段包括 `current_input_history_hash`、`current_input_tools_hash`、`current_input_prompt_hash`、`current_input_cache_hits`、`current_input_cache_misses` 和 `current_input_ref_count`

## 10. 各协议入口的差异
Expand Down
36 changes: 36 additions & 0 deletions internal/httpapi/openai/history/current_input_cache_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package history
import (
"context"
"fmt"
"regexp"
"strings"
"testing"

Expand All @@ -24,6 +25,16 @@ func (inlineFirstTestStore) CurrentInputFileMinChars() int { return 0 }
func (inlineFirstTestStore) CurrentInputFileInlineMaxTokens() int { return 30000 }
func (inlineFirstTestStore) ContextEngineMode() string { return "off" }

type filenamePolicyTestStore struct {
policy string
}

func (s filenamePolicyTestStore) CurrentInputFileEnabled() bool { return true }
func (s filenamePolicyTestStore) CurrentInputFileMinChars() int { return 0 }
func (s filenamePolicyTestStore) CurrentInputFileInlineMaxTokens() int { return 1 }
func (s filenamePolicyTestStore) CurrentInputFileFilenamePolicy() string { return s.policy }
func (s filenamePolicyTestStore) ContextEngineMode() string { return "off" }

type currentInputCacheTestDS struct {
uploads []dsclient.UploadFileRequest
}
Expand Down Expand Up @@ -55,6 +66,31 @@ func TestApplyCurrentInputFileKeepsShortContextInline(t *testing.T) {
}
}

func TestApplyCurrentInputFileUsesNeutralRandomFilenames(t *testing.T) {
ResetCurrentInputToolsFileCacheForTesting()
ds := &currentInputCacheTestDS{}
svc := Service{Store: filenamePolicyTestStore{policy: "neutral_random"}, DS: ds}

_, err := svc.ApplyCurrentInputFile(context.Background(), &auth.RequestAuth{DeepSeekToken: "token"}, cacheTestStandardRequest())
if err != nil {
t.Fatalf("apply failed: %v", err)
}
if len(ds.uploads) != 2 {
t.Fatalf("expected history and tools uploads, got %d", len(ds.uploads))
}
historyName := ds.uploads[0].Filename
toolsName := ds.uploads[1].Filename
if !regexp.MustCompile(`^conversation-notes-[0-9a-f]{6}\.txt$`).MatchString(historyName) {
t.Fatalf("unexpected neutral random history filename: %q", historyName)
}
if !regexp.MustCompile(`^tool-reference-[0-9a-f]{6}\.txt$`).MatchString(toolsName) {
t.Fatalf("unexpected neutral random tools filename: %q", toolsName)
}
if strings.Contains(historyName, "DS2API") || strings.Contains(toolsName, "DS2API") {
t.Fatalf("generated filenames should not expose implementation terms: history=%q tools=%q", historyName, toolsName)
}
}

func TestApplyCurrentInputFileCachesToolsFileByStableHash(t *testing.T) {
ResetCurrentInputToolsFileCacheForTesting()
ds := &currentInputCacheTestDS{}
Expand Down
77 changes: 71 additions & 6 deletions internal/httpapi/openai/history/current_input_file.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ package history

import (
"context"
"crypto/rand"
"encoding/hex"
"errors"
"fmt"
"strings"
Expand All @@ -22,6 +24,13 @@ const (
currentInputPurpose = "assistants"
)

type generatedCurrentInputKind string

const (
generatedCurrentInputContext generatedCurrentInputKind = "context"
generatedCurrentInputTools generatedCurrentInputKind = "tools"
)

type CurrentInputConfigReader interface {
CurrentInputFileEnabled() bool
CurrentInputFileMinChars() int
Expand All @@ -32,6 +41,10 @@ type currentInputInlineMaxTokensReader interface {
CurrentInputFileInlineMaxTokens() int
}

type currentInputFilenamePolicyReader interface {
CurrentInputFileFilenamePolicy() string
}

type CurrentInputUploader interface {
UploadFile(ctx context.Context, a *auth.RequestAuth, req dsclient.UploadFileRequest, maxAttempts int) (*dsclient.UploadFileResult, error)
}
Expand Down Expand Up @@ -71,7 +84,9 @@ func (s Service) ApplyCurrentInputFile(ctx context.Context, a *auth.RequestAuth,
if resolvedType, ok := config.GetModelType(stdReq.ResolvedModel); ok {
modelType = resolvedType
}
fileID, err := s.uploadGeneratedFile(ctx, a, currentInputFilename, modelType, fileText)
historyFilename := currentInputGeneratedFilename(s.Store, generatedCurrentInputContext)
toolsFilename := currentInputGeneratedFilename(s.Store, generatedCurrentInputTools)
fileID, err := s.uploadGeneratedFile(ctx, a, historyFilename, modelType, fileText)
if err != nil {
return stdReq, fmt.Errorf("upload current user input file: %w", err)
}
Expand All @@ -84,7 +99,7 @@ func (s Service) ApplyCurrentInputFile(ctx context.Context, a *auth.RequestAuth,
cacheMisses := 0
if strings.TrimSpace(toolsText) != "" {
var err error
toolFileID, toolCacheHit, err = s.uploadCachedToolsFile(ctx, a, modelType, toolsText, toolsHash)
toolFileID, toolCacheHit, err = s.uploadCachedToolsFile(ctx, a, toolsFilename, modelType, toolsText, toolsHash)
if err != nil {
return stdReq, fmt.Errorf("upload current tools file: %w", err)
}
Expand Down Expand Up @@ -163,6 +178,54 @@ func currentInputInlineMaxTokens(store CurrentInputConfigReader) int {
return reader.CurrentInputFileInlineMaxTokens()
}

func currentInputGeneratedFilename(store CurrentInputConfigReader, kind generatedCurrentInputKind) string {
policy := currentInputFilenamePolicy(store)
switch policy {
case "neutral":
return neutralCurrentInputFilename(kind)
case "neutral_random":
return randomizedCurrentInputFilename(kind)
default:
return legacyCurrentInputFilename(kind)
}
}

func currentInputFilenamePolicy(store CurrentInputConfigReader) string {
reader, ok := store.(currentInputFilenamePolicyReader)
if !ok {
return "legacy"
}
return config.NormalizeCurrentInputFileFilenamePolicy(reader.CurrentInputFileFilenamePolicy())
}

func legacyCurrentInputFilename(kind generatedCurrentInputKind) string {
if kind == generatedCurrentInputTools {
return currentToolsFilename
}
return currentInputFilename
}

func neutralCurrentInputFilename(kind generatedCurrentInputKind) string {
if kind == generatedCurrentInputTools {
return "tool-reference.txt"
}
return "conversation-notes.txt"
}

func randomizedCurrentInputFilename(kind generatedCurrentInputKind) string {
base := neutralCurrentInputFilename(kind)
stem := strings.TrimSuffix(base, ".txt")
return stem + "-" + randomFilenameSuffix() + ".txt"
}

func randomFilenameSuffix() string {
var b [3]byte
if _, err := rand.Read(b[:]); err != nil {
return "local"
}
return hex.EncodeToString(b[:])
}

func (s Service) ReuploadAppliedCurrentInputFile(ctx context.Context, a *auth.RequestAuth, stdReq promptcompat.StandardRequest) (promptcompat.StandardRequest, error) {
if !stdReq.CurrentInputFileApplied || s.DS == nil || a == nil {
return stdReq, nil
Expand All @@ -176,7 +239,9 @@ func (s Service) ReuploadAppliedCurrentInputFile(ctx context.Context, a *auth.Re
if resolvedType, ok := config.GetModelType(stdReq.ResolvedModel); ok {
modelType = resolvedType
}
fileID, err := s.uploadGeneratedFile(ctx, a, currentInputFilename, modelType, stdReq.HistoryText)
historyFilename := currentInputGeneratedFilename(s.Store, generatedCurrentInputContext)
toolsFilename := currentInputGeneratedFilename(s.Store, generatedCurrentInputTools)
fileID, err := s.uploadGeneratedFile(ctx, a, historyFilename, modelType, stdReq.HistoryText)
if err != nil {
return stdReq, fmt.Errorf("upload current user input file: %w", err)
}
Expand All @@ -195,7 +260,7 @@ func (s Service) ReuploadAppliedCurrentInputFile(ctx context.Context, a *auth.Re
cacheMisses := 0
if strings.TrimSpace(toolsText) != "" {
var err error
toolFileID, toolCacheHit, err = s.uploadCachedToolsFile(ctx, a, modelType, toolsText, toolsHash)
toolFileID, toolCacheHit, err = s.uploadCachedToolsFile(ctx, a, toolsFilename, modelType, toolsText, toolsHash)
if err != nil {
return stdReq, fmt.Errorf("upload current tools file: %w", err)
}
Expand Down Expand Up @@ -244,7 +309,7 @@ func (s Service) uploadGeneratedFile(ctx context.Context, a *auth.RequestAuth, f
return strings.TrimSpace(result.ID), nil
}

func (s Service) uploadCachedToolsFile(ctx context.Context, a *auth.RequestAuth, modelType, text, hash string) (string, bool, error) {
func (s Service) uploadCachedToolsFile(ctx context.Context, a *auth.RequestAuth, filename, modelType, text, hash string) (string, bool, error) {
key := generatedFileCacheKey{
AccountScope: currentInputCacheScope(a),
ModelType: strings.TrimSpace(modelType),
Comment on lines +312 to 315

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include filename policy in tools cache key

uploadCachedToolsFile now accepts a generated filename, but the cache lookup key still hard-codes currentToolsFilename. If current_input_file.filename_policy is changed at runtime (for example from legacy to neutral_random), requests within the 5-minute cache TTL can reuse a previously uploaded file_id whose visible filename does not match the new policy, so the policy change does not take effect immediately and can still expose DS2API_* names. The cache key should include a stable policy dimension (or non-randomized filename class) so cache hits respect the active filename policy.

Useful? React with 👍 / 👎.

Expand All @@ -254,7 +319,7 @@ func (s Service) uploadCachedToolsFile(ctx context.Context, a *auth.RequestAuth,
if fileID, ok := currentInputToolsFileCache.lookup(key); ok {
return fileID, true, nil
}
fileID, err := s.uploadGeneratedFile(ctx, a, currentToolsFilename, modelType, text)
fileID, err := s.uploadGeneratedFile(ctx, a, filename, modelType, text)
if err != nil {
return "", false, err
}
Expand Down