Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api调用场景下,如果大量调用带有图片或视频,产生的聊天记录会导致后台mongo数据库异常 #3964

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

gaord
Copy link
Contributor

@gaord gaord commented Mar 4, 2025

程序api调用场景下,如果大量调用带有图片或视频,产生的聊天记录会导致后台mongo数据库异常。这个修改给api客户端一个禁止生成聊天记录的选项,避免这个后果。

@c121914yu
Copy link
Collaborator

🤖 Generated by deploy action

📦 Preview Image: ghcr.io/labring/fastgpt-pr:2ba711ac2f83b85e1131407ee02951c21c6c05a7

@c121914yu
Copy link
Collaborator

是因为传的是 base64导致超出大小么

@gaord
Copy link
Contributor Author

gaord commented Mar 4, 2025

完成几万张照片的大作业,每个照片并不大,几百KB。

@c121914yu
Copy link
Collaborator

具体报错内容是什么?

@gaord
Copy link
Contributor Author

gaord commented Mar 4, 2025

fastgpt后台打不开了。没有看到报错信息。服务器监控显示mongo进程读io满负荷。

@c121914yu
Copy link
Collaborator

c121914yu commented Mar 4, 2025

fastgpt后台打不开了。没有看到报错信息。服务器监控显示mongo进程读io满负荷。

感觉是大量写入的问题,这块直接限制下单次写入感觉更好点。比如超过 1M 的数据,就不保存了。
或者迟点我们再看看有没有分批写入之类的优化。

@gaord
Copy link
Contributor Author

gaord commented Mar 4, 2025

任务量大,需要高并发快速处理才行。每个记录包含的文件300kb左右。如果流程里是两个ai节点处理一个输入图片的话,记录保存的图片就是双份?也许有bug。
在考虑mongo只存储文本,多模态信息在磁盘存储比较好。
现在在API上添加开关也是权宜之计。另外一个办法是在流程里增加一个记录保存设置,满足不同场景的使用需求

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants