fix: use named temp file for writability probe to avoid EIO on FUSE/NAS - #1774
Merged
Merged
Conversation
SAKURA-CAT
requested changes
Sep 21, 2026
SAKURA-CAT
approved these changes
Sep 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
修复
safe_mkdir可写性探针在 FUSE/NAS 文件系统上误报EIO导致训练崩溃的问题。根因: 探针使用
tempfile.TemporaryFile,其依赖匿名临时文件(O_TMPFILE)语义。部分 FUSE/NAS 实现不支持该语义,且返回的是EIO而非可触发 fallback 的错误码,Python 不会降级处理。探针把这个硬错误当作「NAS 同步延迟」不断重试,最终撞 5 秒超时抛TimeoutError,训练进程挂掉。而普通命名文件写入在这类存储上完全正常。改动:
swanlab/sdk/internal/pkg/fs/dir.pytempfile.TemporaryFile改为命名临时文件(mkstemp→ 写入 → 关闭 → 删除),新增_probe_writable()TimeoutError通过raise ... from last_error链上最后一次OSError,避免硬错误被误报为单纯超时Releated