Repository navigation
feat(llm-compressor): 迁移 example 看护到共用引擎 - #220
Merged
Merged
Conversation
香港 runner 直连 huggingface.co，需要登录的模型再回退 ModelScope。定时触发保持关闭，避免和源仓抢同一批 NPU。
每天 04:00 跑一次，避开 accelerate 的 02:00。安装 release 的依赖上限，缓存里只有 config 的快照不再当成命中，也不再改示例的校准数据规模。
昇腾 910B4 没有 FP8 单元，11 条示例在 PR 运行里死于 float8 转换。另有 24 条方案本身要创建 FP8 张量，按方案归类一并移走。块量化示例与上游库在 0.14.0 对不上，同样移走。supported 从 70 条降到 34 条。 视觉示例补装 CPU 版 torchvision，运行时关闭输出缓冲以便定位卡住的位置，并删除不再使用的推理类别代码。
126 个 unsupported 条目各自带一段简体中文注释，写明具体证据：写死 cuda 的行号、模型实测权重大小、FP8 失败的调用栈与运行日志。体积没超限的条目改按真实原因写：6 条是 FP8，3 条是从未在真机验证。 gemma3 示例用 batch 4 且没有自带整理器，在 CPU 数据加载阶段就失败，与设备无关，从 supported 移到 unsupported。supported 降到 33 条。
run_guard.py 改为以 __main__ 运行示例并关闭 datasets 的 map 缓存复用，避免 qwen3_vl 误读 qwen2_vl 留下的 14 像素切块缓存。autoround 的 mxfp4 示例校准加生成总耗时约 275 分钟，超时上调到 360 分钟。 22 条在 0.14.0 实测阻塞的示例移入 unsupported 并逐条附上源码行号、昇腾错误码与运行日志：10 条死于 GPTQModifier 的 prepare_batch 用 torch.stack 的 out 参数做跨精度转换，8 条双卡示例死于 compressed_tensors 的 init_dist 未适配 npu，2 条死于 Hadamard 矩阵乘使用 float64，1 条死于 idefics3 对 bfloat16 调 bucketize，1 条死于 phi3_vision 未向 oneshot 传 processor。supported 收敛为 11 条。
autoround_example.py 对 Qwen3-8B 做 200 轮校准加 MXFP4 伪量化生成 100 个 token，因 compressed_tensors 的 triton_req 未放行 npu，回退到 cast_to_fp4 的 8 次布尔掩码动态索引赋值，实测跑满 180 分钟与 210 分钟仍未完成。移入 unsupported 并附上源码行号与运行日志，supported 收敛为 10 条。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan