You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
🔴 Bug: fail() corrupts original test result — report["passed"] is set to False even when the actual probe passed. If the caller uses report["passed"] to gate deployments, a transient report-write failure will falsely block. Use a separate field instead:
deffail():
# Don't mutate the test's semantic result; use a distinct field.report["report_success"] =Falsereport["report_stage"] ="report-write"report["report_error"] ="Incomplete two-layer IR gate evidence"report["report_write_errors"] =list(failures)
🟡 Retry return value ignored (line ~395) — write("report.json", ...) on retry discards its return value. If the retry fails, failures gets the new entry but no terminal action is taken beyond the stderr print. Consider making this explicit:
ifnotwrite("report.json", json.dumps(report, indent=2, allow_nan=False) +"\n"):
fail()
recover_views()
ifnotwrite("report.json", json.dumps(report, indent=2, allow_nan=False) +"\n"):
report["report_write_errors"] =list(failures) # already implicit, but be explicit
🟡 Double fail() call when both md/html and JSON writes fail — fail() sets report_write_errors then recover_views() overwrites it. The second fail() call (for the JSON failure) repeats this cycle. Net result is correct but fragile. Consider guarding:
💭 Exception broadening is correct here — subprocess.SubprocessError now also catches TimeoutExpired (subclass), which is needed since timeout=10 was added. Good.
💭 .git check is sound — handles the nested-checkout edge case well; .exists() covers both directory and file (submodule) cases.
📁 tests/test_onnx_qwen_numeric_ops.py
🔴 Bug: ReferenceEvaluator API 调用方式不正确 — 第 33-34 行:
🟡 DRY: duplicated condition — The opset == 18 and axes is None and noop == 1 check appears twice in test_reduce_mean. Extract to a local use_ref_eval = ... at the top of the workaround branch (or right before _compare); a future edit to one copy will drift from the other.
🟡 ReferenceEvaluator API assumption — ReferenceEvaluator.run(None, feed) relies on the output_names=None overload returning outputs in graph order, which is what ORT's InferenceSession.run supports and what this call mirrors. Worth verifying against the pinned onnx version (onnx.reference.ReferenceEvaluator was added in 1.12 and has historically had signature drift, e.g. requiring explicit output names in some releases). If your CI matrix spans multiple onnx versions, a tiny _run_reference(path, feed) helper that asserts the output count would make the failure mode clearer than a bare TypeError here.
🟡 ReferenceEvaluator vs ORT semantics drift — Even inside the workaround, you're now comparing your IR against a different reference implementation than every other parametrization of the test. The ONNX spec is the oracle, but ReferenceEvaluator can itself have bugs (it's pure-Python and often lags ORT kernels). Since the workaround is intentionally narrow (one opset + one attr combination), consider adding a follow-up comment/tagging the case so it's easy to re-baseline against ORT once the upstream bug is fixed, e.g. # TODO(upstream-onnxruntime#XXXX): drop reference_evaluator.
💭 Comment placement — The ORT bug rationale is currently above the actual = _compare(...) line, so it reads as documenting the assignment rather than the whole workaround block. Consider moving the two-line comment to just above the if opset == 18 ... line so it scopes the entire special case, not just the captured variable.
💭 Type hint / keyword-only consistency — _compare gained a keyword-only param (*, reference_evaluator=False). Good call. If there's any other wrapper in the file using positional args against _compare, a quick grep will confirm nothing broke.
Nothing blocking — the workaround is a reasonable response to a documented upstream kernel bug, and scoping it to one specific (opset, axes, noop) tuple is the right amount of precision.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR #91 合并后,补齐两层探测的证据可靠性:报告写入/关闭/原子替换失败时必须失败,不能留下成功 JSON;源码副本不能借用父目录 Git 身份,QEMU 报告分别记录当前执行环境与原导出环境。
本 PR 仅包含 7 个 W1 文件;W2 完整解析、运行时、后端及 CI 扩展另行提交,修改文件不重叠。基于已合并主干
3bb88e8。修改
.git时明确未知 Git 身份;扩展源码指纹及当前环境归属检查。验证
1.6689300537109375e-6 < 1e-5。git diff --check通过。源码指纹与提交前执行代码一致。关联 #92;不自动关闭接口冻结、完整模型第二人复现或 W1 出口评审待办,也不改变计算算法、模型配置和精度门槛。