Skip to content

Commit fd73752

Browse files
committed
Share the observer across dropout, which is an identity in eval mode
Quantized inception_v3 fails to load with the XNNPACK delegate: Failed to create squeeze node 344 with code: xnn_status_invalid_parameter Init failed for backend XnnpackBackend: 0x1 XNNPACK refuses a reshape whose input and output carry different quantization scales. The reshape here sits between two, because the operator feeding it was never given the scale of the operator before it. The list of operators that must share their neighbour's observer already holds permute, view, squeeze, flatten and the pooling operators. Dropout was missing. Not training, so it is the identity and cannot change a tensor's range, but the walk that spreads a shared observer only looks one operator back, so it stopped at dropout and left everything after it to be measured on its own. The mismatch itself is not new. Both scales are identical on the previous pin, to the last digit. What is new is that the reshape now ends up inside a delegate, where XNNPACK checks it, because the clone that used to sit in front of it is no longer emitted for an eval-mode dropout. Test Plan: Read the whole list of conditions that make this XNNPACK call return an invalid parameter, then confirmed which one fires: the scales either side of the reshape differ, 0.017268 against 0.007075. Followed the annotation along the tail of the model and found dropout carrying none, which leaves the flatten after it unannotated as well, which is what gives it a fresh scale. Ran the same model on both pins with the same seed and the same source. The two scales come out identical on both, and only the newer one fails to load, which is what shows the mismatch is pre-existing and the delegation changed. Exported and ran the model with this change on the new pin: it loads and runs. On the previous pin it still loads and runs, unchanged. All 57 tests under the XNNPACK quantizer directory pass with this change.
1 parent b630332 commit fd73752

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

‎backends/xnnpack/quantizer/xnnpack_quantizer_utils.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1105,6 +1105,10 @@ def _is_share_obs_or_fq_op(op: Callable) -> bool:
11051105
torch.ops.aten.slice.Tensor,
11061106
torch.ops.aten.slice_copy.Tensor,
11071107
torch.ops.aten.flatten.using_ints,
1108+
# Identity in eval mode, so it cannot change the tensor's range. Listing it
1109+
# keeps propagate_annotation from stopping here and giving the ops after it
1110+
# their own observers.
1111+
torch.ops.aten.dropout.default,
11081112
]
11091113

11101114

0 commit comments

Comments
 (0)