Skip to content

Commit 2f2e82b

Browse files
committed
Only share the observer for a dropout that is not training
Dropout was added to the list of operators that take their input's observer, and that list matches on the operator alone. Dropout is only an identity when it is not training. While training it scales the values it keeps, so its output range is wider than its input's and a shared observer measures the wrong range. The list also matches one spelling, so the in-place form was left with the original problem. Read the training argument off the node, and accept both spellings. Test Plan: Confirmed the argument is there to read: exporting a module with the dropout in eval gives a node whose arguments end in false, and in training gives one ending in true. Confirmed the in-place form is a distinct operator that survives export, in both modes, so the previous list genuinely missed it. Measured the effect on a convolution, dropout, reshape model prepared for quantization-aware training. With the operator-only list the graph comes out with five fake-quantize slots collapsed onto three objects, so the training dropout is sharing. With this change the same graph keeps three slots and three objects, so it is not. Checked the predicate over all four combinations: both spellings share when not training and do not share when training. Eval-mode sharing is unchanged, which is what the reshape needed, and the operators already on the list are untouched.
1 parent 774bcaf commit 2f2e82b

1 file changed

Lines changed: 16 additions & 4 deletions

File tree

‎backends/xnnpack/quantizer/xnnpack_quantizer_utils.py‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1018,7 +1018,7 @@ def _has_annotated_qparam_source(node: Node) -> bool:
10181018
):
10191019
return True
10201020

1021-
if node.op != "call_function" or not _is_share_obs_or_fq_op(node.target):
1021+
if node.op != "call_function" or not _shares_observer_with_input(node):
10221022
return False
10231023

10241024
prev_node = node.args[0]
@@ -1105,14 +1105,26 @@ def _is_share_obs_or_fq_op(op: Callable) -> bool:
11051105
torch.ops.aten.slice.Tensor,
11061106
torch.ops.aten.slice_copy.Tensor,
11071107
torch.ops.aten.flatten.using_ints,
1108-
# Identity in eval mode, so it must not get its own observer.
1109-
torch.ops.aten.dropout.default,
11101108
]
11111109

11121110

1111+
def _shares_observer_with_input(node: Node) -> bool:
1112+
"""Whether a node must take its input's observer instead of its own.
1113+
1114+
Dropout is only an identity when it is not training; while training it scales
1115+
the values it keeps, so its range is wider than its input's.
1116+
"""
1117+
if node.target in (
1118+
torch.ops.aten.dropout.default,
1119+
torch.ops.aten.dropout_.default,
1120+
):
1121+
return len(node.args) > 2 and node.args[2] is False
1122+
return _is_share_obs_or_fq_op(node.target)
1123+
1124+
11131125
def propagate_annotation(model: torch.fx.GraphModule) -> None:
11141126
for n in model.graph.nodes:
1115-
if n.op != "call_function" or not _is_share_obs_or_fq_op(n.target):
1127+
if n.op != "call_function" or not _shares_observer_with_input(n):
11161128
continue
11171129

11181130
prev_node = n.args[0]

0 commit comments

Comments
 (0)