Repository navigation
Commit 2f2e82b
committed
Only share the observer for a dropout that is not training
Dropout was added to the list of operators that take their input's observer, and
that list matches on the operator alone. Dropout is only an identity when it is not
training. While training it scales the values it keeps, so its output range is wider
than its input's and a shared observer measures the wrong range. The list also
matches one spelling, so the in-place form was left with the original problem.
Read the training argument off the node, and accept both spellings.
Test Plan:
Confirmed the argument is there to read: exporting a module with the dropout in eval
gives a node whose arguments end in false, and in training gives one ending in true.
Confirmed the in-place form is a distinct operator that survives export, in both
modes, so the previous list genuinely missed it.
Measured the effect on a convolution, dropout, reshape model prepared for
quantization-aware training. With the operator-only list the graph comes out with
five fake-quantize slots collapsed onto three objects, so the training dropout is
sharing. With this change the same graph keeps three slots and three objects, so it
is not.
Checked the predicate over all four combinations: both spellings share when not
training and do not share when training.
Eval-mode sharing is unchanged, which is what the reshape needed, and the operators
already on the list are untouched.1 parent 774bcaf commit 2f2e82b
1 file changed
Lines changed: 16 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1018 | 1018 | | |
1019 | 1019 | | |
1020 | 1020 | | |
1021 | | - | |
| 1021 | + | |
1022 | 1022 | | |
1023 | 1023 | | |
1024 | 1024 | | |
| |||
1105 | 1105 | | |
1106 | 1106 | | |
1107 | 1107 | | |
1108 | | - | |
1109 | | - | |
1110 | 1108 | | |
1111 | 1109 | | |
1112 | 1110 | | |
| 1111 | + | |
| 1112 | + | |
| 1113 | + | |
| 1114 | + | |
| 1115 | + | |
| 1116 | + | |
| 1117 | + | |
| 1118 | + | |
| 1119 | + | |
| 1120 | + | |
| 1121 | + | |
| 1122 | + | |
| 1123 | + | |
| 1124 | + | |
1113 | 1125 | | |
1114 | 1126 | | |
1115 | | - | |
| 1127 | + | |
1116 | 1128 | | |
1117 | 1129 | | |
1118 | 1130 | | |
| |||
0 commit comments