[Operator] support instance_norm #296
Conversation
ZQPei
force-pushed
the
pzq/instancenorm
branch
from
042ab0a to
c50e12a
Compare
| @@ -70,9 +88,15 @@ def layernorm_input_fn(shape, dtype, device): | |||
| layernorm_input_fn, | |||
| marks=pytest.mark.layer_norm, | |||
| ), | |||
| pytest.param( | |||
| "instance_norm", | |||
| torch.instance_norm, | |||
There was a problem hiding this comment.
torch.nn.functional.instance_norm
There was a problem hiding this comment.
I found that torch.nn.functional.instance_norm uses a different API than torch.instance_norm.
This API here is exactly torch.instance_norm.
There was a problem hiding this comment.
I thought they are the same.
| eps, | ||
| HAS_WEIGHT_BIAS=has_weight_bias, | ||
| ) | ||
| if has_running_stats and use_input_stats: # update running stats |
There was a problem hiding this comment.
Actually No.
has_running_stats means running_mean and running_var is not None.
use_input_stats means use mean and rstd from input, otherwise uses running_mean and running_var from outside.
Condition when updating running stats, is has_running_stats and use_input_stats be True simultaneously.
There was a problem hiding this comment.
use_input_stats is checked in line 500. so there is no need to check it again.
OK, I will rebase to the latest master soon |
ZQPei
force-pushed
the
pzq/instancenorm
branch
from
a1bcd4a to
3876ce6
Compare
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
Signed-off-by: ZQPei <[email protected]>
ZQPei
force-pushed
the
pzq/instancenorm
branch
from
3876ce6 to
06e0ba4
Compare
Signed-off-by: ZQPei <[email protected]>
PR Category
Operator
Type of Change
New Feature
Description
Support instance norm with forward and backward, w/wo running stats
Issue
Progress
Performance
Environment:
A100 nvlink 80G
pytorch 1.13
cuda 11.8