separableConv2d and depthwiseConv2d incorrectly initialize depthwise convolutional weights

[vmukhachev]

This affects initializers like 'glorotUniform' and 'glorotNormal'.

I think computeFans (see https://github.com/tensorflow/tfjs/blob/master/tfjs-layers/src/initializers.ts ) is incorrect:

• fan_in should be: kernelHeight × kernelWidth

Reproducible Example:

const input = tf.randomNormal([1, 64, 64, 64]);
const dw = tf.layers.depthwiseConv2d({
  kernelSize: 3,
  useBias: false,
  depthwiseInitializer: 'leCunUniform'
});
const output = dw.apply(input);
console.log(`Output mean=${tf.moments(output).mean.arraySync().toFixed(3)}, variance=${tf.moments(output).variance.arraySync().toFixed(3)}`);
// Output mean=-0.000, variance=0.016

Corrected Behavior Example:

const input = tf.randomNormal([1, 64, 64, 64]);
const dw = tf.layers.depthwiseConv2d({
  kernelSize: 3,
  useBias: false,
  depthwiseInitializer: tf.initializers.randomUniform({minval: -Math.sqrt(3 / (3**2)), maxval: Math.sqrt(3 / (3**2))})
});
const output = dw.apply(input);
console.log(`Output mean=${tf.moments(output).mean.arraySync().toFixed(3)}, variance=${tf.moments(output).variance.arraySync().toFixed(3)}`);
// Output mean=-0.004, variance=0.964

[Reut2211]

In your code, both the input and the kernel are initialized with random values centered around 0 — the mean of the input is 0 by default in tf.randomNormal, and the leCunUniform initializer also generates weights with a mean close to 0.
Since the convolution operation is effectively a cross-correlation between the input and the kernel weights, the output values are also expected to be centered around 0.

If you want the output to have a different mean, you can generate input values with a different mean and standard deviation, for example:
const input = tf.randomNormal([1, 64, 64, 64], /mean=/2.0, /stdDev=/1.0);

This will likely result in an output mean greater than 0.

Alternatively, if you want to keep the same input distribution but print more precise statistics, you can increase the number of decimal digits displayed using .toFixed():

console.log(Output mean=${tf.moments(output).mean.arraySync().toFixed(6)}, variance=${tf.moments(output).variance.arraySync().toFixed(3)});

[vmukhachev]

@Reut2211 , I want to preserve the variance of the input when applying the layer. I want this because i think this is the purpose of the LeCun's weights initialization.