Question about the KL divergence loss

[marctimjen]

Hello

I hope someone can help me understand why the KL is calculated as:
0.5 * torch.sum(torch.pow(self.mean, 2) + self.var - 1.0 - self.logvar, dim=[1, 2, 3])

In the DiagonalGaussianDistribution listed here:

stable-diffusion/ldm/modules/distributions/distributions.py

Line 44 in 21f890f

return 0.5 * torch.sum(torch.pow(self.mean, 2)

I am asking, because most loss functions for the VAE I can find use (-1 times this calculations) like this: 0.5 * torch.sum(-torch.pow(self.mean, 2) - self.var + 1.0 + self.logvar, dim=[1, 2, 3])

And I cannot see that we multiply by -1 in the contperceptual loss for instance:

https://github.com/CompVis/stable-diffusion/blob/21f890f9da3cfbeaba8e2ac3c425ee9e998d5229/ldm/modules/losses/contperceptual.py#L83C57-L83C65

Thank you very much in advance :)

[marctimjen]

I found this material that does also have the loss on the form that is used here:

https://pyimagesearch.com/2023/10/02/a-deep-dive-into-variational-autoencoders-with-pytorch/

My confusion just happen because most papers write:

(From Bishops Deep learning)

And the original implementation of VAE: