Conditional Likelihood Loss

[MahdiShAhmadi]

Is there any way we can have a conditional likelihood loss like the following? x=<x1,x2> is the input.

Forward pass

lls_all = pc(x)
lls_x2 = pc(x2) # assuming we can handle missing values in training loop (?)
lls_x1_given_x2 = lls_all - lls_x2

Backward pass

lls_x1_given_x2.mean().backward()

Even if we could do this in code, does it make theoretical sense in the E-M algorithm?

[liuanji]

Hi Mahdi,

Autodiff may not work in this case, but there's a simple workaround: we can simply do two manual forward and backward passes and set negate_pflows=True to the second one (that corresponds to lls_x2):

lls_all = pc(x)
pc.backward(x)

lls_x2 = pc(x2)
pc.backward(x2, negate_pflows=True)

In this way, the parameter flows will contain \partial (lls_all - lls_x2) / \partial log(params) (note that the "usual" flow is \partial lls / \partial log(params)). I'll work on making the autodiff code you provide work.

Missing value masks are supported in pyjuice. Simply add an argument missing_mask of size [batch_size, num_vars] to both the forward and the backward pass:

pc(x, missing_mask=missing_mask)
pc.backward(x, missing_mask=missing_mask)

For optimization, I don't think EM makes sense in this case. I plan to add support for gradient-based optimization methods. I'll let you know when there's anything new.

Best,
Anji

[MahdiShAhmadi]

Thanks a lot, Anji — that makes more sense. Appreciate the workaround and the clarification on missing masks and EM. Looking forward to the gradient-based optimization update!