Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it wrong at line 147 of word2vec.py, assigment 2? #5

Open
rockqgf opened this issue Oct 9, 2019 · 3 comments
Open

Is it wrong at line 147 of word2vec.py, assigment 2? #5

rockqgf opened this issue Oct 9, 2019 · 3 comments
Labels
bug Something isn't working

Comments

@rockqgf
Copy link

rockqgf commented Oct 9, 2019

I think the code "gradOutsideVecs[negSampleWordIndices] += np.outer((z-1),centerWordVec)*(-1)", at line 147 of word2vec.py, assigment 2, is not right.
Because as the annotation says, "Note: The same word may be negatively sampled multiple times. For example if an outside word is sampled twice, you shall have to double count the gradient with respect to this word."
I write testing codes, gradOutsideVecs[negSampleWordIndices] could not double the gradient, it only add one time(the last one).

@ZacBi ZacBi added the bug Something isn't working label Oct 20, 2019
@Lalalaashen
Copy link

It should be

for i, negSampleWordIdx in enumerate(negSampleWordIndices):
        gradOutsideVecs[negSampleWordIdx] += (1.0 - z[i]) * centerWordVec

@Zhuifeng414
Copy link

Zhuifeng414 commented May 11, 2020

really help! @Lalalaashen @rockqgf

@naduhrin78
Copy link

Use np.add.at() to avoid for loop, lets keep integrity of vectorization ^^. kudos.

Replace line 147 with:
np.add.at(gradOutsideVecs, negSampleWordIndices, np.outer((z-1),centerWordVec)*(-1))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants