matching Chinese sentence

[etrigger]

Is it possible to compute the similarity between an image and a Chinese sentence?
Which part should I change accordingly?
Thanks.

[M-Eng]

Hello, the current model is not trained for Chinese sentence but there is multiple way you could modify it to work for Chinese.

The most straightforward is to change the dataset to have pairs of images associated with Chinese captions (maybe translating coco?)

The second option would be to start from the current model already trained and train a second textual encoder for Chinese using only textual data of English/Chinese translation.

There might be other methods, but those two should work,
Good luck

[HuDi2018]

Is it possible to compute the similarity between an image and a Chinese sentence?
Which part should I change accordingly?
Thanks.

I have the chinese version of caption retrieval algorithm

[zhufeijuanjuan]

I tried a chinese version with image caption dataset from ai challenger 2017. The loss becomes nan after few steps in eopch 1. It can be trained normally after modifying sru to lstm. Please share your experience if you have more interesting work in Chinese version.