Fashion200k result

[clickmouse]

Hello, I have some misunderstandings with the fashion200k dataset.

My reproduced result is -7~8% lower than the reported one of fashion200k, so I have several questions I wish you could answer for me.

1. Does the gallery(database) contain all the images from data/labels/xxx_test_detect_all.txt?
2. Are query images from data/test_queries.txt?
3. In your paper and Github page, you said that the modifier should be "Change A to B". However, we find that it's actually "Replace A with B" in the code link you provided. Does it have a negative impact?
4. The target image is not unique when testing fashion200k (, which is different from shoes and fashion-IQ). Is it correct to write the code of the fashion200k dataset part following fashion_IQ? Do there exist additional modifications?
5. Could you please release the code of these three datasets.

These questions have been sent to you by email. I'm afraid that you are too busy to notice my email, so I raise this issue. I wish you could choose one to reply at your convenience. Thanks a lot.

[numpee]

Hi @clickmouse,

Regarding F200K, please refer to the TIRG code base: https://github.com/google/tirg
We follow the dataset processing method in this code.

[clickmouse]

@numpee Thanks for your reply. I try to follow the code of your provided link but still encounter the same problem. Maybe I made some mistakes or had some misunderstandings, so, could you please give a brief answer to the first four questions I asked before? This means a lot to me. Thanks again.

It would be better if you could provide the code, even though it may not have been sorted out due to time constraints

[numpee]

From what I can remember, the TIRG code pre-generates all test queries when the dataset is initialized. Please check this function of their code.
And yes, we used "replace" as well (followed TIRG code) - sorry for the confusion.

Finally for Fashion200K, we use the target attributes, rather than the image ID, to evaluate whether the predicted sample is correct. As you mentioned, target attributes are not necessarily unique to each image, which means that multiple images can be correct as long as the target attribute matches.