how to choose base pretrained model

[Gladiator566]

Thanks for your great job, if I want to train my own model use my custom data, how should I choose the base pretained model?
model_name_or_path can only be the base chat model like Qwen3-0.6B or I can directly use a pretrained reranker model like Qwen3-Reranker-0.6B instead?

[Gladiator566]

Hello @nilesh2797 , when I replicate your work by using train_full_blockrank_10p_msmsarco.yaml, which use 10% MS MARCO data to finetine Mistral-7B-Instruct-v0.3, I got a error like AssertionError: Bracket token not found in last h1 positions., how to solve it?
Thanks !

[nilesh2797]

Hey, sorry for missing this earlier, will take a look today!

[Gladiator566]

Hey, sorry for missing this earlier, will take a look today!

@nilesh2797 Hi, still got the same problem, anything in progress?

[nilesh2797]

Hi @Gladiator566, can you try now? Also, I tried adding initial support for training qwen models (see configs/train_full_blockrank_qwen_4b_10p_msmsarco.yaml), let me know if you are able to get that running too.

[Gladiator566]

Hi @Gladiator566, can you try now? Also, I tried adding initial support for training qwen models (see configs/train_full_blockrank_qwen_4b_10p_msmsarco.yaml), let me know if you are able to get that running too.

@nilesh2797 Hi, sorry for the late reply, I have tried training qwen models using your new config yaml, it works perfect! I have trained my own qwen reranker using blockrank succesfully, thanks a lot for it!

But I encounter another new minor issue, when I use scripts/eval_attn.py to evaluate my model like in README recommended, I got a error like AssertionError: Number of blocks mismatch, how to solve it?

One more thing, how can I use my own pretrained reranker to predict scores between some queries and documents? Can I use blockrank as a library to predict it? or I have to realize a api-like predict function like in scripts/eval_attn.py?

Thanks!