How to train together？

[paleblackless]

Can you give me suggestion about training detection and recognition branch？
I meet conditios that recognition loss does a convergence failure.
Thanks.