How to better stabilize yaw prediction using centerpoint

[gauthamnarayan]

I'm facing yaw instability where on some lidar frames where the yaw angle flickers. For example, in the sequence of 3 frames below, the yaw is quite wrong for some objects in frame 2, but is okay in frame 1 and 3.

I am basically using this config file. from mmdetection3d on a custom dataset. I'm seeing the same yaw issue on some of the pillar based centerpoint models as well.

However when I try the onnx files provided under lidar_centerpoint here the detections are quite stable. Would someone be able to comment on how the yaw and overall detection stability was improved while developing the reference models?

cc'ing @knzo25 @esteve @yukke42. Any insight would be super helpful. Thank you!

Frame 1

Frame 2

Frame 3

[yukke42]

@gauthamnarayan Hello,

1. Outputs with the norm of yaw near 0 are filtered out as orientation unstable objects and then you can choose other outpus before NMS. (ref)
2. In my experiment, it maight be caused by the underfitting.
3. The gap of lidar sensor between train and inference data might canse the orientation unstable output.

I hope these ideas help you.

[knzo25]

@yukke42
Submitted the PR in autowarefoundation/autoware.universe#3571
🙇

[gauthamnarayan]

Thanks @knzo25 for the update. By any chance will you be able to share the mmdetection3d training logs/output for the model? I'm trying to replicate results and am not able to.

Any help to reproduce results will be very much appreciated!

[knzo25]

@gauthamnarayan
As far as I know, releasing training-related data is an ongoing discussion in the AWF, specifically in the sensing/perception working group (https://docs.google.com/presentation/d/1MujD8dof3Qci6A9eH6-J_feQ3D7iI4OcTd4Ey3iTL6c/edit#slide=id.g2224b7ea427_3_7).

[yukke42]

@gauthamnarayan
Sorry for the late reply, I was on the vacation.

1 We didn't do something special for the centerpoint model. Data augmentation params are slightly changed but I don't think it has a big impact to the orientation stability. As the documentation, our centerpoint model was trained for 60 epochs with nuscenes dataset, but that of mmdet3d is just for 20 epochs.
3 If you're fine, we can continue here.

[gauthamnarayan]

Thanks @yukke42 and @knzo25 for your replies. It seems like this was mostly a data distribution issue. We were able to get some more labelled data and things look much better.

Thanks for your help!