[note on the paper] Related work on canonical orientation estimation

[akshay-krishnan]

Great work! Thanks for making the model available too.

About the baselines you compared to in the paper, please note that Omni3D (CubeRCNN) is not great for the orientation task, since it does not attempt to predict canonical orientations (it uses a Chamfer loss to align the cuboid without explicit orientation supervision). We reported this in our previous work on OmniNOCS (ECCV 24 oral). Please refer to Section 5.3 and Table 6 in the paper. The NOCSformer model is better than CubeRCNN at the orientation task and generalizes to in-the-wild images.

[wufeim]

Totally agree. The literature review and baseline experiments ignored all related works on category-level pose estimation and only compared with Cube RCNN for 3D Object Detection.

• Xiang et al. Beyond PASCAL: A Benchmark for 3D Object Detection in the Wild. WACV'14.
• Zhou et al. StarMap for Category-Agnostic Keypoint and Viewpoint Estimation. ECCV'18.
• Wang et al. NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation. ICLR'21.
• Ma et al. ImageNet3D: Towards General-Purpose Object-Level 3D Understanding. NeurIPS'24.

[zehanwang01]

Thanks for your reminder. We have noticed these omissions as well. We will update baseline comparisons on more benchmarks in the next few days. Please stay tuned.

[zehanwang01]

Sorry for the delayed update, things were a bit hectic lately. Here are the results of Orient Anything on ImageNet3D:

Setting	Model	Avg.	Electronics	Furniture	Household	Music	Sports	Vehicles	Work
Zero-shot	ImageNet3D-ResNet50	37.1	30.1	35.6	28.1	11.8	51.7	36.7	40.9
	Orient Anything-B	48.5	61.0	66.8	37.9	27.3	25.6	70.8	33.4
Fine-tuning	ImageNet3D-ResNet50	53.6	49.2	52.4	45.8	26.0	65.2	56.5	58.5
	ImageNet3D-DINOv2-B	64.0	75.3	47.9	32.9	23.5	74.7	38.1	64.0
	Orient Anything-B	71.3	77.6	89.7	64.4	54.4	47.6	87.4	61.2

Note that we couldn't find detailed definitions for the major categories like Electronics, Furniture, and Household in ImageNet3D, so we used GPT to map its 200 categories into the 7 broader categories. The average score provides a more meaningful comparison.

[zehanwang01]

And here are the results on Pascal3D+ [1] and Pix3d [2]:

	Pascal3D+		Pix3D
	Acc@30°	Abs	Acc@30°
FSDetView (Few-Shot) [3]	46.00	38.30	49.00
Orient-Anything-S (Zero-shot)	46.00	37.50	68.00
Orient-Anything-B (Zero-shot)	48.00	35.17	69.00
Orient-Anything-L (Zero-shot)	55.00	22.90	66.00

[1] Xiang Y, Mottaghi R, Savarese S. Beyond pascal: A benchmark for 3d object detection in the wild[C]//IEEE winter conference on applications of computer vision. IEEE, 2014: 75-82.
[2] Sun X, Wu J, Zhang X, et al. Pix3d: Dataset and methods for single-image 3d shape modeling[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 2974-2983.
[3] Xiao Y, Lepetit V, Marlet R. Few-shot object detection and viewpoint estimation for objects in the wild[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 45(3): 3090-3106.