Skip to content

Fix fp16 ONNX export API + document RoiAlign TensorRT limitation - #13

Merged
kouya-marino merged 2 commits into
mainfrom
fix/fp16-convert-and-roialign-doc
May 31, 2026
Merged

kouya-marino merged 2 commits into
mainfrom
fix/fp16-convert-and-roialign-doc

Conversation

@kouya-marino

Copy link
Copy Markdown
Owner

Fixed

fp16 ONNX export crashed:

AttributeError: module 'onnxconverter_common.float16' has no attribute 'convert_float_to_fp16'

The exporter called a non-existent function. Corrected to convert_float_to_float16(..., keep_io_types=True) — the real API in onnxconverter-common, and keep_io_types=True keeps the model's input/output float32 so the Deploy runtime's float32 preprocessing still matches.

Documented

docs/TensorRT.md gets a "Known limitation" note: two-stage detectors (Faster/Mask R-CNN) fail the TensorRT engine build at the RoiAlign op because TensorRT maps it to a plugin that isn't registered by default (ROIAlign plugin was not found in the plugin registry), and the exporter doesn't supply it. Plus a matching troubleshooting entry. (Discovered while attempting a real engine build on the user's model.)

Docs + a one-line code fix. ruff clean; exporter tests pass (the fp16 path needs onnx/onnxconverter-common which CI doesn't install, so it's covered by the corrected API name + manual verification against onnxconverter-common 1.16.0).

prashantrawat11 added 2 commits May 31, 2026 15:33
exporter._convert_fp16 called float16.convert_float_to_fp16, which doesn't exist in onnxconverter-common; use convert_float_to_float16(..., keep_io_types=True) so the I/O stays float32 for the Deploy runtime's preprocessing.

docs/TensorRT.md: add a 'Known limitation' note that two-stage detectors (Faster/Mask R-CNN) fail TensorRT engine builds at the RoiAlign op (plugin not in registry), plus a troubleshooting entry. CHANGELOG updated.
onnxconverter-common's float16 pass produces an invalid graph for Detectron2 models (type-mismatched Cast nodes onnxruntime rejects), and it had already overwritten model.onnx before validation caught it. _convert_fp16 now converts to a temp file, verifies it loads, and only replaces the model if valid — otherwise it keeps the fp32 model and logs a warning. validate_onnx is also non-fatal so a post-write check can't fail a completed export.
@kouya-marino
kouya-marino merged commit 4d60f9b into main May 31, 2026
4 checks passed
@kouya-marino
kouya-marino deleted the fix/fp16-convert-and-roialign-doc branch May 31, 2026 07:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant