-
Notifications
You must be signed in to change notification settings - Fork 11
[NVFP4] Expand dynamic types, clean-up conditions #325
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
1af02a7
to
c2d56d7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few comments
Co-authored-by: Brian Dellabetta <[email protected]>
Co-authored-by: Brian Dellabetta <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As long as we feel confident that there's no expected behavior resulting from "local" being evaluated to True, then this looks good
yeah I agree. Used the explicit check for quant_config for bettter readability |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice nice nice
SUMMARY: - Requires neuralmagic/compressed-tensors#325 - Uses the new `tensor_group` strategy for nvfp4a16 quantization - Removes global_scale as an observer class parameter and passes in as a function call, similar to g_idx
Summary:
local
- with this change, the following conditions are now accepted:tensor_group
--> this strategy is now assocciated with the initialization of global_scales for weight and activations, notis_fp4
Testing