Skip to content

[GPTQ] Fix actorder resolution, add sentinel #1453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 22, 2025

Conversation

kylesayrs
Copy link
Collaborator

@kylesayrs kylesayrs commented May 20, 2025

Purpose

  • Fix false assumption that actorder field is of enum type
    • Despite the fact that actorder passes through a field_validator, QuantizationArgs has the use_enum_values configuration set, meaning that enum values are converted to strings.
    • This was done in relation to this fix
  • Remove conflict with recipes which manually specify activation ordering by using a sentinel value

Follow ups

Testing

  • Ran llama3 example with manually specified actorder=group

Signed-off-by: Kyle Sayers <[email protected]>
Copy link

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

Signed-off-by: Kyle Sayers <[email protected]>
@kylesayrs kylesayrs changed the title [GPTQ] Fix actorder resolution [GPTQ] Fix actorder resolution, add sentinel May 20, 2025
@kylesayrs kylesayrs added the ready When a PR is ready for review label May 20, 2025
@kylesayrs kylesayrs marked this pull request as ready for review May 20, 2025 21:28
Signed-off-by: Kyle Sayers <[email protected]>
Copy link
Collaborator

@dsikka dsikka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question before approving

@kylesayrs kylesayrs enabled auto-merge (squash) May 22, 2025 17:09
@kylesayrs kylesayrs merged commit a2675dc into main May 22, 2025
11 checks passed
@kylesayrs kylesayrs deleted the kylesayrs/fix-default-actorder branch May 22, 2025 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready When a PR is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants