Skip to content

[Examples] Standardize AWQ example #1412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 14, 2025
Merged

Conversation

kylesayrs
Copy link
Collaborator

@kylesayrs kylesayrs commented May 5, 2025

Purpose

  • Standardize awq example to follow the same format as the other examples

Changes

  • Rearrange code to match the format of other examples
  • Use the chat template to match format of other examples
  • Do not load 100x more samples than are needed: instead, only load the number of examples that is required
  • Do not manually truncate input ids, instead utilize the truncation provided by the tokenizer

Testing

  • Ran example to completion and confirmed good generation

kylesayrs added 2 commits May 5, 2025 16:19
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Copy link

github-actions bot commented May 5, 2025

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

@dsikka
Copy link
Collaborator

dsikka commented May 5, 2025

Please close this ticket when this lands:
INFERENG-605

@kylesayrs kylesayrs marked this pull request as ready for review May 5, 2025 21:24
@kylesayrs kylesayrs added the ready When a PR is ready for review label May 5, 2025
@kylesayrs
Copy link
Collaborator Author

Still want a proper readme for AWQ

Copy link
Collaborator

@brian-dellabetta brian-dellabetta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this is in I can add another PR to resolve this issue

and update it with the QuantizationMixin changes

Signed-off-by: Kyle Sayers <[email protected]>
Copy link
Collaborator

@markurtz markurtz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small NIT, otherwise looks good

markurtz
markurtz previously approved these changes May 12, 2025
Signed-off-by: Kyle Sayers <[email protected]>
Copy link
Collaborator

@brian-dellabetta brian-dellabetta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling this!

@kylesayrs kylesayrs enabled auto-merge (squash) May 14, 2025 18:08
@kylesayrs kylesayrs merged commit c993050 into main May 14, 2025
11 checks passed
@kylesayrs kylesayrs deleted the kylesayrs/fix-awq-example-typo branch May 14, 2025 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready When a PR is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants