Add all fbgemm kernel Tensors into Int4WeightOnlyConfig and Float8DynamicActivationInt4WeightConfig

[jerryzh168]

Stacked PRs:

• ->Add all fbgemm kernel Tensors into Int4WeightOnlyConfig and Float8DynamicActivationInt4WeightConfig #2474
• Add Float8Tensor #2463

---

Add all fbgemm kernel Tensors into Int4WeightOnlyConfig and Float8DynamicActivationInt4WeightConfig

Summary:
we will

• deprecate FbgemmConfig since it's a single kernel (later).
• we'd like to categorize things to derived dtype + packed format, e.g. int4 preshuffled, float8 plain
• Added PackingFormat that has preshuffled, plain and _legacy for legacy implementation

Test Plan:
python test/quantization/quantize_/workflows/int4/test_int4_tensor.py
python test/quantization/quantize_/workflows/int4/test_int4_preshuffled_tensor.py
python test/quantization/quantize_/workflows/float8/test_float8_tensor.py

Reviewers:

Subscribers:

Tasks:

Tags:

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2474

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Cancelled Job, 1 Unrelated Failure

As of commit 21d492e with merge base 11f1a76 ():

NEW FAILURES - The following jobs have failed:

• Run Float8 Tests / test (H100, linux.aws.h100.4, --pre torch torchvision torchaudio fbgemm-gpu-genai --index-url htt... / linux-job (gh)
  RuntimeError: Command docker exec -t 71c337b0bf05c096b089c865beeb2722776dcdd00f1b277b602c87398c483c61 /exec failed with exit code 2
• Run Float8 Tests / test (SM-89, linux.g6.4xlarge.experimental.nvidia.gpu, --pre torch --index-url https://download.p... / linux-job (gh)
  test/quantization/quantize_/workflows/float8/test_float8_tensor.py::TestFloat8Tensor::test_slice_granularity1

CANCELLED JOB - The following job was cancelled. Please retry:

• Run Library Integration Tests / test (SM-89, linux.g6.4xlarge.experimental.nvidia.gpu, --pre fbgemm-gpu-genai --index-url https:/... / linux-job (gh)
  ##[error]The operation was canceled.

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• Run Library Integration Tests / test (H100, linux.aws.h100, --pre torchvision torchaudio fbgemm-gpu-genai --index-url https://dow... / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.