feat: INT4/INT8 quantization + expert offloading for consumer hardware

[oyi77]

Summary

Enables running OpenMythos MoE models on consumer hardware (RTX 3060 12GB) through INT4/INT8 weight quantization and GPU↔CPU↔NVMe expert offloading.

Changes

open_mythos/quantization.py (388 lines)

• QuantizedLinear: Memory-efficient quantized linear layer with INT4/INT8 support
• INT4 packing: two 4-bit values per byte (4x compression)
• Group-wise scaling (configurable group_size)
• quantize_model(): Model-level quantization (MoE experts only by default)

open_mythos/expert_offloader.py (330 lines)

• ExpertOffloader: LRU-based expert caching across memory hierarchy (GPU→CPU→NVMe)
• Automatic expert loading on-demand during inference
• Statistics tracking (hit rates, evictions)

Supporting files

• examples/quantized_inference.py: Demo script
• tests/test_quantization.py: Unit tests

Usage

from open_mythos import OpenMythos, mythos_1b
from open_mythos.quantization import quantize_model
from open_mythos.expert_offloader import ExpertOffloader

model = OpenMythos(mythos_1b())
model = quantize_model(model, bits=4, group_size=128)

offloader = ExpertOffloader(model, gpu_experts=4, cache_experts=16)
offloader.prepare()

All existing functionality preserved. Quantization is opt-in.