What Is AWQ in LLM Quantization and How to Use It

AWQ is a post-training quantization technique that packs large language models into 4-bit weight formats while shielding the ~1% of salient weights that actually drive quality. In practice it cuts VRAM roughly in half and buys 1.5–3× faster inference than FP16, which is exactly the kind of resource…