How modern quantization actually works
A classroom walkthrough on a 2-layer network
1 / 6
Setup
Our toy model — fp32 baseline
A 2-input, 2-hidden, 1-output binary classifier. Every edge is a learned weight stored as a 32-bit float. These are the numbers we want to compress to INT4.
Input layer
Hidden layer
Output layer
Edge weight (fp32)
0.43 0.07 0.91 0.02 0.65 0.38 x₁ 1.20 x₂ 0.80 h₁ 0.56 h₂ 0.74 ŷ 0.81 Input Hidden Output All weights in fp32 — baseline
Six weights in total: four from input→hidden (0.43, 0.07, 0.91, 0.02) and two from hidden→output (0.65, 0.38). In fp32, each weight uses 4 bytes. Goal: compress each to 4 bits (INT4) — 8× smaller — without destroying the model's predictions.
The core challenge
One outlier ruins the whole scale
INT4 gives you exactly 16 integer buckets. The scale is set by the full weight range. One large outlier forces a scale so coarse that all normal weights collapse into the same bucket.
Outlier weight
Normal weights (clustered)
INT4 buckets
INT4 buckets (16 total, naive): 5 weights all in bucket 0! 47.3 ← outlier! 0.0 47.3 ~23.6 15 buckets wasted on empty space Naive INT4 scale: s = (max − min) / (2⁴ − 1) = (47.3 − 0.0) / 15 = 3.15 per bucket Normal weights span 0.02 → 0.91 = 0.89 total range. At scale 3.15, they all round to bucket 0.
This is the enemy. Not just theoretical — transformer residual streams regularly contain activation values 100× larger than the typical magnitude. The three methods on the next slides each solve this differently.
Method 1
GPTQ — quantize by importance, compensate errors
GPTQ runs a calibration set through the model to compute the Hessian diagonal — a measure of how sensitive the loss is to each weight. High-sensitivity weights get quantized carefully; their rounding error is pushed to neighbors.
High Hessian (very important)
Medium importance
Low importance
Error propagation
Weight Value (fp32) Hessian diag H What GPTQ does w₁₁ = 0.43 input→hidden edge 0.43 H = 8.4 Very sensitive ↑↑ ① Quantize first. High sensitivity → careful bucketing w₁₂ = 0.91 input→hidden edge 0.91 H = 3.1 Medium sensitivity ② Quantize second. Absorbs error from w₁₁ rounding w₂₁ = 0.07 input→hidden edge 0.07 H = 0.3 Low sensitivity ↓↓ ③ Quantize last. Low impact — coarse rounding ok error flows ↓ Key insight: Hessian diagonal = ∂²Loss / ∂w² Large H means the loss surface curves steeply near this weight — small perturbations matter a lot. After quantizing w₁₁, the rounding error δ is compensated in w₁₂ so the layer output stays accurate.
GPTQ treats quantization as an optimization problem over the full weight matrix. It uses the Hessian to find the order that minimizes total output error, compensating downstream weights for the rounding error introduced at each step.
Limitation: Computing the Hessian needs a calibration dataset and takes hours for large models. GPTQ is weight-only — activations still run at fp16. Excellent quality at INT3/INT4 but expensive to run.
Method 2
SmoothQuant — migrate the difficulty
The problem with quantizing activations: they vary per input, so you can't measure their range ahead of time. SmoothQuant's insight: mathematically move the outlier "difficulty" from activations into weights — which are static and can be pre-calibrated.
Activation outlier
Smooth factor s = √outlier
After smoothing — safe to quantize
Before smoothing Activation x Weight W 1.20 48.2 ⚠ 0.80 0.43 0.65 0.38 scale = 48.2/15 ≈ 3.21 ← bad x̂ = x / s Ŵ = W × s s = √48.2 ≈ 6.94 After smoothing Activation x̂ Weight Ŵ 0.17 6.94 0.12 2.98 4.51 2.64 scale = 6.94/15 ≈ 0.46 ← 7× better! The magic: output is mathematically identical y = X · W = (X · diag(s)⁻¹) · (diag(s) · W) = X̂ · Ŵ Insert s then s⁻¹ — cancel out, output unchanged. But now both X̂ and Ŵ are quantization-friendly.
SmoothQuant is the key to quantizing activations, not just weights. By migrating the outlier scale from X into W, both tensors become well-behaved — and you can quantize them both to INT8 using standard methods.
What this enables: W8A8 inference — weight 8-bit, activation 8-bit. NVIDIA's INT8 tensor cores run integer matrix multiplications natively, giving up to 4× throughput vs fp16. SmoothQuant is used in TensorRT-LLM and vLLM for high-throughput serving.
Method 3
AWQ — protect the salient 1%
AWQ asks a simpler question than GPTQ: which weight channels matter most? Its answer: the ones with consistently large activations. Scale those channels up before quantization so they use more of the INT4 range.
Salient channel (large avg |x|)
Medium activation
Inactive channel
Step 1 — score each weight channel by average activation magnitude Channel 1 Channel 2 Channel 3 ← salient 0.43 0.65 0.91 0.07 0.38 0.02 avg |activation|: 0.2 0.6 1.0 ★ scale up ch3 × s After: ch3 expanded 0.43 0.07 × 4.5 4.10 0.09 more INT4 buckets used ↑ Why activation magnitude identifies salient weights Output contribution of weight w in channel c ≈ w × activation_c If |activation_c| is always large, a small rounding error δw gets amplified: output error ≈ δw × activation_c Scale up ch3 before quantization → finer buckets → smaller δw → smaller amplified error.
AWQ protects the 1% of weight channels that matter most. Finding them takes one forward pass — no Hessian needed. Scale up before quantization, scale outputs back down — the math is exact, the quality gain is large.
Why AWQ won in practice: Nearly as accurate as GPTQ but runs in minutes instead of hours. Most production INT4 checkpoints today (Mistral, Qwen, LLaMA) use AWQ. It's the default in Hugging Face's AutoAWQ library.
Summary
Three methods, three angles of attack
All three share the same goal — minimize output error under INT4/INT8 quantization — but each attacks the problem differently. In production, they're often combined.
GPTQ
Core idea
Hessian-weighted quantization + error compensation
Quant target
Weights only
Speed
Slow — hours
Best for
Max accuracy at INT3 / INT4
SmoothQuant
Core idea
Migrate activation outliers into weights via scale factor
Quant target
Weights + activations
Speed
Fast — one forward pass
Best for
W8A8 on INT8 tensor core hardware
AWQ
Core idea
Scale up salient channels (top 1% by activation)
Quant target
Weights only
Speed
Very fast — minutes
Best for
Production INT4 — most widely used today
The progression tells a story: GPTQ proved INT4 was viable for LLMs (2022). SmoothQuant extended that to activations, unlocking hardware speedups (2022). AWQ simplified the whole thing into something fast enough for everyday use (2023). Each paper fixed a specific failure mode of the previous one.
In practice today: Local inference (llama.cpp) → GGUF Q4_K_M uses AWQ-style channel scaling. Production serving (vLLM) → AWQ INT4 for memory, SmoothQuant W8A8 for throughput. Fine-tuning → QLoRA, which quantizes base model weights with NF4 (a 4-bit float variant), then fine-tunes small adapters in fp16.