LLM Quantization

This page collects notes on llm quantizaiton.

Data Format

\[ \begin{aligned} \text{quantized_weight} &= \text{round}\left(\frac{\text{weight}}{\text{scale}}\right) + \text{zero_point} \\ \text{dequantized_weight} &= (\text{quantized_weight} - \text{zero_point}) \times \text{scale} \end{aligned} \]