LLM Quantization

This page collects notes on llm quantizaiton.

Data Format

\[ \begin{aligned} \text{quantized\_weight} &= \text{round}\left(\frac{\text{weight}}{\text{scale}}\right) + \text{zero\_point} \\ \text{dequantized\_weight} &= (\text{quantized\_weight} - \text{zero\_point}) \times \text{scale} \end{aligned} \]