Technical definition
Model quantization
Reducing the numerical precision of model weights or activations to lower compute and memory use.
What It Means
Quantization can make inference faster and allow models to run on smaller hardware by using lower-precision representations. The trade-off may include reduced output quality or hardware-specific constraints.