Post-training quantization for diffusion transformers
AssigneeSamsung Electronics Co., Ltd.
InventorsMostafa EL-KHAMY, Qingfeng LIU, Sanghyun YI
Life of this patent
Estimating the term…
Abstract
A technique for quantization in diffusion transformers is disclosed. A weight quantizer is configured to quantize a weight matrix of a layer in a diffusion transformer block to generate a quantized weight matrix. An activation quantizer is configured to quantize an activation matrix of the layer to generate a quantized activation matrix. A time-step quantizer is configured to estimate a quantization parameter based on at least one of the quantized weight matrix or the quantized activation matrix for a time step based on a per-step calibration set.
Claims20
An apparatus comprising:
- a weight quantizer configured to quantize a weight matrix of a layer in a diffusion transformer block to generate a quantized weight matrix;
- an activation quantizer configured to quantize an activation matrix of the layer to generate a quantized activation matrix;
- and
- a time-step quantizer configured to estimate a quantization parameter based on at least one of the quantized weight matrix or the quantized activation matrix for a time step based on a per-step calibration set.
A method comprising:
- quantizing a weight matrix of a layer in a diffusion transformer block to generate a quantized weight matrix;
- quantizing an activation matrix of the layer to generate a quantized activation matrix;
- and
- estimating a quantization parameter based on at least one of the quantized weight matrix or the quantized activation matrix for a time step based on a per-step calibration set.
A system comprising:
- a layer in a diffusion transformer block;
- and
- a layer quantizer configured to quantize the layer, the layer quantizer comprising:
- a weight quantizer configured to quantize a weight matrix of a layer in a diffusion transformer block to generate a quantized weight matrix;
- an activation quantizer configured to quantize an activation matrix of the layer to generate a quantized activation matrix;
- and
- a time-step quantizer configured to estimate a quantization parameter based on at least one of the quantized weight matrix or the quantized activation matrix for a time step based on a per-step calibration set.
Dependencies are inferred from the claim text.