Back to results
US 12,737,609 B2Granted
Systems and methods for quantization aware training of a neural network for heterogeneous hardware platform
AssigneeBeijing Transtreams Technology Co. Ltd.
InventorsYang Liu, Yongxiong Ren, Lingzhi Liu
Life of this patent
Estimating the term…
Abstract
Systems and methods are provided for quantization aware training of a neural network for heterogeneous hardware platform. In the method, the system acquires hardware profiles with respect to a plurality of hardware components of a heterogeneous hardware platform. The system determines a plurality of hardware configurations based on the hardware profiles. The system acquires a set of training data and performing a quantization aware training using the training data on a network model based on the hardware configurations. The system obtains the network model with model weights for the heterogeneous hardware platform.
Claims20
A method of performing quantization aware training (QAT) of a neural network, comprising:
- acquiring hardware profiles with respect to a plurality of hardware components of a heterogeneous hardware platform, wherein the hardware profiles comprise performance data associated with the execution of different types of neural networks;
- determining hardware configurations based on the performance data without imposing architecture constraints by:
- selecting a computational component from the hardware components for each layer of the quantization aware training based on the hardware profiles comprising the performance data;
- determining a computing precision for each layer of the quantization aware training based on the hardware profiles comprising the performance data;
- and
- generating the hardware configurations associated with the selected computational component and the determined computing precision with respect to performing the neural network based on the hardware profile corresponding to the selected computational component, wherein the hardware configurations comprise one or more of quantization configurations and a plurality of computing precisions with respect to performing the neural network;
- acquiring a set of training data and performing a quantization aware training using the training data on a network model based on the hardware configurations to mimic the hardware configurations of the heterogeneous hardware platform;
- and
- obtaining the network model with model weights for the heterogeneous hardware platform based at least in part on the hardware configurations.
6 more elements
A quantization aware training (QAT) system, comprising:
- at least one computer storage memory operable to store data along with computer-executable instructions;
- and
- at least one processor operable to read the data and operate the computer-executable instructions to:
- acquiring hardware profiles with respect to a plurality of hardware components of a heterogeneous hardware platform, wherein the hardware profiles comprise performance data associated with the execution of different types of neural networks;
- determining hardware configurations based at least in part on the performance data without imposing architecture constraints by:
- selecting a computational component from the hardware components for each layer of the quantization aware training based on the hardware profiles comprising the performance data;
- determining a computing precision for each layer of the quantization aware training based on the hardware profiles comprising the performance data;
- and
- generating the hardware configurations associated with the selected computational component and the determined computing precision with respect to performing the neural network based on the hardware profile corresponding to the selected computational component, wherein the hardware configurations comprise one or more of quantization configurations and a plurality of computing precisions with respect to performing the neural network;
- acquiring a set of training data and performing a quantization aware training using the training data on a network model based on the hardware configurations to mimic hardware configurations of the heterogeneous hardware platform;
- and
- outputting the network model with model weights for the heterogeneous hardware platform based at least in part on the hardware configurations.
9 more elements
A non-transitory computer readable medium having stored thereon a program for executing a method of performing quantization aware training of a neural network, the method comprising:
- acquiring hardware profiles with respect to a plurality of hardware components of a heterogeneous hardware platform, wherein the hardware profiles comprise performance data associated with the execution of different types of neural networks;
- determining hardware configurations based at least in part on the performance data without imposing architecture constraints by:
- selecting a computational component from the hardware components for each layer of the quantization aware training based on the hardware profiles comprising the performance data;
- determining a computing precision for each layer of the quantization aware training based on the hardware profiles comprising the performance data;
- and
- generating the hardware configurations associated with the selected computational component and the determined computing precision with respect to performing the neural network based on the hardware profile corresponding to the selected computational component, wherein the hardware configurations comprise one or more of quantization configurations and a plurality of computing precisions with respect to performing the neural network;
- acquiring a set of training data and performing a quantization aware training using the training data on a network model based on the hardware configurations to mimic hardware configurations of the heterogeneous hardware platform;
- and
- obtaining the network model with model weights for the heterogeneous hardware platform based at least in part on the hardware configurations.
6 more elements
Dependencies are inferred from the claim text.