Convolutional neural network pruning processing method, data processing method, and device
AssigneeHuawei Technologies Co., Ltd.
InventorsYehui TANG, Yixing XU, Yunhe WANG, Chunjing XU
Life of this patent
Estimating the term…
Abstract
Embodiments of this application disclose a convolutional neural network pruning processing method, a data processing method, and a device, which may be applied to the field of artificial intelligence. The convolutional neural network pruning processing method includes: performing sparse training on a convolutional neural network by using a constructed objective loss function, where the objective loss function may include three sub-loss functions.
Claims20
A convolutional neural network pruning processing method, comprising:
- determining a first sub-loss function based on a target task, wherein the first sub-loss function represents a difference between a training sample input to a target convolutional neural network and an output prediction result, the target convolutional neural network is a convolutional neural network used for training, the target convolutional neural network has different network parameters and loss functions in training processes for different training samples, and the training sample is a sample in a training set;
- obtaining a second sub-loss function based on a channel importance function and a dynamic weight, wherein when a value of the first sub-loss function does not reach a first threshold, a value of the dynamic weight is obtained based on the value of the first sub-loss function, and the value of the dynamic weight is in an inverse correlation with the value of the first sub-loss function;
- obtaining an objective loss function based on the first sub-loss function and the second sub-loss function;
- and
- performing sparse training on the target convolutional neural network by using the objective loss function to obtain a trained target convolutional neural network, wherein the sparse training is pruning a convolution kernel of the target convolutional neural network based on a value of the channel importance function.
A data processing method, comprising:
- obtaining input data related to a target task;
- pruning a trained target convolutional neural network based on the input data to obtain a pruned sub-network, wherein the trained target convolutional neural network is obtained through training by using an objective loss function, the objective loss function is obtained based on a first sub-loss function and a second sub-loss function, the first sub-loss function is determined based on the target task, the second sub-loss function is determined based on a channel importance function and a dynamic weight, the first sub-loss function represents a difference between a training sample input to the target convolutional neural network and an output prediction result, when a value of the first sub-loss function does not reach a first threshold, a value of the dynamic weight is obtained based on the value of the first sub-loss function, the value of the dynamic weight is in an inverse correlation with the value of the first sub-loss function, the objective loss function is used to perform sparsening on the target convolutional neural network during training of the target convolutional neural network, and the sparsening is pruning a convolution kernel of the target convolutional neural network based on a value of the channel importance function;
- and
- processing the input data by using the sub-network to obtain output data.
A training device, comprising:
- a first determining module, configured to determine a first sub-loss function based on a target task, wherein the first sub-loss function represents a difference between a training sample input to a target convolutional neural network and an output prediction result, the target convolutional neural network is a convolutional neural network used for training, the target convolutional neural network has different network parameters and loss functions in training processes for different training samples, and the training sample is a sample in a training set;
- a second determining module, configured to obtain a second sub-loss function based on a channel importance function and a dynamic weight, wherein when a value of the first sub-loss function does not reach a first threshold, a value of the dynamic weight is obtained based on the value of the first sub-loss function, and the value of the dynamic weight is in an inverse correlation with the value of the first sub-loss function;
- a third determining module, configured to obtain an objective loss function based on the first sub-loss function and the second sub-loss function;
- and
- a training module, configured to perform sparse training on the target convolutional neural network by using the objective loss function to obtain a trained target convolutional neural network, wherein the sparse training is pruning a convolution kernel of the target convolutional neural network based on a value of the channel importance function.
Dependencies are inferred from the claim text.