US 11,379,724 B1 · System and method for domain specific neural network pruning

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for domain-specific pruning of neural networks are described. An exemplary method includes obtaining a first neural network trained based on a first training dataset; obtaining one or more second training datasets respectively from one or more domains; and training, based on the first neural network and the one or more second training datasets, a second neural network comprising the first neural network and one or more branches extended from the first neural network, wherein the second neural network is applicable for inferencing in the one or more domains, and the training comprises: training the one or more branches based respectively on the one or more second training datasets and an output of the first neural network.

Claims20

Dependent claims

A computer-implemented method, comprising:

obtaining a first neural network trained based on a first training dataset;
obtaining one or more second training datasets respectively from one or more domains;
and
training, based on the first neural network and the one or more second training datasets, a second neural network comprising the first neural network and one or more branches extended from the first neural network, wherein the second neural network is applicable for inferencing in the one or more domains, and the training comprises an iterative process comprising one or more iterations, one iteration comprising:
pruning the second neural network to reach a sparsity ratio;
training the one or more branches of the pruned second neural network based respectively on the one or more second training datasets and an output of the first neural network until an objective function converges;
and
increasing the sparsity ratio for a next iteration.

5 more elements

A system comprising one or more processors and one or more non-transitory computer-readable memories coupled to the one or more processors and configured with instructions executable by the one or more processors to cause the system to perform operations comprising:

obtaining a first neural network trained based on a first training dataset;
obtaining one or more second training datasets respectively from one or more domains, wherein the one or more second training datasets comprise images collected by one or more cameras representing the one or more domains, and each of the images comprises a camera identifier and at least one label;
and
training, based on the first neural network and the one or more second training datasets, a second neural network comprising the first neural network and one or more branches extended from the first neural network, wherein the second neural network is applicable for inferencing in the one or more domains, and the training comprises:
training the one or more branches based respectively on the one or more second training datasets and an output of the first neural network.

2 more elements

A non-transitory computer-readable storage medium configured with instructions executable by one or more processors to cause the one or more processors to perform operations comprising:

obtaining a first neural network trained based on a first training dataset;
obtaining one or more second training datasets respectively from one or more domains;
and
training, based on the first neural network and the one or more second training datasets, a second neural network comprising the first neural network and one or more branches extended from the first neural network, wherein the second neural network is applicable for inferencing in the one or more domains, and the training comprises:
training the one or more branches based respectively on the one or more second training datasets and an output of the first neural network;
receiving an object detection request comprising a domain identifier;
determining one of the one or more branches in the second neural network based on the domain identifier;
and
performing object detection based on the second neural network through the one determined branch.

6 more elements

Dependencies are inferred from the claim text.

System and method for domain specific neural network pruning

Life of this patent

System and method for domain specific neural network pruning

Life of this patent

Abstract

Claims20