System and method for torque-based structured pruning for deep neural networks

An apparatus comprising:

• at least one processing device configured to:
• access a machine learning model, the machine learning model trained using a torque-based constraint, wherein using the torque-based constraint includes using weaker constraints in channels near a pivot point channel of a layer of the machine learning model, and using stronger constraints in channels distal from the pivot point channel, resulting in a higher concentration of weights in the channels near the pivot point channel, and wherein at least one of the channels distal from the pivot point channel is pruned from the machine learning model;
• receive an input from an input source;
• provide the input to the machine learning model;
• receive an output from the machine learning model;
• and
• instruct at least one action based on the output from the machine learning model.

A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:

• access a machine learning model, the machine learning model trained using a torque-based constraint, wherein using the torque-based constraint includes using weaker constraints in channels near a pivot point channel of a layer of the machine learning model, and using stronger constraints in channels distal from the pivot point channel, resulting in a higher concentration of weights in the channels near the pivot point channel, and wherein at least one of the channels distal from the pivot point channel is pruned from the machine learning model;
• receive an input from an input source;
• provide the input to the machine learning model;
• receive an output from the machine learning model;
• and
• instruct at least one action based on the output from the machine learning model.

A method comprising:

• training, using at least one processor of an electronic device, a machine learning model, wherein the training includes:
• applying a torque-based constraint on one or more filters of the machine learning model, wherein using the torque-based constraint includes using weaker constraints in channels near a pivot point channel of a layer of the machine learning model, and using stronger constraints in channels distal from the pivot point channel, resulting in a higher concentration of weights in the channels near the pivot point channel, and wherein at least one of the channels distal from the pivot point channel is pruned from the machine learning model;
• adjusting, based on applying the torque-based constraint, a first set of one or more filters of the machine learning model to have the higher concentration of the weights than a second set of one or more filters of the machine learning model;
• and
• pruning at least one channel of the machine learning model based on an average weight for the at least one channel.