Data Free Neural Network Pruning
AssigneeDeeplite Inc.
InventorsMartin FERIANC, Anush SANKARAN, Olivier MASTROPIETRO, Ehsan SABOORI, Davis Mangan SAWYER
Life of this patent
Estimating the term…
Abstract
A system, method and computer readable medium are provided for implementing data free neural network pruning. The illustrative method include determining mutual information between outputs of two or more of the plurality neurons and a respective two or more inputs used to generate the outputs, the two or more neurons being activated as a result of synthetically created inputs for measuring entropy. The method includes determining a sparser neural network by pruning the plurality of neurons based on the determined mutual information.
Claims20
A method for pruning a neural network comprised of a plurality of neurons, the method comprising:
- determining mutual information between outputs of two or more of the plurality of neurons and a respective two or more inputs used to generate the outputs, the two or more neurons being activated as a result of synthetically created inputs for measuring entropy;
- and
- determining a sparser neural network by pruning the plurality of neurons based on the determined mutual information.
A system for pruning a neural network comprised of a plurality of neurons, the system comprising:
- a processor and memory, the memory comprising computer executable instructions which cause the processor to:
- determine mutual information between outputs of two or more of the plurality of neurons and a respective two or more inputs used to generate the outputs, the two or more neurons being activated as a result of synthetically created inputs for measuring entropy;
- and
- determine a sparser neural network by pruning the plurality of neurons based on the determined mutual information.
A computer readable medium storing computer executable instructions which cause a processor to:
- determine mutual information between outputs of two or more of a plurality of neurons and a respective two or more inputs used to generate the outputs, the two or more neurons being activated as a result of synthetically created inputs for measuring entropy;
- and
- determine a sparser neural network by pruning the plurality of neurons based on the determined mutual information.
Dependencies are inferred from the claim text.