US 11,055,617 B1 · Partial-activation of neural network based on heat-map of neural network activity

Abstract

A device, system, and method for training or prediction of a neural network. A current value may be stored for each of a plurality of synapses or filters in the neural network. A historical metric of activity may be independently determined for each individual or group of the synapses or filters during one or more past iterations. A plurality of partial activations of the neural network may be iteratively executed. Each partial-activation iteration may activate a subset of the plurality of synapses or filters in the neural network. Each individual or group of synapses or filters may be activated in a portion of a total number of iterations proportional to the historical metric of activity independently determined for that individual or group of synapses or filters. Training or prediction of the neural network may be performed based on the plurality of partial activations of the neural network.

Claims30

Dependent claims

A method for training or prediction of a neural network, the method comprising:

storing a current value of each of a plurality of synapses or filters in a neural network;
determining a historical metric of activity independently for each individual or group of one or more of the synapses or filters during one or more past iterations;
iteratively executing a plurality of partial activations of the neural network, wherein each partial-activation iteration activates a subset of the plurality of synapses or filters in the neural network, and wherein each individual or group of synapses or filters is activated in a portion of a total number of partial-activation iterations proportional to the historical metric of activity independently determined for that individual or group of synapses or filters;
and
performing training or prediction of the neural network based on the plurality of partial activations of the neural network.

2 more elements

A system for training or prediction of a neural network, the system comprising:

one or more memories to store a current value of each of a plurality of synapses or filters in the neural network;
one or more processors configured to:
determine a historical metric of activity independently for each individual or group of one or more of the synapses or filters during one or more past iterations,
iteratively execute a plurality of partial activations of the neural network, wherein each partial-activation iteration activates a subset of the plurality of synapses or filters in the neural network, and wherein each individual or group of synapses or filters is activated in a portion of a total number of partial-activation iterations proportional to the historical metric of activity independently determined for that individual or group of synapses or filters, and
train or predict using the neural network based on the plurality of partial activations of the neural network.

2 more elements

Dependencies are inferred from the claim text.

Partial-activation of neural network based on heat-map of neural network activity

Life of this patent

Partial-activation of neural network based on heat-map of neural network activity

Life of this patent

Abstract

Claims30