Back to results
US 12,008,466 B1Granted
Processor with control flow
AssigneeAmazon Technologies, Inc.
InventorsRandy Huang, Ron Diamant, Thomas A. Volpe
Life of this patent
Estimating the term…
Abstract
In various implementations, provided are systems and methods for operating a neural network that includes conditional structures. In some implementations, an integrated circuit can compute a result using a set of intermediate results, where the intermediate results are computed from the outputs of a hidden layer of the neural network. The integrated circuit can further test the result against a condition. The outcome of the test can determine a next layer that the integrated circuit is to execute, or can be used to determine that further execution of the neural network can be terminated.
Claims11
An integrated circuit, comprising:
- an array of processing engines;
- and
- a memory operable to store weight values and instructions for a neural network, wherein, during operation of the integrated circuit, the memory is further operable to store intermediate results computed by the array of processing engines;
- wherein, upon receiving input data, the integrated circuit is operable to use the weight values to perform a task defined for the neural network, and wherein performing the task includes:
- computing results using a set of intermediate results, wherein the array of processing engines compute the intermediate results using a first set of weight values stored in the memory, the first set of weight values associated with a first layer of the neural network;
- reading an instruction from the memory, wherein the instruction is a conditional instruction associated with a conditional layer of the neural network;
- determining whether the results meet a first condition or a second condition indicated by the instruction;
- writing a value to a storage location, the value indicating whether the results meet the first condition or the second condition;
- based on a determination that the results meet the first condition, continuing performance of the task using a second set of weight values for a second layer of the neural network but not continuing performance of the task using a third set of weight values for a third layer of the neural network, such that the second layer receives an output of the first layer as input;
- based on a determination that the results meet the second condition, continuing performance of the task using the third set of weight values for the third layer but not continuing performance of the task using the second set of weight values for the second layer, such that the third layer receives the output of the first layer as input, and wherein the third layer further receives an output of a fourth layer of the neural network as input, the fourth layer being separate from the first layer, the second layer, the third layer, and the conditional layer;
- and
- based on a determination that the results do not meet each of the first condition and the second condition, continuing performance of the task using both the second set of weight values for the second layer and the third set of weight values for the third layer, wherein the set of weight values, the second set of weight values, and the third set of weight values are different.
9 more elements
A method implemented using an integrated circuit, comprising:
- receiving input data at the integrated circuit, the integrated circuit including an array of processing engines and a memory operable to store weight values for a neural network and instructions for the neural network, and wherein the integrated circuit is operable to use the weight values to perform a task defined for the neural network;
- computing results using a set of intermediate results, wherein the array of processing engines compute the intermediate results using a first set of weight values stored in the memory, the first set of weight values associated with a first layer of the neural network;
- reading a conditional instruction from the memory associated with a conditional layer of the neural network;
- writing a value to a storage location, the value indicating whether the results meet a first condition or a second condition indicated by the conditional instruction;
- based on a determination that the results meet the first condition, continuing performance of the task using a second set of weight values for a second layer of the neural network but not continuing performance of the task using a third set of weight values for a third layer of the neural network, such that the second layer receives an output of the first layer as input;
- based on a determination that the results meet the second condition, continuing performance of the task using the third set of weight values for the third layer but not continuing performance of the task using the second set of weight values for the second layer, such that the third layer receives the output of the first layer as input, and wherein the third layer further receives an output of a fourth layer of the neural network as input, the fourth layer being separate from the first layer, the second layer, the third layer, and the conditional layer;
- and
- based on a determination that the results do not meet each of the first condition and the second condition, continuing performance of the task using both the second set of weight values for the second layer and the third set of weight values for the third layer, and wherein the set of weight values, the second set of weight values, and the third set of weight values are different.
5 more elements
Dependencies are inferred from the claim text.