Parallel depth-wise processing architectures for neural networks
AssigneeQUALCOMM Incorporated
InventorsMustafa BADAROGLU, Zhongze WANG, Francois Ibrahim ATALLAH
Life of this patent
Estimating the term…
Abstract
Methods and apparatus for performing machine learning tasks, and in particular, to a neural-network-processing architecture and circuits for improved performance through depth parallelism. One example neural-network-processing circuit generally includes a plurality of groups of processing element (PE) circuits, wherein each group of PE circuits comprises a plurality of PE circuits configured to process in parallel an input at a plurality of depths.
Claims29
A processing circuit comprising a plurality of groups of processing element (PE) circuits, wherein:
- each group of PE circuits comprises a plurality of PE circuits configured to process in parallel an input at a plurality of depths, and
- each PE circuit comprises:
- one or more multiplication circuits, each multiplication circuit being configured to calculate a partial product, and
- a local accumulator having an input coupled to an output of the one or more multiplication circuits, the local accumulator being configured to generate a sum from the partial product calculated by each of the one or more multiplication circuits.
A method of neural network processing, comprising:
- receiving an input for processing;
- and
- for each segment of a plurality of segments of the received input:
- generating, substantially in parallel, a respective intermediate output for a respective depth of a plurality of depths in a neural network based on weights in the neural network associated with the respective depth in the neural network;
- accumulating the intermediate output for each respective depth into a final output;
- and
- outputting the final output to a memory bus.
An apparatus, comprising:
- a memory bus;
- a processor configured to:
- receive an input for processing;
- and
- for each segment of a plurality of segments of the received input:
- generate, substantially in parallel, a respective intermediate output for a respective depth of a plurality of depths in a neural network based on weights in the neural network associated with the respective depth in the neural network;
- accumulate the intermediate output for each respective depth into a final output;
- and
- output the final output to the memory bus;
- and
- a digital post-processing block configured to retrieve the final output from the memory bus and perform one or more operations based on the final output.
An apparatus comprising:
- means for receiving an input for processing;
- means for generating, for each segment of a plurality of segments of the received input and substantially in parallel, an intermediate output for a respective depth of a plurality of depths in a neural network based on weights in a neural network associated with the respective depth in the neural network;
- means for accumulating, for each segment of a plurality of segments of the received input, the intermediate output for each respective depth into a final output;
- and
- means for outputting, for each segment of a plurality of segments of the received input, the final output to a memory bus.
Dependencies are inferred from the claim text.