# US 10,871,964 B2 · Architecture for sparse neural network acceleration > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,871,964 B2 - Document ID: US10871964B2 - Status: Granted - Kind: B2 - Application number: 15/393,670 - Filed: 2016-12-29 - Published: 2018-07-05 - Granted: 2020-12-22 - Earliest claimed: 2016-12-29 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Yatish Girish Turakhia; Javid Jaffari; Amrit Panda; Karamvir Chatha - Web page: https://patentgrep.com/patent/US10871964B2 ## CPC - G06F 9/3001: {Arithmetic instructions} - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0495: Quantised networks; Sparse networks; Compressed networks - G06N 3/063: using electronic means - G06N 3/09: Supervised learning - G06N 3/045: Combinations of networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method, a computer-readable medium, and an apparatus for a sparse neural network are provided. The apparatus may include a hardware accelerator. The apparatus may determine, for each pair of operands to be processed by a MAR unit, whether both operands of the pair are non-zero. The apparatus may prevent a pair of operands to be processed by the MAR unit from being loaded to a multiplier of the MAR unit when an operand of the pair of operands is zero. The apparatus may place the pair of operands into one of a plurality of queues when both operands of the pair of operands are non-zero. ## Claims ### Claim 1 A method for a deep convolutional neural network, comprising: determining, by a load unit, for a pair of operands fetched from an operand storage to be placed into one queue of a plurality of queues for processing by a multiply-accumulate-reduce (MAR) unit, whether both operands of the pair of operands are non-zero; and preventing, by the load unit, the pair of operands from being placed into the plurality of queues when an operand of the pair of operands is determined to be zero. ### Claim 2 (depends on 1) The method of claim 1, further comprising placing the pair of operands into the one queue of the plurality of queues when both operands of the pair of operands are determined to be non-zero. ### Claim 3 (depends on 2) The method of claim 2, wherein non-zero operand pairs are placed into the plurality of queues randomly or in a pre-determined order that repeats. ### Claim 4 (depends on 2) The method of claim 2, further comprising: selecting a queued pair of operands from the plurality of queues; and loading the queued pair of operands to a multiplier of the MAR unit. ### Claim 5 (depends on 4) The method of claim 4, wherein the selecting and the loading are performed by an arbiter that randomly selects the queued pair of operands from the plurality of queues and loads the queued pair of operands to the multiplier of the MAR unit. ### Claim 6 (depends on 4) The method of claim 4, wherein the selecting and the loading are performed by an allocator that employs a set of heuristic rules to select the queued pair of operands from the plurality of queues and to load the queued pair of operands to the multiplier of the MAR unit. ### Claim 7 (depends on 2) The method of claim 2, further comprising synchronizing the plurality of queues after a particular number of MAR instructions. ### Claim 8 (depends on 7) The method of claim 7, wherein the synchronizing the plurality of queues comprises preventing placing new operands into the plurality of queues before each queue of the plurality of queues is empty. ### Claim 9 (depends on 7) The method of claim 7, wherein the particular number is a fixed number or a variable number. ### Claim 10 (depends on 2) The method of claim 2, wherein the plurality of queues includes a certain number of queues, and wherein the certain number is determined based on a sparsity of the deep convolutional neural network. ### Claim 11 (depends on 1) The method of claim 1, wherein the preventing comprises disabling, by the load unit, the plurality of queues when the operand of the pair of operands is determined to be zero. ### Claim 12 (depends on 1) The method of claim 1, wherein each of the pair of operands correspond to a weight and an activation associated with the deep convolutional neural network. ### Claim 13 (depends on 1) The method of claim 1 further comprising: fetching, by the load unit, based at least on a sparsity of the deep convolutional neural network, the pair of operands at a rate that is one of 2 times, 4 times, and 8 times a count of operands that the MAR unit handles. ### Claim 14 An apparatus for a deep convolutional neural network, comprising: means for determining, for a pair of operands fetched from an operand storage to be placed into one queue of a plurality of queues for processing by a multiply-accumulate-reduce (MAR) unit, whether both operands of the pair of operands are non-zero; and means for preventing the pair of operands from being placed into the plurality of queues when an operand of the pair of operands is determined to be zero. ### Claim 15 (depends on 14) The apparatus of claim 14, further comprising means for placing the pair of operands into the one queue of the plurality of queues when both operands of the pair of operands are determined to be non-zero. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein non-zero operand pairs are placed into the plurality of queues randomly or in a pre-determined order that repeats. ### Claim 17 (depends on 15) The apparatus of claim 15, further comprising: means for selecting a queued pair of operands from the plurality of queues; and means for loading the queued pair of operands to a multiplier of the MAR unit. ### Claim 18 (depends on 17) The apparatus of claim 17, wherein the means for selecting and the means for loading comprise an arbiter that randomly selects the queued pair of operands from the plurality of queues and loads the queued pair of operands to the multiplier of the MAR unit. ### Claim 19 (depends on 17) The apparatus of claim 17, wherein the means for selecting and the means for loading comprise an allocator that employs a set of heuristic rules to select the queued pair of operands from the plurality of queues and to load the queued pair of operands to the multiplier of the MAR unit. ### Claim 20 (depends on 15) The apparatus of claim 15, further comprising means for synchronizing the plurality of queues after a particular number of MAR instructions. ### Claim 21 (depends on 20) The apparatus of claim 20, wherein the means for synchronizing the plurality of queues is configured to prevent placing new operands into the plurality of queues before each queue of the plurality of queues is empty. ### Claim 22 (depends on 20) The apparatus of claim 20, wherein the particular number is a fixed number or a variable number. ### Claim 23 (depends on 15) The apparatus of claim 15, wherein the plurality of queues includes a certain number of queues, and wherein the certain number is determined based on a sparsity of the deep convolutional neural network. ### Claim 24 An apparatus for a deep convolutional neural network, comprising: a memory; and at least one processor coupled to the memory and configured to: determine, for a pair of operands fetched from an operand storage in the memory to be placed into one queue of a plurality of queues for processing by a multiply-accumulate-reduce (MAR) unit, whether both operands of the pair of operands are non-zero; and prevent the pair of operands from being placed into the plurality of queues when an operand of the pair of operands is determined to be zero. ### Claim 25 (depends on 24) The apparatus of claim 24, wherein the at least one processor is further configured to place the pair of operands into the one queue of the plurality of queues when both operands of the pair of operands are determined to be non-zero. ### Claim 26 (depends on 25) The apparatus of claim 25, wherein non-zero operand pairs are placed into the plurality of queues randomly or in a pre-determined order that repeats. ### Claim 27 (depends on 25) The apparatus of claim 25, wherein the at least one processor is further configured to: select a queued pair of operands from the plurality of queues; and load the queued pair of operands to a multiplier of the MAR unit. ### Claim 28 (depends on 27) The apparatus of claim 27, wherein the selecting and the loading are performed by an arbiter that randomly selects the queued pair of operands from the plurality of queues and loads the queued pair of operands to the multiplier of the MAR unit. ### Claim 29 (depends on 22) The apparatus of claim 22, wherein the at least one processor is further configured to synchronize the plurality of queues after a particular number of MAR instructions, and wherein, to synchronize the plurality of queues, the at least one processor is configured to prevent placing new operands into the plurality of queues before each queue of the plurality of queues is empty. ### Claim 30 A non-transitory computer-readable medium storing computer executable code for a deep convolutional neural network, comprising code to: determine, for a pair of operands fetched from an operand storage to be placed into one queue of a plurality of queues for processing by a multiply-accumulate-reduce (MAR) unit, whether both operands of the pair of operands are non-zero; and prevent the pair of operands from being placed into the plurality of queues when an operand of the pair of operands is determined to be zero.