# US 2019/0171932 A1 · Device and method for executing neural network operation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0171932 A1 - Document ID: US20190171932A1 - Status: Published application - Kind: A1 - Application number: 16/268,468 - Filed: 2019-02-05 - Published: 2019-06-06 - Earliest claimed: 2016-08-05 - Assignees: Cambricon Technologies Corporation Limited - Applicants: Cambricon Technologies Corporation Limited - Inventors: Yunji CHEN; Shaoli LIU; Dong HAN; Tianshi CHEN - Web page: https://patentgrep.com/patent/US20190171932A1 ## CPC - G06N 3/063: using electronic means - G06F 15/17325: {Synchronisation; Hardware support therefor (intertask synchronisation G06F9/52)} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0499: Feedforward networks - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Aspects for performing neural network operations are described herein. The aspects may include a first neural network processing module configured to process at least a portion of neural network data and an on-chip interconnection module communicatively connected to the first neural network processing module and one or more second neural network processing modules. The on-chip interconnection module may include a first layer interconnection module configured to communicate with an external storage device and one or more second layer interconnection modules respectively configured to communicate with the first neural network processing module and the one or more second neural network processing modules. Further, the first neural network processing module may include a neural network processor configured to perform one or more operations on the portion of the neural network data and a high-speed storage device configured to store results of the one or more operations. ## Claims ### Claim 1 An apparatus for neural network processing, comprising: a first neural network processing module configured to process at least a portion of neural network data; and an on-chip interconnection module communicatively connected to the first neural network processing module and one or more second neural network processing modules, wherein the first neural network processing module includes: a neural network processor configured to perform one or more operations on the portion of the neural network data, and a high-speed storage device configured to store results of the one or more operations. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the neural network processor includes: an instruction caching unit configured to store one or more instructions that indicate the one or more operations to be performed; and a neural network computing device configured to execute one of the one or more instructions. ### Claim 3 (depends on 2) The apparatus of claim 2, wherein the one or more instructions include: an ACK instruction that includes one or more ACK operation codes respectively corresponding to the one or more second neural network processing modules, wherein each of the ACK operation codes indicates whether the first neural network processing module is to transmit a first ACK signal to the corresponding second neural network processing module to indicate data transmission is completed. ### Claim 4 (depends on 2) The apparatus of claim 2, wherein the one or more instructions include: a FENCE instruction that includes one or more FENCE operation codes respectively corresponding to the one or more second neural network processing modules, wherein each of the FENCE operation codes indicates whether the first neural network processing module is to check a second ACK signal is received from the corresponding second neural network processing module. ### Claim 5 (depends on 2) The apparatus of claim 2, wherein the one or more instructions include: a SYNC instruction that includes one or more SYNC operation codes respectively corresponding to the one or more second neural network processing modules, wherein each of the SYNC operation codes indicates whether the corresponding second neural network processing module is to be paused for synchronization. ### Claim 6 (depends on 2) The apparatus of claim 2, wherein the one or more instructions include: a COMPUTE instruction that includes a first COMPUTE operation code that indicates a computation operation, a second COMPUTE operation code that indicates a first address of a first portion of data, a third COMPUTE operation code that indicates a first length of the first portion of data, a fourth COMPUTE operation code that indicates a second address of a second portion of data, and a fifth COMPUTE operation code that indicates a second length of the second portion of data. ### Claim 7 (depends on 2) The apparatus of claim 2, wherein the one or more instructions include: an IO instruction that includes a first IO operation code that indicates a source address, a second IO operation code that indicates a destination address, and a third TO operation code that indicates a size of a data block stored to be moved from the source address to the destination address. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the neural network processor includes a synchronization unit configured to transmit one or more synchronization signals to one or more of the second neural network processing modules. ### Claim 9 (depends on 1) The apparatus of claim 1, wherein the neural network processor includes an I/O controller configured to receive data from the high-speed storage device and the second neural network processing modules. ### Claim 10 (depends on 1) The apparatus of claim 1, wherein the neural network processor includes a high-speed caching unit configured to temporarily store one or more execution results generated by the neural network computing device. ### Claim 11 (depends on 1) The apparatus of claim 1, wherein the on-chip interconnection module includes: a first layer interconnection module configured to communicate with an external storage device, and one or more second layer interconnection modules respectively configured to communicate with the first neural network processing module and the one or more second neural network processing modules. ### Claim 12 (depends on 1) The apparatus of claim 1, wherein the on-chip interconnection module includes a first layer interconnection module configured to communicate with an external storage device, and one or more groups of interconnection modules, wherein at least one of the one or more groups of interconnection modules are communicatively connected to the first layer interconnection module, wherein at least another of the one or more groups of interconnection modules are communicatively connected to the first neural network processing module and the one or more second neural network processing modules, and wherein the one or more groups of interconnection modules are structured as a tree that includes the first layer interconnection module as a root node. ### Claim 13 A method for neural network processing, comprising: processing, by a first neural network processing module, at least a portion of neural network data; communicating, by an on-chip interconnection module, with an external storage device, wherein the on-chip interconnection module is communicatively connected to the first neural network processing module and one or more second neural network processing modules; communicating, by the on-chip interconnection module, with the first neural network processing module and the one or more second neural network processing modules; performing, by a neural network processor included in the first neural network processing module, one or more operations on the portion of the neural network data; and storing, by a high-speed storage device included in the first neural network processing module, results of the one or more operations. ### Claim 14 (depends on 13) The method of claim 13, further comprising: storing, by an instruction caching unit of the neural network processor, one or more instructions that indicate the one or more operations to be performed; and executing, by a neural network computing device of the neural network processor, one of the one or more instructions. ### Claim 15 (depends on 14) The method of claim 14, wherein the one or more instructions include: an ACK instruction that includes one or more ACK operation codes respectively corresponding to the one or more second neural network processing modules, wherein each of the ACK operation codes indicates whether the first neural network processing module is to transmit a first ACK signal to the corresponding second neural network processing module to indicate data transmission is completed. ### Claim 16 (depends on 14) The method of claim 14, wherein the one or more instructions include: a FENCE instruction that includes one or more FENCE operation codes respectively corresponding to the one or more second neural network processing modules, wherein each of the FENCE operation codes indicates whether the first neural network processing module is to check a second ACK signal is received from the corresponding second neural network processing module. ### Claim 17 (depends on 14) The method of claim 14, wherein the one or more instructions include: a SYNC instruction that includes one or more SYNC operation codes respectively corresponding to the one or more second neural network processing modules, wherein each of the SYNC operation codes indicates whether the corresponding second neural network processing module is to be paused for synchronization. ### Claim 18 (depends on 14) The method of claim 14, wherein the one or more instructions include: a COMPUTE instruction that includes a first COMPUTE operation code that indicates a computation operation, a second COMPUTE operation code that indicates a first address of a first portion of data, a third COMPUTE operation code that indicates a first length of the first portion of data, a fourth COMPUTE operation code that indicates a second address of a second portion of data, and a fifth COMPUTE operation code that indicates a second length of the second portion of data. ### Claim 19 (depends on 14) The method of claim 14, wherein the one or more instructions include: an IO instruction that includes a first IO operation code that indicates a source address, a second IO operation code that indicates a destination address, and a third IO operation code that indicates a size of a data block stored to be moved from the source address to the destination address. ### Claim 20 (depends on 13) The method of claim 13, further comprising transmitting, by a synchronization unit of the neural network processor, one or more synchronization signals to one or more of the second neural network processing modules. ### Claim 21 (depends on 13) The method of claim 13, further comprising receiving, by an I/O controller of the neural network processor, data from the high-speed storage device and the second neural network processing modules. ### Claim 22 (depends on 13) The method of claim 13, further comprising temporarily storing, by a high-speed caching unit, one or more execution results generated by the neural network computing device. ### Claim 23 (depends on 13) The method of claim 13, wherein the on-chip interconnection module includes a first layer interconnection module configured to communicate with an external storage device, and one or more groups of interconnection modules, wherein at least one of the one or more groups of interconnection modules are communicatively connected to the first layer interconnection module, wherein at least another of the one or more groups of interconnection modules are communicatively connected to the first neural network processing module and the one or more second neural network processing modules, and wherein the one or more groups of interconnection modules are structured as a tree that includes the first layer interconnection module as a root node.