# US 2018/0322381 A1 · Apparatus and method for performing a forward operation of artificil neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322381 A1
- Document ID: US20180322381A1
- Status: Published application
- Kind: A1
- Application number: 16/039,567
- Filed: 2018-07-19
- Published: 2018-11-08
- Earliest claimed: 2016-01-20
- Assignees: Cambricon Technologies Corporation Limited
- Applicants: Cambricon Technologies Corporation Limited
- Inventors: Shaoli LIU; Qi GUO; Yunji CHEN; Tianshi CHEN
- Web page: https://patentgrep.com/patent/US20180322381A1

## CPC

- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/063: using electronic means
- G06F 9/3885: {using a plurality of independent parallel functional units}
- G06N 3/0499: Feedforward networks
- G06N 3/02: Neural networks
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

Aspects for executing forward propagation of artificial neural network are described here. As an example, the aspects may include a plurality of computation modules connected via an interconnection unit; and a controller unit configured to decode an instruction into one or more groups of micro-instructions, wherein the plurality of computation modules are configured to perform respective groups of the micro-instructions.

## Claims

### Claim 1

An apparatus for forward propagation of a multilayer neural network, comprising:
a plurality of computation modules connected via an interconnection unit; and
a controller unit configured to decode an instruction into one or more groups of micro-instructions, wherein the plurality of computation modules are configured to perform respective groups of the micro-instructions.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the plurality of computation modules further comprise:
multiple slave computation modules configured to parallelly calculate multiple groups of slave output values based on an input vector received via the interconnection unit; and
a master computation module configured
to calculate a merged intermediate vector based on one or more intermediate result vectors combined by the interconnection unit based on the multiple groups of slave output values calculated by the multiple slave computation modules, and
to generate an output vector based on the merged intermediate vector.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the master computation module is configured to perform one operation selected from the group consisting of:
adding a bias value to the merged intermediate vector;
activating the merged intermediate vector with an activation function;
outputting a predetermined value based on a comparison between the merged intermediate vector and a random number; and
pooling the merged intermediate vector.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein each of the slave computation modules includes a slave neuron caching unit configured to store the input vector.

### Claim 5 (depends on 2)

The apparatus of claim 2, wherein the interconnection unit is structured as a binary tree including one or more levels, each of which includes one or more nodes,
wherein each of the nodes at one level is connected to two nodes at a lower level, and
wherein each of the nodes transmits same data to the two nodes at the lower level and combines data received from the two nodes at the lower level.

### Claim 6 (depends on 3)

The apparatus of claim 3, wherein the master computation module includes:
a master neuron caching unit configured to cache data;
a master computation unit configured to perform the selected operation;
a master data dependency relationship determination unit configured to prevent the instruction from being executed based on a determination that a conflict exists between the instruction and other instructions.

### Claim 7 (depends on 2)

The apparatus of claim 2, wherein the slave computation module includes:
a slave computation unit configured to receive the one or more groups of micro-instructions and to perform arithmetic logical operations;
a slave data dependency relationship determination unit configured to perform reading/writing operations to a slave neuron caching unit, a weight value caching unit, and a weight gradient caching unit based on a determination that no conflict exists between the reading/writing operations; and
a weight value caching unit configured to store weight vectors.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the slave data dependency relationship determination unit configured to
determine whether there is dependent relationship between a first micro-instruction which has not been executed and a second micro-instruction which is being executed;
if there is no dependent relationship, allow the micro-instruction which has not been executed to be executed immediately, otherwise, the micro-instruction which has not been executed will not be allowed to execute until the execution of all the micro-instructions upon which that micro-instruction which has not been executed depend is completed.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the instruction is selected from the group consisting of
a CONFIG instruction for configuring constants required by computation of the current layer prior to starting computation of the artificial neural network of each layer,
a COMPUTE instruction for completing arithmetical logic computation of the multilayer neural network of each layer, and
an IO instruction for reading in the input data required by computation from an external address space and storing the data back into the external space after completion of computation.

### Claim 10 (depends on 1)

The apparatus of claim 1, further comprising:
an instruction caching unit configured to store the instruction; and
a direct memory access unit configured to transmit data to the computation modules from an external storage device and receive data from the computation modules to the external storage device.

### Claim 11

A method for forward propagation of a multilayer neural network, comprising:
receiving, by a controller unit, an instruction;
decoding, by the controller unit, the instruction into one or more groups of micro-instructions; and
assigning, by the controller unit, the one or more groups of micro-instruction respectively to a plurality of computation modules.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
parallelly calculating, by multiple slave computation modules, multiple groups of slave output values based on an input data vector received via an interconnection unit;
combining, by the interconnection unit, the multiple groups of slave output values to generate an intermediate result vector;
calculating, by a master computation module, a merged intermediate vector based on at least the intermediate result vector; and
generating, by the master computation module, an output vector based on the merged intermediate vector.

### Claim 13 (depends on 12)

The method of claim 12, further comprising performing an operation selected from the group consisting of:
adding a bias value to the merged intermediate vector;
activating the merged intermediate vector with an activation function;
outputting a predetermined value based on a comparison between the merged intermediate vector and a random number; and
pooling the merged intermediate vector.

### Claim 14 (depends on 12)

The method of claim 12, further comprising storing, by a slave neuron caching unit included in each of the slave computation modules, the input vector.

### Claim 15 (depends on 12)

The method of claim 12, wherein the interconnection unit is structured as a binary tree including one or more levels, each of which includes one or more nodes,
wherein each of the nodes at one level is connected to two nodes at a lower level, and
wherein each of the nodes transmits same data to the two nodes at the lower level and combines data received from the two nodes at the lower level.

### Claim 16 (depends on 13)

The method of claim 13, further comprising:
caching data by a master neuron caching unit included in the mast computation module; and
preventing, by a master data dependency relationship determination unit, the instruction from being executed based on a determination that a conflict exists between the instruction and other instructions.

### Claim 17

The method of 12, further comprising:
receiving, by a slave computation unit included in the slave computation module, the one or more groups of micro-instructions;
performing arithmetic logical operations according to the micro-instructions;
perform, by a slave data dependency relationship determination unit, reading/writing operations to a slave neuron caching unit, a weight value caching unit, and a weight gradient caching unit based on a determination that no conflict exists between the reading/writing operations; and
storing, by a weight value caching unit include in the slave computation module, weight vectors.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
determining, by the slave data dependency relationship determination unit, whether there is dependent relationship between a first micro-instruction which has not been executed and a second micro-instruction which is being executed;
allowing, by the slave data dependency relationship determination unit, the micro-instruction which has not been executed to be executed immediately if there is no dependent relationship;
preventing, by the slave data dependency relationship determination unit, the micro-instruction which has not been executed from being executed if dependent relationship exists.

### Claim 19 (depends on 12)

The method of claim 12, wherein the instruction is selected from the group consisting of
a CONFIG instruction for configuring constants required by computation of the current layer prior to starting computation of the artificial neural network of each layer,
a COMPUTE instruction for completing arithmetical logic computation of the multilayer neural network of each layer, and
an IO instruction for reading in the input data required by computation from an external address space and storing the data back into the external space after completion of computation.

### Claim 20 (depends on 12)

The method of claim 12, further comprising:
storing, by an instruction caching unit, the instruction;
transmitting data, by a direct memory access unit, to the computation modules from an external storage device; and
receiving data, by a direct memory access unit, from the computation modules to the external storage device.