# US 2022/0036196 A1 · Reconfigurable computing architecture for implementing artificial neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036196 A1
- Document ID: US20220036196A1
- Status: Published application
- Kind: A1
- Application number: 17/389,414
- Filed: 2021-07-30
- Published: 2022-02-03
- Earliest claimed: 2020-08-03
- Applicants: Commissariat A L'energie Atomique Et Aux Energies Alternatives
- Inventors: Michel HARRAND
- Web page: https://patentgrep.com/patent/US20220036196A1

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 7/50: Adding; Subtracting (G06F7/483 - G06F7/491, G06F7/544 - G06F7/556 take precedence)
- G06F 7/523: Multiplying only
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A computer for computing a layer (Ck, Ck+1) of an artificial neural network is provided. The computer is able to be configured in accordance with two separate configurations and comprises: a transmission line; a set of computing units; a set of weight memories each associated with a computing unit, each weight memory containing a subset of synaptic coefficients required and sufficient for the associated computing unit to carry out the computations necessary for either one of the two configurations and control means for configuring the computing units of the computer in accordance with either one of the two configurations. In the first configuration, the computing units are configured such that a weighted sum is computed in full by one and the same computing unit. In the second configuration, the computing units are configured such that a weighted sum is computed by a chain of multiple computing units arranged in series.

## Claims

### Claim 1

A computer (CALC) for computing a layer (Ck, Ck+1) of an artificial neural network, the neural network being formed of a sequence of layers (Ck, Ck+1) each consisting of a set of neurons, each layer being associated with a set of synaptic coefficients (wi,jk+1) forming at least one weight matrix ([MP]k+1, WP,Q), the computer (CALC) being able to be configured in accordance with two separate configurations (CONF1, CONF2) and comprising:
a transmission line (L_data) for distributing input data (Xjk, δik+1, xi,j),
a set of computing units (PE0, PE1, PE2, PE3) of ranks n=0 to N, where N is an integer greater than or equal to 1, for computing an input data sum weighted by synaptic coefficients,
a set of weight memories (MEM_POIDS0, MEM_POIDS1, MEM_POIDS2, MEM_POIDS3) each associated with a computing unit (PE0, PE1, PE2, PE3), each weight memory containing a subset of synaptic coefficients required and sufficient for the associated computing unit (PE0, PE1, PE2, PE3) to carry out the computations necessary for either one of the two configurations (CONF1, CONF2),
control means for configuring the computing units (PE0, PE1, PE2, PE3) of the computer (CALC) in accordance with either one of the two configurations (CONF1, CONF2),
in the first configuration (CONF1), the computing units being configured such that a weighted sum is computed in full by one and the same computing unit,
in the second configuration (CONF2), the computing units being configured such that a weighted sum is computed by a chain of multiple computing units arranged in series.

### Claim 2 (depends on 1)

The computer (CALC) according to claim 1, wherein the first configuration and the second configuration correspond, respectively, to operation of the computer in either one of the phases from among a data propagation phase and an error back-propagation phase.

### Claim 3 (depends on 2)

The computer (CALC) according to claim 2, wherein the input data (Xjk, δik+1, xi,j) are data (Xjk, xi,j) propagated in the data propagation phase or errors (δik+1) back-propagated in the error back-propagation phase.

### Claim 4 (depends on 1)

The computer (CALC) according to claim 1, wherein the number of computing units (PE0, PE1, PE2, PE3) is lower than the number of neurons in a layer (Ck, Ck+1).

### Claim 5 (depends on 1)

The computer (CALC) according to claim 1, wherein each computing unit comprises:
i. an input register (Reg_in0, Reg_in1, Reg_in2, Reg_in3) for storing an input datum (Xjk, δik+1, xi,j);
ii. a multiplier circuit (MULT) for computing the product of an input datum (Xik, δik+1, xi,j) and a synaptic coefficient (wi,jk);
iii. an adder circuit (ADD0, ADD1, ADD2, ADD3) having a first input connected to the output of the multiplier circuit (MULT0, MULT1, MULT2, MULT3) and being configured so as to carry out operations of summing partial computing results of a weighted sum;
iv. at least one accumulator (ACC00, ACCS0, ACC01, ACCS1, ACC02, ACCS2, ACC03, ACCS3) for storing partial or final computing results of the weighted sum.

### Claim 6 (depends on 5)

The computer (CALC) according to claim 5, comprising:
a data distribution element (D1) having N+1 outputs, each output being connected to the register (Reg_in0, Reg_in1, Reg_in2, Reg_in3) of a computing unit of rank n (PE0, PE1, PE2, PE3),
the distribution element (D1) being commanded by the control means so as to simultaneously distribute an input datum (Xik, δik+1, xi,j) to all of the computing units (PE0, PE1, PE2, PE3) when the first configuration (CONF1) is activated.

### Claim 7 (depends on 5)

The computer (CALC) according to claim 5, furthermore comprising a memory stage operating in accordance with a “first in first out” principle so as to propagate a partial result from the last computing unit of rank n=N (PE3) to the first computing unit (PE0) of rank n=0, the output of said memory stage being connected to the first computing unit (PE0), and the memory stage being activated by the control means when the second configuration (CONF2) is activated.

### Claim 8 (depends on 5)

The computer (CALC) according to claim 5, wherein each computing unit (PE0, PE1, PE2, PE3) comprises at least a number of accumulators (ACC00, ACCS0) equal to the number of neurons per layer divided by the number of computing units (PE0, PE1, PE2, PE3) rounded up to the nearest integer.

### Claim 9 (depends on 8)

The computer (CALC) according to claim 8, wherein:
each set of accumulators (ACC00, ACCS0, ACC01, ACCS1, ACC02, ACCS2, ACC03, ACCS3) comprises a write input (E10, E11, E12, E13) able to be selected from among the inputs of each accumulator of the set and a read output (S10, S11, S12, S13) able to be selected from among the outputs of each accumulator of the set;
each computing unit (PE1, PE2, PE3) of rank n=1 to N comprising:
a multiplexer (MUX1) having a first input (I1) connected to the output (S11, S12, S13) of the set of accumulators (ACC01, ACCS1, ACC02, ACCS2, ACC03, ACCS3) of the computing unit of rank n, a second input (I2) connected to the output (S10, S11, S12) of the set of accumulators (ACC00, ACCS0, ACC01, ACCS1, ACC02, ACCS2) of a computing unit of rank n−1 and an output connected to a second input of the adder circuit (ADD1, ADD2, ADD3) of the computing unit of rank n;
the computing unit of rank n=0 (PE0) comprising:
a multiplexer (MUX0) having a first input (I1) connected to the output of the set of accumulators (ACC00, ACCS0) of the computing unit of rank n=0, a second input (I2) connected to the output (S10) of the set of accumulators (ACC03, ACCS3) of the computing unit of rank n=0 and an output connected to a second input of the adder circuit (ADD0) of the computing unit of rank n=0;
the control means being configured so as to select the first input (I1) of each multiplexer (MUX0, MUX1, MUX2, MUX3) when the first configuration (CONF1) is chosen and to select the second input (I2) of each multiplexer (MUX0, MUX1, MUX2, MUX3) when the second configuration (CONF2) is activated.

### Claim 10 (depends on 8)

The computer (CALC) according to claim 8, wherein all of the sets of accumulators (ACC00, ACCS0, ACC01, ACCS1, ACC02, ACCS2, ACC03, ACCS3) are interconnected so as to form a memory stage for propagating a partial result from the last computing unit of rank n=N (PE3) to the first computing unit (PE0) of rank n=0, the memory stage operating in accordance with a “first in first out” principle when the second configuration (CONF2) is activated.

### Claim 11 (depends on 1)

The computer (CALC) according to claim 1, comprising a set of error memories (MEM_err0, MEM_err1, MEM_err2, MEM_err3), each one being associated with a computing unit (PE0, PE1, PE2, PE3), for storing a subset of computed errors (δjk).

### Claim 12 (depends on 11)

The computer (CALC) according to claim 11, wherein, for each computing unit (PE0, PE1, PE2, PE3), the multiplier (MULT) is connected to the error memory associated with the same computing unit (MEM_err0, MEM_err1, MEM_err2, MEM_err3) so as to compute the product of an input datum (Xik, xi,j) and a stored error signal (δik+1) during a phase of updating the weights.

### Claim 13 (depends on 1)

The computer (CALC) according to claim 1, comprising a read circuit (LECT) connected to each weight memory (MEM_POIDS0, MEM_POIDS1, MEM_POIDS2, MEM_POIDS3) for commanding the reading of the synaptic coefficients (wi,jk).

### Claim 14 (depends on 1)

The computer (CALC) according to claim 1, wherein a computed layer (Ck+1) is fully connected to the preceding layer (Ck), and the associated synaptic coefficients (wi,jk) form a weight matrix ([MP]k) of size M×M′, where M and M are the respective numbers of neurons in the two layers.

### Claim 15 (depends on 14)

The computer (CALC) according to claim 14, wherein the distribution element (D1) is commanded by the control means so as to distribute an input datum (Xik, δik+1) associated with a neuron of rank i to a computing unit (PE0, PE1, PE2, PE3) of rank n, such that i modulo N+1 is equal to n, when the second configuration (CONF2) is activated.

### Claim 16 (depends on 14)

The computer (CALC) according to claim 14, wherein, when the first configuration (CONF1) is activated, all of the multiplication and addition operations for computing the weighted sum (Xik+1, δjk) associated with the neuron of rank i are carried out exclusively by the computing unit (PE0, PE1, PE2, PE3) of rank n, such that i modulo N+1 is equal to n.

### Claim 17 (depends on 14)

The computer (CALC) according to claim 14, wherein, when the second configuration (CONF2) is activated, each computing unit (PE1, PE2, PE3) of rank n=1 to N carries out the operation of multiplying each input datum (Xjk, δik+1) associated with the neuron of rank j by a synaptic coefficient (wi,jk), such that j modulo N+1 is equal to n, followed by addition of the output from the computing unit (PE0, PE1, PE2, PE3) of rank n−1, so as to obtain a partial or total result of a weighted sum (Xik+1, δjk).

### Claim 18 (depends on 14)

The computer (CALC) according to claim 14, wherein the subset of synaptic coefficients stored in the weight memory (MEM_POIDS0, MEM_POIDS1, MEM_POIDS2, MEM_POIDS3) of rank n corresponds to the synaptic coefficients (wi,jk) of all of the rows of rank i of the weight matrix ([MP]k), such that i modulo N+1 is equal to n, when the first configuration (CONF1) is a computing configuration for the data propagation phase and the second configuration (CONF2) is a computing configuration for the error back-propagation phase.

### Claim 19 (depends on 14)

The computer (CALC) according to claim 14, wherein the subset of synaptic coefficients stored in the weight memory (MEM_POIDS0, MEM_POIDS1, MEM_POIDS2, MEM_POIDS3) of rank n corresponds to the synaptic coefficients (wi,jk) of all of the columns of rank j of the weight matrix ([MP]k), such that j modulo N+1 is equal to n, when the first configuration (CONF1) is a computing configuration for the error back-propagation phase and the second configuration (CONF2) is a computing configuration for the data propagation phase.

### Claim 20 (depends on 1)

The computer (CALC) according to claim 1, wherein the neural network comprises at least one convolutional layer of neurons, the layer having a plurality of output matrices of rank q=0 to Q, where Q is a positive integer, each output matrix being obtained from at least one input matrix of rank p=0 to P, where P is a positive integer,
for each input matrix of rank p and output matrix of rank q pair, the associated synaptic coefficients (wi,j) forming a weight matrix (WP,Q).

### Claim 21 (depends on 20)

The computer (CALC) according to claim 20, wherein, when the first configuration (CONF1) is activated, all of the multiplication and addition operations for computing an output matrix of rank q are carried out exclusively by the computing unit (PE0, PE1, PE2, PE3) of rank n, such that q modulo N+1 is equal to n.

### Claim 22 (depends on 20)

The computer (CALC) according to claim 20, wherein, when the second configuration (CONF2) is activated, each computing unit (PE1, PE2, PE3) of rank n=1 to N carries out the operations of computing the partial results obtained from each input matrix of rank p, such that p modulo N+1 is equal to n, followed by addition of the partial result from the computing unit (PE0, PE1, PE2, PE3) of rank n−1.

### Claim 23 (depends on 20)

The computer (CALC) according to claim 20, wherein the subset of synaptic coefficients stored in the weight memory (MEM_POIDS0, MEM_POIDS1, MEM_POIDS2, MEM_POIDS3) of rank n corresponds to the synaptic coefficients (wi,j,p,qk) belonging to all of the weight matrices (WP,Q) associated with the output matrix of rank q, such that q modulo N+1 is equal to n, when the first configuration (CONF1) is a computing configuration for the data propagation phase and the second configuration (CONF2) is a computing configuration for the error back-propagation phase.

### Claim 24 (depends on 20)

The computer (CALC) according to claim 20, wherein the subset of synaptic coefficients stored in the weight memory (MEM_POIDS0, MEM_POIDS1, MEM_POIDS2, MEM_POIDS3) of rank n corresponds to the synaptic coefficients (wi,j) belonging to all of the weight matrices (WP,Q) associated with the input matrix of rank p, such that p modulo N+1 is equal to n, when the first configuration (CONF1) is a computing configuration for the error back-propagation phase and the second configuration (CONF2) is a computing configuration for the data propagation phase.
