# US 2022/0036176 A1 · Artificial Neural Network Implementations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036176 A1
- Document ID: US20220036176A1
- Status: Published application
- Kind: A1
- Application number: 16/944,698
- Filed: 2020-07-31
- Published: 2022-02-03
- Earliest claimed: 2020-07-31
- Assignees: NanoSemi, Inc.
- Applicants: NanoSemi, Inc.
- Inventors: Alexandre Megretski; Alexandre Marques
- Web page: https://patentgrep.com/patent/US20220036176A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/048: Activation functions
- G06N 3/0499: Feedforward networks
- G06N 3/063: using electronic means
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Disclosed are systems, methods, and other implementations that include a machine-implemented artificial neural network including a plurality of nodes, with the nodes forming at plurality of layers including an input layer, at least one hidden layer, and an output layer, and a plurality of links, with each link coupling a corresponding source node and a receiving node. At least one link is configured to evaluate a piecewise linear function of a value provided by a first source node, from the plurality of nodes, to yield a value for a first receiving node coupled to the at least one link Each node of a hidden layer is configured to aggregate values provided by links for which that node is the receiving node of the link, with the first receiving node providing non-linear output resulting, in part, from the at least one link configured to evaluate the piecewise linear function.

## Claims

### Claim 1

A machine-implemented artificial neural network comprising:
a plurality of nodes, the nodes forming at plurality of layers including an input layer, at least one hidden layer, and an output layer; and
a plurality of links, each link coupling a corresponding source node and a receiving node of the plurality of nodes;
wherein at least one link of the plurality of links is configured to evaluate a piecewise linear function of a value provided by a first source node, from the plurality of nodes, to yield a value for a first receiving node coupled to the at least one link;
and wherein each node of a hidden layer is configured to aggregate values provided by links for which that node is the receiving node of the link, with the first receiving node providing non-linear output resulting, in part, from the at least one link, directed to the first receiving node, configured to evaluate the piecewise linear function.

### Claim 2 (depends on 1)

The neural network of claim 1, wherein multiple ones of the plurality of links are configured to evaluate respective piecewise linear functions based on input values from respective source nodes coupled to the multiple ones of the plurality of links.

### Claim 3 (depends on 2)

The neural network of claim 2, wherein the respective piecewise linear functions correspond to different piecewise linear functions.

### Claim 4 (depends on 1)

The neural network of claim 1, wherein the at least one link configured to evaluate the piecewise linear function is configured to evaluate the piecewise linear function defined according to a plurality of adjustable coefficients defining respective linear sections of the piecewise linear function.

### Claim 5 (depends on 4)

The neural network of claim 4, wherein the adjustable coefficients are retrieved from a table storing data for the plurality of sections of the piecewise linear function, wherein each linear section is represented based on breakpoints associated with the respective linear sections.

### Claim 6 (depends on 4)

The neural network of claim 4, wherein the adjustable coefficients are computed according to training data used to minimize a loss measure between predictions made by the neural network and pre-labeled instances.

### Claim 7 (depends on 4)

The neural network of claim 4, wherein the piecewise linear function is represented according to:
f
⁡
(
x
;
w
→
)
=
{
(
x
-
a
0
a
1
-
a
0
)
⁢
w
1
+
(
a
1
-
x
a
1
-
a
0
)
⁢
w
0
x
<
a
1
(
x
-
a
k
a
k
+
1
-
a
k
)
⁢
w
k
+
1
+
(
a
k
+
1
-
x
a
k
+
1
-
a
k
)
⁢
w
k
,
a
k
≤
x
≤
a
k
+
1
,
k
=
1
,
…
⁢
,
n
-
2
(
x
-
a
n
-
1
a
n
-
a
n
-
1
)
⁢
w
n
+
(
a
n
-
x
a
n
-
a
n
-
1
)
⁢
w
n
-
1
x
≥
a
n
-
1
where a0<a1< . . . <an are pre-determined n+1 ordered break points, and wk, 0≤k≤n+1, are the corresponding adjustable coefficients.

### Claim 8 (depends on 1)

The neural network of claim 1, wherein the non-linear output of the first receiving node is computed entirely based on the sum of the values produced by the links to the first receiving node without requiring an activation function applied to the computed sum of values at the first receiving node.

### Claim 9 (depends on 1)

The neural network of claim 1, wherein the at least one link of the plurality of links configured to evaluate the piecewise linear function comprises:
a first set of intermediate links extending from the first source node into intermediate nodes located between a first layer comprising the first source node and a second layer comprising the first receiving node, and a second set of intermediate links extending from output of the intermediate nodes to the first receiving node, each of the multiple links and respective intermediate nodes realizing a respective basis function such that the composite of outputs directed from the intermediate nodes to the first receiving node through the second set of intermediate links realizes the piecewise linear function.

### Claim 10 (depends on 9)

The neural network of claim 9, wherein each of the respective basis function comprises a rectifier linear unit (ReLU) function.

### Claim 11 (depends on 1)

The neural network of claim 1, wherein the output layer is configured to produce a plurality of time-varying gain components (g) in response to a plurality of phase-invariant derived signals (r) provided at the input layer, wherein each derived signal (rj) of the plurality of derived signals is determined as a non-linear function of one or more transformed signals (w) derived from an input signal (u);
wherein a distortion term is formed by accumulating a plurality of terms, each term being a product of a transformed signal of the one or more transformed signals (w) and a time-varying gain, the time-varying gain being a function of a one or more of the phase-invariant derived signals (r), the function of the one or more of the phase-invariant derived signals being decomposable into a combination of one or more parametric functions of a corresponding single one of the time-varying gain components (gi), and wherein the distortion term is combined with the input signal (u) to mitigate non-linear distortions caused by an amplification system configured to process the input signal (u).

### Claim 12

A method comprising:
receiving input values at input nodes of an input layer of a neural network comprising a plurality of nodes, at least some of the plurality of nodes further arranged in at least one hidden layer and an output layer;
computing for a receiving node, from the plurality of nodes, a resultant value according to functions realized by respective links coupling the receiving node to respective source nodes, wherein computing the resultant value for the receiving node comprises evaluating, by at least one link of the respective links coupling the receiving node to the respective source nodes, a piecewise linear function implemented on the at least one link and applied to a first source node from the respective source nodes; and
aggregating at the receiving node values provided by the respective links coupling the receiving node to the respective source nodes to provide a non-linear output resulting, in part, from the evaluated piecewise linear function applied to the first source node by the piecewise linear function implemented by the at least one link.

### Claim 13 (depends on 12)

The method of claim 12, wherein computing the resultant value for the receiving node comprises:
evaluating by multiple links of the respective links coupling the receiving node to the respective source nodes, piecewise linear functions implemented on the multiple links.

### Claim 14 (depends on 12)

The method of claim 12, wherein evaluating the piecewise linear function comprises:
evaluating the piecewise linear function according to a plurality of adjustable coefficients defining respective linear sections of the piecewise linear function.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
retrieving from a table storing data for the plurality of sections of the piecewise linear function, an adjustable coefficient and data for a particular linear segment of the piecewise linear function based on a value of the first node.

### Claim 16 (depends on 14)

The method of claim 14, further comprising:
deriving the adjustable coefficients according to training data used to minimize a loss measure between predictions made by the neural network and pre-labeled instances.

### Claim 17 (depends on 14)

The method of claim 14, wherein evaluating the piecewise linear function comprises:
evaluating the piecewise linear function according to:
f
⁡
(
x
;
w
→
)
=
{
(
x
-
a
0
a
1
-
a
0
)
⁢
w
1
+
(
a
1
-
x
a
1
-
a
0
)
⁢
w
0
x
<
a
1
(
x
-
a
k
a
k
+
1
-
a
k
)
⁢
w
k
+
1
+
(
a
k
+
1
-
x
a
k
+
1
-
a
k
)
⁢
w
k
,
a
k
≤
x
≤
a
k
+
1
,
k
=
1
,
…
⁢
,
n
-
2
(
x
-
a
n
-
1
a
n
-
a
n
-
1
)
⁢
w
n
+
(
a
n
-
x
a
n
-
a
n
-
1
)
⁢
w
n
-
1
x
≥
a
n
-
1
where a0<a1< . . . <an are pre-determined n+1 ordered break points, and wk, 0≤k≤n+1, are the corresponding trainable coefficients.

### Claim 18 (depends on 12)

The method of claim 12, wherein the at least one link implementing the piecewise linear function comprises:
a first set of intermediate links extending from the first source node into intermediate nodes located between a first layer comprising the first source node and a second layer comprising the receiving node, and a second set of intermediate links extending from output of the intermediate nodes to the receiving node, each of the multiple links and respective intermediate nodes realizing a respective basis function such that the composite of outputs directed from the intermediate nodes to the first receiving node through the second set of intermediate links realizes the piecewise linear function.

### Claim 19 (depends on 12)

The method of claim 12, further comprising:
generating, at the output layer a plurality of time-varying gain components (g) in response to a plurality of phase-invariant derived signals (r) received at the input nodes of the input layer, wherein each derived signal (rj) of the plurality of derived signals is determined as a non-linear function of one or more transformed signals (w) derived from an input signal (u);
forming a distortion term by accumulating a plurality of terms, each term being a product of a transformed signal of the one or more transformed signals (w) and a time-varying gain, the time-varying gain being a function of a one or more of the phase-invariant derived signals (r), the function of the one or more of the phase-invariant derived signals being decomposable into a combination of one or more parametric functions of a corresponding single one of the time-varying gain components (gi); and
combining the distortion term with the input signal (u) to mitigate non-linear distortions caused by an amplification system configured to process the input signal (u).

### Claim 20

A design structure encoded on a non-transitory machine-readable medium, said design structure comprising elements that, when processed in a computer-aided design system, generate a machine-executable representation of a neural network comprising:
a receiving circuit to receiving input values at input nodes of an input layer of the neural network comprising a plurality of nodes, at least some of the plurality of nodes further arranged in at least one hidden layer and an output layer;
a computing circuit to compute for a receiving node, from the plurality of nodes, a resultant value according to functions realized by respective links coupling the receiving node to respective source nodes, wherein computing the resultant value for the receiving node comprises evaluating, by at least one link of the respective links coupling the receiving node to the respective source nodes, a piecewise linear function implemented on the at least one link and applied to a first source node from the respective source nodes; and
an aggregation circuit to aggregate at the receiving node values provided by the respective links coupling the receiving node to the respective source nodes to produce a non-linear output based, in part, on the evaluated piecewise linear function applied to the first source node by the piecewise linear function implemented by the at least one link.
