# US 2024/0394019 A1 · System, method, and apparatus for recurrent neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0394019 A1
- Document ID: US20240394019A1
- Status: Published application
- Kind: A1
- Application number: 18/794,981
- Filed: 2024-08-05
- Published: 2024-11-28
- Earliest claimed: 2018-02-27
- Assignees: New York University
- Applicants: New York University
- Inventors: David J. HEEGER; Wayne E. MACKEY
- Web page: https://patentgrep.com/patent/US20240394019A1

## CPC

- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/063: using electronic means
- G06N 3/065: Analogue means
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 20/00: Machine learning
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for computation with recurrent neural networks includes receiving an input drive and a recurrent drive, producing at least one modulatory response; computing at least one output response, each output response including a sum of: (1) the input drive multiplied by a function of at least one of the at least one modulatory response, each input drive including a function of at least one input, and (2) the recurrent drive multiplied by a function of at least one of the at least one modulatory response, each recurrent drive including a function of the at least one output response, each modulatory response including a function of at least one of (i) the at least one input, (ii) the at least one output response, or (iii) at least one first offset, and computing a readout of the at least one output response.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A method for computation with recurrent neural networks, comprising:
producing, by each of one or more modulators of one or more computational engines, at least one modulatory response;
computing, by the one or more computational engines, at least one output response, each output response comprising a sum of:
(1) an input drive multiplied by the at least one modulatory response, the input drive comprising a weighted sum based on a plurality of inputs and at least one first offset, and
(2) a recurrent drive multiplied by the at least one modulatory response, each recurrent drive comprising a weighted sum based on the at least one output response and at least one second offset,
wherein each modulatory response comprises a function of at least one of (i) the plurality of inputs, (ii) the at least one output response, or (iii) at least one third offset,
wherein the recurrent drive multiplied by the at least one modulatory response recurrently normalizes the at least one output response; and
computing a readout comprising a function of the at least one output response.

### Claim 22 (depends on 21)

The method of claim 21, comprising determining one or more weights of each weighted sum using back propagation.

### Claim 23 (depends on 21)

The method of claim 21, comprising rectifying the at least one modulatory response.

### Claim 24 (depends on 21)

The method of claim 21, wherein the at least one modulatory response comprises a first modulatory response and a second modulatory response, wherein computing the at least one output response comprises multiplying the input drive the first modulatory response and multiplying the recurrent drive by the second modulatory response.

### Claim 25 (depends on 21)

The method of claim 21, comprising driving the at least one output response based on an energy function.

### Claim 26 (depends on 21)

The method of claim 21, wherein an input to a first computational engine of the one or more computational engines comprises an output of a second computational engine of the one or more computational engines.

### Claim 27 (depends on 21)

The method of claim 21, comprising applying at least one of halfwave rectification or squaring to the at least one output response.

### Claim 28 (depends on 21)

The method of claim 21, wherein producing, by each of the one or more modulators of the one or more computational engines, the at least one modulatory response, comprises using one or more processors to execute the one or more computational engines in memory.

### Claim 29

A system, comprising:
one or more processors configured to:
produce, by each of one or more modulators, at least one modulatory response;
compute at least one output response, each output response comprising a sum of:
(1) an input drive multiplied the at least one modulatory response, the input drive comprising a weighted sum based on a plurality of inputs and at least one first offset, and
(2) a recurrent drive multiplied by the at least one modulatory response, each recurrent drive comprising a weighted sum based on the at least one output response and at least one second offset, such that the recurrent drive multiplied by the at least one modulatory response recurrently normalizes the at least one output response,
wherein each modulatory response comprises a function of at least one of (i) the plurality of inputs, (ii) the at least one output response, or (iii) at least one third offset; and
compute a readout comprising a function of the at least one output response.

### Claim 30 (depends on 29)

The system of claim 29, wherein the one or more processors are configured to determine one or more weights of each weighted sum using back propagation.

### Claim 31 (depends on 29)

The system of claim 29, wherein the one or more processors are configured to rectify the at least one modulatory response.

### Claim 32 (depends on 29)

The system of claim 29, wherein the at least one modulatory response comprises a first modulatory response and a second modulatory response, and the one or more processors are configured to compute the at least one output response by multiplying the input drive by the first modulatory response and multiplying the recurrent drive by the second modulatory response.

### Claim 33 (depends on 29)

The system of claim 29, wherein the one or more processors are configured to drive the at least one output response based on an energy function.

### Claim 34 (depends on 29)

The system of claim 29, wherein an input to a first computational engine of the one or more computational engines comprises an output of a second computational engine of the one or more computational engines.

### Claim 35 (depends on 29)

The system of claim 29, wherein the one or more processors are configured to apply at least one of halfwave rectification or squaring to the at least one output response.

### Claim 36 (depends on 29)

The system of claim 29, wherein the function of the at least one output response comprises a weighted sum of the at least one output response.

### Claim 37

A device, comprising:
an analog electrical circuit configured to:
produce, by each of one or more modulators, at least one modulatory response;
compute at least one output response, each output response comprising a sum of:
(1) an input drive multiplied by the at least one modulatory response, the input drive comprising a weighted sum based on a plurality of inputs and at least one first offset, and
(2) a recurrent drive multiplied by the at least one modulatory response, each recurrent drive comprising a weighted sum based on the at least one output response and at least one second offset, such that the recurrent drive multiplied by at least one modulatory response recurrently normalizes the at least one output response,
wherein each modulatory response comprises a function of at least one of (i) the plurality of inputs, (ii) the at least one output response, or (iii) at least one third offset; and
compute a readout comprising a function of the at least one output response.

### Claim 38 (depends on 37)

The device of claim 37, wherein the analog electrical circuit comprises a first computational unit to compute the readout, and is configured to provide the readout as input to a second computational unit.

### Claim 39 (depends on 37)

The device of claim 37, wherein the analog electrical circuit comprises a very large scale integration (VLSI) circuit.

### Claim 40 (depends on 37)

The device of claim 37, wherein the analog electrical circuit is configured to apply at least one of halfwave rectification or squaring to the at least one output response.
