# US 2024/0193414 A1 · Physics aware training for deep physical neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0193414 A1
- Document ID: US20240193414A1
- Status: Published application
- Kind: A1
- Application number: 18/287,808
- Filed: 2022-04-21
- Published: 2024-06-13
- Earliest claimed: 2021-04-22
- Assignees: NTT Research, Inc.
- Applicants: Ntt Research, Inc.
- Inventors: Logan G. WRIGHT; Tatsuhiro ONODERA; Peter L. MCMAHON; Martin STEIN
- Web page: https://patentgrep.com/patent/US20240193414A1

## CPC

- A61N 1/36128: {Control systems}
- G06N 3/08: Learning methods
- G06J 3/00: Systems for conjoint operation of complete digital and complete analogue computers
- G06N 3/063: using electronic means
- G06N 3/067: using optical means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- A61N: ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY (measurement of bioelectric currents A61B; surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body A61B18/00; anaesthetic apparatus in general A61M; incandescent lamps H01K; infrared radiators for heating H05B)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06J: HYBRID COMPUTING ARRANGEMENTS (optical hybrid computing devices G06E3/00)

## Abstract

A physical neural network system includes a physical and digital component. The digital component includes a computing system. The physical component and the digital component work in conjunction to execute a physics aware training process. The physics aware training process includes generating, by the digital component, an input data set for input to the physical component, applying, by the physical component, one or more transformations to the input data set to generate an output for a forward pass of the physics aware training process, comparing, by the digital component, the generated output to a canonical output to determine an error, generating, by the digital component, a loss gradient using a differentiable digital model for a backward pass of the physics aware training process, and updating, by the digital component, training parameters for subsequent input to the physical component based on the loss gradient.

## Claims

### Claim 1

A physical neural network system, comprising:
a physical component; and
a digital component comprising a computing system, wherein the physical component and the digital component work in conjunction to execute a physics aware training process, comprising:
generating, by the digital component, an input data set for input to the physical component;
applying, by the physical component, one or more transformations to the input data set to generate an output for a forward pass of the physics aware training process;
based on the generated output, comparing, by the digital component, the generated output to a canonical output to determine an error;
generating, by the digital component, a loss gradient using a differentiable digital model for a backward pass of the physics aware training process; and
updating, by the digital component, training parameters for subsequent input to the physical component based on the loss gradient.

### Claim 2 (depends on 1)

The physical neural network system of claim 1, wherein applying, by the physical component, the one or more transformations to the input data set to generate the output for the forward pass of the physics aware training process comprises:
employing at least one non-differentiable transformation function to the input data set in the forward pass.

### Claim 3 (depends on 2)

The physical neural network system of claim 2, wherein generating, by the digital component, the loss gradient using the differentiable digital model for the backward pass of the physics aware training process comprises:
approximating the loss gradient using the differentiable digital model, wherein the differentiable digital model in the backward pass is different from the at least one non-differentiable transformation function of the forward pass.

### Claim 4 (depends on 1)

The physical neural network system of claim 1, wherein the physical component comprises a first layer and a second layer.

### Claim 5 (depends on 4)

The physical neural network system of claim 4, wherein applying, by the physical component, the one or more transformations to generate the output for the forward pass of the physics aware training process comprises:
applying, by the first layer, a first transformation to the input data set to generate an intermediary output; and
applying, by the second layer, a second transformation to the input data set and the intermediary output to generate the output.

### Claim 6 (depends on 1)

The physical neural network system of claim 1, wherein generating, by the digital component, the input data set for input to the physical component comprises:
encoding input data and initial parameters to generate the input data set.

### Claim 7 (depends on 6)

The physical neural network system of claim 6, further comprising:
generating, by the digital component, an updated input data set by encoding the input data and the updated training parameters.

### Claim 8

A method of training a physical neural network, comprising:
generating, by a digital component of the physical neural network, an input data set for input to the physical component;
applying, by a physical component of the physical neural network, one or more transformations to the input data set to generate an output for a forward pass of the training;
based on the generated output, comparing, by the digital component, the generated output to a canonical output to determine an error;
generating, by the digital component, a loss gradient using a differentiable digital model for a backward pass of the training; and
updating, by the digital component, training parameters for subsequent input to the physical component based on the loss gradient.

### Claim 9 (depends on 8)

The method of claim 8, wherein applying, by the physical component, the one or more transformations to the input data set to generate the output for the forward pass of the training comprises:
employing at least one non-differentiable transformation function to the input data set in the forward pass.

### Claim 10 (depends on 9)

The method of claim 9, wherein generating, by the digital component, the loss gradient using the differentiable digital model for the backward pass of the training comprises:
approximating the loss gradient using the differentiable digital model, wherein the differentiable digital model in the backward pass is different from the at least one non-differentiable transformation function of the forward pass.

### Claim 11 (depends on 8)

The method of claim 8, wherein the physical component comprises a first layer and a second layer.

### Claim 12 (depends on 11)

The method of claim 11, wherein applying, by the physical component, the one or more transformations to generate the output for the forward pass of the training comprises:
applying, by the first layer, a first transformation to the input data set to generate an intermediary output; and
applying, by the second layer, a second transformation to the input data set and the intermediary output to generate the output.

### Claim 13 (depends on 8)

The method of claim 8, wherein generating, by the digital component, the input data set for input to the physical component comprises:
encoding input data and initial parameters to generate the input data set.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
generating, by the digital component, an updated input data set by encoding the input data and the updated training parameters.

### Claim 15

A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by one or more processors, causes a computing system to perform operations, comprising:
generating an input data set for input to a physical component of a physical neural network;
causing the physical component of the physical neural network to apply one or more transformations to the input data set to generate an output for a forward pass of a training process;
based on the generated output, comparing the generated output to a canonical output to determine an error;
generating a loss gradient using a differentiable digital model for a backward pass of the training process; and
updating training parameters for subsequent input to the physical component based on the loss gradient.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein applying, by the physical component, the one or more transformations to the input data set to generate the output for the forward pass of the training process comprises:
employing at least one non-differentiable transformation function to the input data set in the forward pass.

### Claim 17 (depends on 16)

The non-transitory computer readable medium of claim 16, wherein generating the loss gradient using the differentiable digital model for the backward pass of the training process comprises:
approximating the loss gradient using the differentiable digital model, wherein the differentiable digital model in the backward pass is different from the at least one non-differentiable transformation function of the forward pass.

### Claim 18 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the physical component comprises a first layer and a second layer.

### Claim 19 (depends on 18)

The non-transitory computer readable medium of claim 18, wherein causing the physical component to apply the one or more transformations to generate the output for the forward pass of the training process comprises:
causing the first layer to apply a first transformation to the input data set to generate an intermediary output; and
causing the second layer to apply a second transformation to the input data set and the intermediary output to generate the output.

### Claim 20 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein generating the input data set for input to the physical component comprises:
encoding input data and initial parameters to generate the input data set.
