# US 2019/0378018 A1 · Information processing device and information processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378018 A1
- Document ID: US20190378018A1
- Status: Published application
- Kind: A1
- Application number: 16/551,435
- Filed: 2019-08-26
- Published: 2019-12-12
- Earliest claimed: 2015-10-29
- Assignees: Preferred Networks, Inc.
- Applicants: Preferred Networks, Inc.
- Inventors: Seiya TOKUI; Yuya UNNO; Kenta OONO; Ryosuke OKUTA
- Web page: https://patentgrep.com/patent/US20190378018A1

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 8/311: {Functional or applicative languages; Rewrite languages}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G06N 3/105: {Shells for specifying net layout}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

There is provided an information processing device which efficiently executes machine learning. The information processing device according to one embodiment includes: an obtaining unit which obtains a source code including a code which defines Forward processing of each layer constituting a neural network; a storage unit which stores an association relationship between each Forward processing and Backward processing associated with each Forward processing; and an executing unit which successively executes each code included in the source code, and which calculates an output value of the Forward processing defined by the code based on an input value at a time of execution of each code, and generates a reference structure for Backward processing in a layer associated with the code based on the association relationship stored in the storage unit.

## Claims

### Claim at position 1

1-6. (canceled)

### Claim 7

An information processing device comprising:
at least one memory configured to store first code for a forward processing of a neural network; and
at least one processor configured to:
calculate, by executing the first code, an output value of the forward processing based on an input value, and
generate, by executing the first code, a structure for a backward processing of the neural network.

### Claim 8 (depends on 7)

The information processing device according to claim 7, wherein
the at least one processor is configured to concurrently execute both the calculation of the output value and the generation of the structure by the execution of the first code.

### Claim 9 (depends on 7)

The information processing device according to claim 7, wherein
the at least one memory is further configured to store second code for the backward processing of the neural network, and
the at least one processor is further configured to execute, after the execution of the first code, the backward processing of the neural network based on the structure by executing the second code.

### Claim 10 (depends on 7)

The information processing device according to claim 7, wherein
the at least one memory is further configured to store third code for unchaining the structure, and
the at least one processor is further configured to unchain the structure from a corresponding layer as a point of origin to an input layer by executing the third code.

### Claim 11 (depends on 7)

The information processing device according to claim 7, wherein
the at least one memory is further configured to store a combination of the first code a control syntax.

### Claim 12 (depends on 7)

The information processing device according to claim 7, wherein
the forward processing is associated with a corresponding backward processing on a one-to-one basis.

### Claim 13 (depends on 7)

The information processing device according to claim 7, wherein
the at least one processor is configured to generate the structure based on a mutual reference of an instance of a function class and an instance of a variable class to each other.

### Claim 14 (depends on 7)

The information processing device according to claim 7, wherein
the at least one processor is configured to cause the structure for the backward processing to grow when a basic calculation of a variable object is performed.

### Claim 15 (depends on 7)

The information processing device according to claim 7, wherein
the structure for the backward processing is a reference structure for the backward processing.

### Claim 16 (depends on 7)

The information processing device according to claim 7, wherein
the at least one processor is further configured to execute the backward processing in reverse order of the forward processing by tracking the structure for the backward processing generated during execution of the forward processing.

### Claim 17 (depends on 7)

The information processing device according to claim 7, wherein
the at least one processor is further configured to calculate an output value of a forward processing of the neural network based on an input value which has a volatile attribute without generating a structure for a backward processing of the neural network.

### Claim 18

A method for processing information using at least one information processing device including at least one processor, comprising:
calculating, by the at least one processor executing first code for a forward processing of a neural network, an output value of the forward processing based on an input value, and
generating, by the at least one processor executing the first code, a structure for a backward processing of the neural network.

### Claim 19 (depends on 18)

The method according to claim 18, further comprising:
concurrently executing, by the at least one processor executing the first code, both the calculation of the output value and the generation of the structure.

### Claim 20 (depends on 18)

The method according to claim 18, further comprising:
after the execution of the first code, executing, by the at least one processor executing second code for the backward processing of the neural network, the backward processing of the neural network based on the structure.

### Claim 21 (depends on 18)

The method according to claim 18, further comprising:
unchaining, by the at least one processor executing third code for unchaining the structure, the structure from a corresponding layer as a point of origin to an input layer.

### Claim 22

(canceled)

### Claim 23 (depends on 18)

The method according to claim 18, further comprising:
associating the forward processing with a corresponding backward processing on a one-to-one basis.

### Claim 24 (depends on 18)

The method according to claim 18, wherein
the structure is generated based on a mutual reference of an instance of a function class and an instance of a variable class to each other.

### Claim 25 (depends on 18)

The method according to claim 18, further comprising:
causing, by the at least one processor, the structure for the backward processing to grow when a basic calculation of a variable object is performed.

### Claim 26 (depends on 18)

The method according to claim 18, wherein
the structure for the backward processing is a reference structure for the backward processing.

### Claim 27 (depends on 18)

The method according to claim 18, further comprising:
executing, by the at least one processor, the backward processing in reverse order of the forward processing by tracking the structure for the backward processing generated during execution of the forward processing.

### Claim 28 (depends on 18)

The method according to claim 18, further comprising:
calculating, by the at least one processor, an output value of a forward processing of the neural network based on an input value which has a volatile attribute without generating a structure for a backward processing of the neural network.

### Claim 29

A non-transitory computer readable medium storing therein a program which, when executed by at least one processor of at least one computer performs a method comprising:
calculating, by executing first code for a forward processing of a neural network, an output value of the forward processing based on an input value; and
generating, by executing the first code, a structure for a backward processing of the neural network.

### Claim 30 (depends on 29)

The non-transitory computer readable medium according to claim 29, wherein the method further comprises:
concurrently executing both the calculation of the output value and the generation of the structure by the execution of the first code.

### Claim 31 (depends on 29)

The non-transitory computer readable medium according to claim 29, wherein
the method further comprises executing, after the execution of the first code, the backward processing of the neural network based on the structure by executing second code for the backward processing of the neural network.

### Claim 32 (depends on 29)

The non-transitory computer readable medium according to claim 29, wherein
the structure for the backward processing is a reference structure for the backward processing.

### Claim 33

An information processing device comprising:
at least one memory; and
at least one processor configured to:
calculate an output value of a forward processing of a neural network based on an input value, and
generate, before defining overall structure of the neural network, a structure for a backward processing of the neural network.

### Claim at position 29

34-35. (canceled)

### Claim 36 (depends on 33)

The information processing device according to claim 33, wherein
the structure for the backward processing is a reference structure for the backward processing.

### Claim 37 (depends on 33)

The information processing device according to claim 33, wherein
the at least one processor is further configured to calculate, after generating the structure for the backward processing, another output value of another forward processing of the neural network based on the calculated output value.

### Claim 38 (depends on 33)

The information processing device according to claim 33, wherein
the at least one processor is further configured to execute, after generating the structure for the backward processing, the backward processing of the neural network.

### Claim 39 (depends on 33)

The information processing device according to claim 33, wherein
a calculation procedure for the backward processing of the neural network is decided based on the structure for the backward processing.
