# US 11,113,606 B2 · Learning method, learning device, program, and recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,113,606 B2
- Document ID: US11113606B2
- Status: Granted
- Kind: B2
- Application number: 16/679,620
- Filed: 2019-11-11
- Published: 2020-06-04
- Granted: 2021-09-07
- Earliest claimed: 2018-11-30
- Assignees: Konica Minolta, Inc.
- Applicants: Konica Minolta, Inc.
- Inventors: Taiki Sekii
- Web page: https://patentgrep.com/patent/US11113606B2

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 5/045: Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/776: Validation; Performance evaluation
- G06V 10/82: using neural networks
- G06V 20/64: Three-dimensional [3D] objects
- G06V 20/80: Recognising image objects characterised by unique random patterns
- G06T 2207/20081: Training; Learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Disclosed is a learning method that may include generating learning data that may contain a composite image including a CG model of an object and may contain a training signal of the object. Further disclosed is a way of learning a recognition function of recognizing information regarding the object from the composite image by neuro computation using the learning data, the generated learning data including new learning data on the basis of a gradient of error for each of pixels of the composite image, which may be calculated from the composite image and the training signal by backpropagation. The disclosed learning may include learning the recognition function by using the new learning data.

## Claims

### Claim 1

A learning method, comprising:
generating learning data that contains a composite image, including a computer graphics (CG) model of an object, and contains a training signal of the object; and
learning a recognition function of recognizing information regarding the object from the composite image by neuro computation using the learning data,
wherein the learning data is generated by generating new learning data on the basis of a gradient of error for each pixel of the composite image calculated from the composite image and the training signal by backpropagation, and
wherein the learning includes learning the recognition function by using the new learning data.

### Claim 2 (depends on 1)

The learning method according to claim 1, wherein, on the basis of the gradient of error for each pixel calculated for each of a plurality of CG models included in the composite image, and
wherein the learning data is generated by replacing the CG model included in the composite image with another CG model.

### Claim 3 (depends on 1)

The learning method according to claim 1, wherein the learning data is generated by converting a quality of the composite image into a different image quality by neuro computation.

### Claim 4 (depends on 1)

The learning method according to claim 1, wherein the learning includes (i) calculating a parameter gradient related to a parameter of the CG model of the object by using the gradient of error and (ii) correcting the parameter of the CG model of the object on the basis of the parameter gradient.

### Claim 5 (depends on 4)

The learning method according to claim 4, wherein the learning includes correcting the parameter by adding a value obtained by multiplying the calculated parameter gradient by a predetermined negative coefficient, to the parameter of the CG model of the object.

### Claim 6 (depends on 4)

The learning method according to claim 4, wherein the parameter of the CG model of the object includes information regarding at least one of a three-dimensional position, an angle, a color, a pattern, a shape, a reflection characteristic, or an illumination condition of the object.

### Claim 7

A learning device, comprising:
a learning data generator configured to generate learning data that contains a composite image, including a computer graphics (CG) model of an object, and contains a training signal of the object;
an object recognizer configured to recognize information regarding an object included in an input image by neuro computation; and
a learning processor configured to learn a recognition function of the object recognizer,
wherein the learning processor calculates a gradient of error for each pixel of the composite image, from the composite image and the training signal, by backpropagation,
wherein the learning data generator generates new learning data on the basis of the gradient of error, and
wherein the learning processor learns the recognition function by using the new learning data.

### Claim 8 (depends on 7)

The learning device according to claim 7, wherein the learning processor calculates the gradient of error for each pixel calculated for each of a plurality of CG models included in the composite image, and
wherein the learning data generator replaces the CG model included in the composite image with another CG model on the basis of the gradient of error for each pixel.

### Claim 9 (depends on 7)

The learning device according to claim 7, wherein the learning data generator converts a quality of the composite image into a different image quality by neuro computation.

### Claim 10 (depends on 7)

The learning device according to claim 7, wherein the learning processor (i) calculates a parameter gradient related to a parameter of the CG model of the object by using the gradient of error and (ii) corrects the parameter of the CG model of the object on the basis of the parameter gradient.

### Claim 11 (depends on 10)

The learning device according to claim 10, wherein the learning processor corrects the parameter by adding a value obtained by multiplying the calculated parameter gradient by a predetermined negative coefficient, to the parameter of the CG model of the object.

### Claim 12 (depends on 10)

The learning device according to claim 10, wherein the parameter of the CG model of the object includes information regarding at least one of a three-dimensional position, an angle, a color, a pattern, a shape, a reflection characteristic, or an illumination condition of the object.

### Claim 13 (depends on 1)

A non-transitory recording medium storing a computer readable program for causing a computer to execute the learning method according to claim 1.