# US 2022/0284567 A1 · Teacher data generation method, trained learning model, and system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284567 A1
- Document ID: US20220284567A1
- Status: Published application
- Kind: A1
- Application number: 17/713,956
- Filed: 2022-04-05
- Published: 2022-09-08
- Earliest claimed: 2019-10-08
- Assignees: Canon Kabushiki Kaisha
- Applicants: Canon Kabushiki Kaisha
- Inventors: Takeru Ohya; Kazuya Hizume; Yuta Okabe; Takayuki Hashimoto
- Web page: https://patentgrep.com/patent/US20220284567A1

## CPC

- G06T 7/0004: {Industrial image inspection}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 7/12: Edge-based segmentation
- G01N 2021/8883: {involving the calculation of gauges, generating models}
- G06T 2207/10024: Color image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30164: Workpiece; Machine component
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)

## Abstract

According to one aspect of the present invention, a method of generating teacher data for image recognition includes acquiring image data by capturing an image of a workpiece, and segmenting the image data into a plurality of first areas, marking whether predetermined information is included in each of the plurality of first areas, and generating a plurality of pieces of teacher data.

## Claims

### Claim 1

A method of generating teacher data for image recognition, comprising:
acquiring image data by capturing an image of a workpiece; and
segmenting the image data into a plurality of first areas, marking whether predetermined information is included in each of the plurality of first areas, and generating a plurality of pieces of teacher data.

### Claim 2 (depends on 1)

The teacher data generation method according to claim 1, wherein in the creation of the plurality of pieces of teacher data, the plurality of pieces of teacher data is generated by segmenting the image data into a plurality of second areas by shifting a position of the image data to be segmented and marking whether the predetermined information is included in each of the plurality of second areas.

### Claim 3 (depends on 1)

The teacher data generation method according to claim 1, wherein the predetermined information indicates whether a defect is present.

### Claim 4 (depends on 1)

The teacher data generation method according to claim 1,
wherein the workpiece is a colored part, and
wherein the predetermined information indicates whether the workpiece is colored with a color different from a predetermined color.

### Claim 5 (depends on 1)

The teacher data generation method according to claim 1, wherein the acquisition of the image data and the generation of the plurality of pieces of teacher data are performed on each of a non-defective workpiece and a defective workpiece.

### Claim 6 (depends on 1)

A method of performing machine learning, comprising:
inputting teacher data generated by the teacher data generation method according to claim 1 to a learning model.

### Claim 7

A trained learning model obtained by machine learning using image data obtained by capturing an image of a workpiece, the image data including a first portion with a first background design and a second portion with a second background design different from the first background design, each of the first portion and the second portion being segmented into a plurality of first areas,
the trained learning model comprising:
a first trained learning model; and
a second trained learning model,
wherein the first trained learning model is trained by machine learning using a plurality of pieces of teacher data marking whether predetermined information is included in each of the plurality of first areas in the first portion, and
wherein the second trained learning model is trained by machine learning using a plurality of pieces of teacher data marking whether the predetermined information is included in each of the plurality of first areas in the second portion.

### Claim 8 (depends on 7)

The trained learning model according to claim 7,
wherein an area including the predetermined information and an area not including the predetermined information are each separated into a plurality of components, and
wherein teacher data for obtaining estimated values of the plurality of components in the area including the predetermined information and teacher data for obtaining estimated values of the plurality of components in the area not including the predetermined information are input to a learning model and the learning model is trained by machine learning.

### Claim 9 (depends on 8)

The trained learning model according to claim 8, wherein the plurality of components includes a color component, hue, saturation, and value.

### Claim 10 (depends on 7)

The trained learning model according to claim 7, wherein the learning model uses one of a nearest neighbor method, a naive Bayes method, a decision tree, a support-vector machine, and a neural network.

### Claim 11

A learning method, comprising:
inputting image data including a first portion with a first background design and a second portion with a second background design different from the first background design;
training by machine learning using a plurality of pieces of teacher data marking whether predetermined information is included in each of a plurality of first areas in the first portion; and
training by machine leaning using a plurality of pieces of teacher data marking whether the predetermined information is included in each of the plurality of first areas in the second portion.

### Claim 12 (depends on 11)

The learning method according to claim 11,
wherein an area including the predetermined information and an area not including the predetermined information are each separated into a plurality of components, and
wherein teacher data for obtaining estimated vales of the plurality of components in the area including the predetermined information and teacher data for obtaining estimated values of the plurality of components in the area not including the predetermined information are input to a learning model and the learning model is trained by machine learning.

### Claim 13

A learning method, comprising:
inputting image data obtained by capturing an image of a workpiece;
segmenting the image data into a first area including predetermined information and a second area not including the predetermined information;
separating each of the first area and the second area into a plurality of components; and
training by machine learning using teacher data for obtaining estimated values of the plurality of components in the first area and teacher data for obtaining estimated values of the plurality of components in the second area.

### Claim 14 (depends on 11)

The learning method according to claim 11, wherein the predetermined information indicates whether a defect is present.

### Claim 15 (depends on 13)

The learning method according to claim 13, wherein the plurality of components includes a color component, hue, saturation, and value.

### Claim 16

A system comprising:
an image capturing apparatus; and
a processing apparatus including a trained learning model configured to receive image data obtained by capturing an image by the image capturing apparatus,
wherein the trained learning model is created by marking whether predetermined information is included in each of a plurality of first areas in image data obtained by capturing an image of a first workpiece by the image capturing apparatus, and
wherein second image data obtained by capturing an image of a second workpiece by the image capturing apparatus is input to the trained learning model and a determination is made on the second workpiece based on the second image data.

### Claim 17

A system comprising:
an image capturing apparatus; and
a processing apparatus including a trained learning model configured to receive image data obtained by capturing an image by the image capturing apparatus,
wherein the trained learning model is created by inputting difference information indicating a difference between first image data obtained by capturing an image of a first workpiece by the image capturing apparatus and second image data different from the first image data, and
wherein difference information indicating a difference between third image data obtained by capturing an image of a second workpiece by the image capturing apparatus and fourth image data different from the third image data is input to the trained learning model and a determination is made on the second workpiece.

### Claim 18 (depends on 17)

The system according to claim 17, wherein the second image data and the fourth image data are image data obtained by capturing an image of a non-defective workpiece.

### Claim 19 (depends on 17)

The system according to claim 17,
wherein the first workpiece includes a repetitive pattern,
wherein the second image data indicates a shift image obtained by shifting the first image data by an amount corresponding to one or more repetitive patterns in image data obtained by capturing an image of the first workpiece, and
wherein the fourth image data indicates a shift image obtained by shifting the third image data by an amount corresponding to one or more repetitive patterns in image data obtained by capturing an image of the second workpiece.

### Claim 20 (depends on 19)

The system according to claim 19,
wherein the difference information indicating the difference between the first image data and the second image data to be input to the learning model indicates information subjected to predetermined processing, and
wherein in a case where an absolute value of a difference value in an area including the difference indicated by the difference information between the second image data and the first image data exceeds a predetermined threshold, the predetermined processing is processing of correcting the area to an area including no difference.

### Claim 21 (depends on 17)

The system according to claim 17, wherein the predetermined information indicates whether a defect is present.

### Claim 22 (depends on 17)

The system according to claim 17,
wherein the first workpiece is a colored part, and
wherein the predetermined information indicates whether the first workpiece is colored with a color different from a predetermined color.

### Claim 23 (depends on 17)

The system according to claim 17, further comprising:
a sensor; and
a trigger generation circuit configured to transmit an image capturing trigger signal to the image capturing apparatus,
wherein upon detection of the second workpiece within a predetermined range, the sensor outputs a signal to the trigger generation circuit, and
wherein the image capturing apparatus captures an image of the first workpiece based on the image capturing trigger signal output from the trigger generation circuit based on the signal.

### Claim 24 (depends on 23)

The system according to claim 23, further comprising a robot,
wherein the system causes a workpiece determined to be defective based on the trained learning model to move.
