# US 2021/0176394 A1 · Calibration operation support device for industrial camera

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0176394 A1
- Document ID: US20210176394A1
- Status: Published application
- Kind: A1
- Application number: 17/110,082
- Filed: 2020-12-02
- Published: 2021-06-10
- Earliest claimed: 2019-12-05
- Assignees: Denso Wave Incorporated
- Applicants: Denso Wave Incorporated
- Inventors: Keiji OKUHARA; Tsuyoshi UEYAMA
- Web page: https://patentgrep.com/patent/US20210176394A1

## CPC

- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay
- H04N 17/002: {for television cameras}
- G06T 2207/30108: Industrial image inspection
- G06T 2207/30204: Marker
- G06T 2207/30244: Camera pose
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A calibration operation support device is applied to a system including an industrial camera and a display device displaying a predetermined image that is an image of a predetermined area being captured by the industrial camera, and includes a processing device configured to perform a calibration of the industrial camera based on captured images that are obtained by capturing a calibration pattern by the industrial camera. The processing device is further configured to instruct the display device to display positions of the calibration patterns in the captured images acquired as images to be used for the calibration in a manner superimposed on the predetermined image.

## Claims

### Claim 1

A calibration operation support device to be applied to a system including an industrial camera capturing a predetermined area set in advance and a display device displaying a predetermined image that is an image of the predetermined area being captured by the industrial camera, the calibration operation support device comprising
a processing device configured to perform a calibration of the industrial camera based on a plurality of captured images obtained by capturing a calibration pattern by the industrial camera, wherein
the processing device is further configured to instruct the display device to display positions of the calibration patterns in the captured images acquired as images to be used for the calibration in a manner superimposed on the predetermined image.

### Claim 2

A calibration operation support device to be applied to a system including an industrial camera capturing a predetermined area set in advance and a display device displaying a predetermined image that is an image of the predetermined area being captured by the industrial camera, the calibration operation support device comprising
a processing device configured to perform a calibration of the industrial camera based on a plurality of captured images obtained by capturing a calibration pattern by the industrial camera, wherein
the processing device is further configured to instruct the display device to display a distribution of an effect that the calibration patterns in the captured images acquired as images to be used for the calibration improve an accuracy of the calibration in the predetermined area in a manner superimposed on the predetermined image.

### Claim 3 (depends on 2)

The calibration operation support device according to claim 2, wherein
the calibration pattern includes a plurality of feature points,
a position of each of the feature points is set as a vertex, a predetermined value is a value that becomes smaller as a distance from the vertex increases, and the distribution of the effect is a distribution of total values obtained by summing the predetermined values at each position.

### Claim 4 (depends on 3)

The calibration operation support device according to claim 3, wherein
the predetermined value is a value based on each normal distribution with the position of each of the feature points as the vertex.

### Claim 5 (depends on 3)

The calibration operation support device according to claim 3, wherein
the predetermined value is a value larger than 0 in the entire predetermined area.

### Claim 6 (depends on 3)

The calibration operation support device according to claim 3, wherein
the processing device is further configured to instruct the display device to display a position of the calibration pattern at which a sum of the total values is smallest in a region corresponding to the calibration pattern in the predetermined area as a position at which the calibration pattern should be arranged in the manner superimposed in the predetermined image.

### Claim 7 (depends on 3)

The calibration operation support device according to claim 3, wherein
the processing device is further configured to regard the predetermined area as a two-dimensional plane, regard the position of each of the feature points as a position on the two-dimensional plane, and instruct the display device to display the distribution of the effect in the manner superimposed on the predetermined image.

### Claim 8 (depends on 7)

The calibration operation support device according to claim 7, wherein
the processing device is further configured to instruct the display device to display a position where the calibration pattern should be arranged in the predetermined area as the two-dimensional plane in the manner superimposed on the predetermined image.

### Claim 9 (depends on 7)

The calibration operation support device according to claim 7, wherein
the processing device is further configured to, after executing the calibration, regard the predetermined area as a three-dimensional space by using parameters related to the industrial camera acquired by the calibration, regard the position of each of the feature points as the position in the three-dimensional space, and instruct the display device to display the distribution of the effect in the manner superimposed on the predetermined image.

### Claim 10 (depends on 9)

The calibration operation support device according to claim 9, wherein
the processing device is further configured to instruct the display device to display a position where the calibration pattern should be arranged in the predetermined area as the three-dimensional space in the manner superimposed on the predetermined image.

### Claim 11 (depends on 1)

The calibration operation support device according to claim 1, wherein
the processing device is further configured to instruct the display device to display a position where the calibration pattern should be arranged in the predetermined area in the manner superimposed on the predetermined image.

### Claim 12 (depends on 1)

The calibration operation support device according to claim 1, wherein
the processing device is further configured to instruct the display device to display heights of the calibration patterns in the captured images acquired as images to be used for the calibration in the manner superimposed on the predetermined image.

### Claim 13 (depends on 1)

The calibration operation support device according to claim 1, wherein
the processing device is further configured to instruct the display device to display a height at which the calibration pattern should be arranged in the predetermined area in the manner superimposed on the predetermined image.

### Claim 14 (depends on 1)

The calibration operation support device according to claim 1, wherein
the processing device is further configured to instruct the display device to display inclinations of the calibration patterns in the captured images acquired as images to be used for the calibration in the manner superimposed on the predetermined image.

### Claim 15 (depends on 1)

The calibration operation support device according to claim 1, wherein
the processing device is further configured to instruct the display device to display how much inclination the calibration pattern should be arranged in the predetermined area in the manner superimposed on the predetermined image.

### Claim 16 (depends on 1)

The calibration operation support device according to claim 1, wherein
the processing device is further configured to acquire the captured image captured by the camera as an image to be used for the calibration when an arrangement state of the calibration pattern in the predetermined area satisfies a predetermined condition.

### Claim 17 (depends on 1)

The calibration operation support device according to claim 1, wherein
the processing device is further configured to perform the calibration when the number of the plurality of captured images acquired as the images to be used for the calibration exceeds a predetermined number.

### Claim 18 (depends on 3)

The calibration operation support device according to claim 3, wherein
the processing device is further configured to instruct the display device to display a position of the calibration pattern at which a sum of the total values in a point cloud coordinate is smallest in a region corresponding to the calibration pattern in the predetermined area as a position at which the calibration pattern should be arranged in the manner superimposed in the predetermined image.

### Claim 19 (depends on 3)

The calibration operation support device according to claim 3, wherein
the processing device is further configured to instruct the display device to display a position of the calibration pattern at which a sum of a distribution created from a point cloud coordinate and the total values is largest in a region corresponding to the calibration pattern in the predetermined area as a position where the calibration pattern should be arranged in the manner superimposed on the predetermined image.

### Claim 20 (depends on 1)

The calibration operation support device according to claim 1, wherein
the processing device includes a central processing unit (CPU) and a storage device storing a program, and the program is configured to, when executed by the CPU, cause the CPU to:
perform the calibration of the industrial camera; and
instruct the display device to display the positions of the calibration patterns in the captured images acquired as the images to be used for the calibration in the manner superimposed on the predetermined image.

### Claim 21 (depends on 2)

The calibration operation support device according to claim 2, wherein
the processing device includes a central processing unit (CPU) and a storage device storing a program, and the program is configured to, when executed by the CPU, cause the CPU to:
perform the calibration of the industrial camera; and
instruct the display device to display the distribution of the effect that the calibration patterns in the captured images acquired as the images to be used for the calibration improve the accuracy of the calibration in the predetermined area in the manner superimposed on the predetermined image.
