# US 2025/0166293 A1 · Method for generating three-dimensional diagram of object, three-dimensional diagram generation apparatus and three-dimensional shape inspection apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166293 A1
- Document ID: US20250166293A1
- Status: Published application
- Kind: A1
- Application number: 18/727,349
- Filed: 2022-01-18
- Published: 2025-05-22
- Earliest claimed: 2022-01-18
- Assignees: Yamaha Robotics Holdings Co., Ltd.
- Applicants: Yamaha Robotics Holdings Co., Ltd.
- Inventors: Hirokatsu MURAMATSU; Takuya ADACHI
- Web page: https://patentgrep.com/patent/US20250166293A1

## CPC

- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G01B 11/245: using a plurality of fixed, simultaneously operating transducers ({G01B11/2408 - G01B11/2425, } G01B11/255 take precedence)
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/0008: {checking presence/absence}
- G06T 7/13: Edge detection
- G06T 7/564: from contours
- G06V 10/46: Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56)
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30141: Printed circuit board [PCB]
- G06T 2207/30148: Semiconductor; IC; Wafer
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A three-dimensional diagram generation apparatus (100) includes a controller (50) that generates a three-dimensional diagram of a semiconductor device based on an image obtained by imaging the semiconductor device. The controller (50) acquires a vertical view image and multiple oblique view images of the semiconductor device, extracts a contour line in the vertical view image to generate a vertical view diagram of the semiconductor device, converts the vertical view diagram into multiple oblique view diagrams based on a shape parameter stored in a storage part, repeatedly execute adjustment of the shape parameter and conversion of the vertical view diagram into each oblique view diagram until each converted oblique view diagram overlaps with each oblique view image, and synthesizes the generated vertical view diagram and each oblique view diagram overlapping with each oblique view image to generate the three-dimensional diagram.

## Claims

### Claim 1

A method for generating a three-dimensional diagram of a three-dimensional object based on a vertical view image of the object imaged from vertically above and multiple oblique view images of the object imaged from multiple obliquely upper directions, the method comprising:
a vertical view diagram generation step which extracts a contour line in the vertical view image to generate a vertical view diagram of the object;
a diagram conversion step which respectively converts the vertical view diagram into multiple oblique view diagrams based on a shape parameter including height information of the object, and repeatedly executes adjustment of the shape parameter and conversion of the vertical view diagram into each of the oblique view diagrams until each of the converted oblique view diagrams overlaps with each of the oblique view images; and
a diagram synthesis step which synthesizes the generated vertical view diagram and each of the converted oblique view diagrams to generate the three-dimensional diagram of the object.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the object is a device composed of multiple components and multiple wires connecting between the components,
the vertical view diagram generation step extracts each contour line of each component and each of the wires in the vertical view image to generate the vertical view diagram of the device, and
the shape parameter is height of each component from a reference surface, inclination of a surface of each component, and a bending parameter of each of the wires.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the vertical view diagram generation step generates a contour line in the vertical view image drawn by a user as the vertical view diagram, and
the diagram conversion step converts the vertical view diagram into each of the oblique view diagrams based on the adjusted shape parameter input by the user.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the object is a device composed of multiple components and multiple wires connecting between the components,
the vertical view diagram generation step generates each contour line of each component and each of the wires in the vertical view image drawn by the user as the vertical view diagram of the device, and
the shape parameter is height of each component from a reference surface, inclination of a surface of each component, and a bending parameter of each of the wires.

### Claim 5 (depends on 2)

The method according to claim 2, wherein each of the wires has multiple bending points between a starting point and an ending point,
the bending parameter is a three-dimensional coordinate position of each of the bending points of each of the wires, and
the three-dimensional coordinate position of each of the bending points is a combination of a longitudinal direction coordinate position, a lateral direction coordinate position, and a height direction coordinate position in a coordinate system composed of a longitudinal direction axis extending from the starting point to the ending point of the wire in a plane of the reference surface, a lateral direction axis extending in a direction orthogonal to the longitudinal direction axis from the starting point of the wire in the plane of the reference surface, and a height direction axis extending in a vertical direction with respect to the reference surface through the starting point.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the three-dimensional coordinate position of each of the bending points comprises a three-dimensional proportional coordinate position which is a combination of a proportional longitudinal direction coordinate position proportional to a wire total length between the starting point and the ending point, a proportional lateral direction coordinate position proportional to the wire total length, and a proportional height direction coordinate position proportional to the wire total length.

### Claim 7 (depends on 2)

The method according to claim 2, wherein the vertical view diagram generation step groups the multiple wires into multiple groups composed of the wires having the starting points located on the same component, the ending points located on the same component, and the same thickness, and extracts each contour line of each component and each of the wires in the vertical view image to generate the vertical view diagram of the device,
the bending parameter is composed of multiple group-specific bending parameters defined for each of the groups, and
the diagram conversion step respectively converts each vertical view diagram of each of the wires included in each group into multiple oblique view diagrams based on each of the group-specific bending parameters, and repeatedly executes adjustment of each of the group-specific bending parameters of each of the groups and conversion of each vertical view diagram of the wires included in each of the groups into the oblique view diagrams until each of the converted oblique view diagrams of each of the wires included in each of the groups overlaps with each of the oblique view images of each of the wires included in each of the groups.

### Claim 8 (depends on 4)

The method according to claim 4, wherein the vertical view diagram generation step groups the multiple wires into multiple groups composed of the wires having the starting points located on the same component, the ending points located on the same component, and the same thickness, and extracts each contour line of each component and each of the wires in the vertical view image to generate the vertical view diagram of the device,
the bending parameter is composed of multiple group-specific bending parameters defined for each of the groups, and
the diagram conversion step respectively converts each vertical view diagram of each of the wires included in each group into multiple oblique view diagrams based on each of the group-specific bending parameters, and then performs conversion of the vertical view diagram of each of the wires included in each of the groups into each oblique view diagram based on each adjusted group-specific bending parameter input by the user.

### Claim 9 (depends on 7)

The method according to claim 7, wherein the multiple wires grouped in one group respectively have multiple bending points between the starting point and the ending point,
the group-specific bending parameter is a three-dimensional coordinate position of each of the bending points that the wires grouped in one group have in common, and
the three-dimensional coordinate position of each of the bending points is a combination of a longitudinal direction coordinate position, a lateral direction coordinate position, and a height direction coordinate position in a coordinate system composed of a longitudinal direction axis extending from the starting point to the ending point of the wire in a plane of the reference surface, a lateral direction axis extending in a direction orthogonal to the longitudinal direction axis from the starting point of the wire in the plane of the reference surface, and a height direction axis extending in a vertical direction with respect to the reference surface through the starting point.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the three-dimensional coordinate position of each of the bending points comprises a three-dimensional proportional coordinate position which is a combination of a proportional longitudinal direction coordinate position proportional to a wire total length between the starting point and the ending point, a proportional lateral direction coordinate position proportional to the wire total length, and a proportional height direction coordinate position proportional to the wire total length.

### Claim 11

A three-dimensional diagram generation apparatus for generating a three-dimensional diagram of a three-dimensional object, the three-dimensional diagram generation apparatus comprising:
a controller that generates the three-dimensional diagram of the object based on an image obtained by imaging the object,
wherein the controller
acquires a vertical view image of the object imaged from vertically above and multiple oblique view images of the object imaged from multiple obliquely upper directions,
extracts a contour line in the vertical view image of the object to generate a vertical view diagram of the object,
respectively converts the vertical view diagram into multiple oblique view diagrams based on a shape parameter including height information of the object stored in a storage part, and repeatedly executes adjustment of the shape parameter and conversion of the vertical view diagram into each of the oblique view diagrams until each of the converted oblique view diagrams overlaps with each of the oblique view images, and
synthesizes the generated vertical view diagram and each of the oblique view diagrams overlapping with each of the oblique view images to generate the three-dimensional diagram.

### Claim 12 (depends on 11)

The three-dimensional diagram generation apparatus according to claim 11, wherein the object is a device composed of multiple components and multiple wires connecting between the components,
the controller extracts each contour line of each component and each of the wires in the vertical view image to generate the vertical view diagram of the device, and
the shape parameter is height of each component from a reference surface, inclination of a surface of each component, and a bending parameter of each of the wires.

### Claim 13 (depends on 11)

The three-dimensional diagram generation apparatus according to claim 11, comprising:
a display part that displays images and diagrams; and
an input part for a user to input data,
wherein the controller repeatedly
displays the vertical view image of the object on the display part, and generates a contour line in the vertical view image drawn by the user by operating the input part as the vertical view diagram, and
performs conversion of the vertical view diagram into each of the oblique view diagrams based on the adjusted shape parameter input from the input part by the user, and displays each of the converted oblique view diagrams and each of the oblique view images on the display part by superimposing each of the converted oblique view diagrams and each of the oblique view images.

### Claim 14 (depends on 13)

The three-dimensional diagram generation apparatus according to claim 13, wherein the object is a device composed of multiple components and multiple wires connecting between the components,
the controller generates each contour line of each component and each of the wires in the vertical view image drawn by the user by operating the input part as the vertical view diagram of the device, and
the shape parameter is height of each component from a reference surface, inclination of a surface of each component, and a bending parameter of each of the wires.

### Claim 15 (depends on 12)

The three-dimensional diagram generation apparatus according to claim 12, wherein each of the wires has multiple bending points between a starting point and an ending point,
the bending parameter is a three-dimensional coordinate position of each of the bending points of each of the wires, and
the three-dimensional coordinate position of each of the bending points is a combination of a longitudinal direction coordinate position, a lateral direction coordinate position, and a height direction coordinate position in a coordinate system composed of a longitudinal direction axis extending from the starting point to the ending point of the wire in a plane of the reference surface, a lateral direction axis extending in a direction orthogonal to the longitudinal direction axis from the starting point of the wire in the plane of the reference surface, and a height direction axis extending in a vertical direction with respect to the reference surface through the starting point.

### Claim 16 (depends on 15)

The three-dimensional diagram generation apparatus according to claim 15, wherein the three-dimensional coordinate position of each of the bending points comprises a three-dimensional proportional coordinate position which is a combination of a proportional longitudinal direction coordinate position proportional to a wire total length between the starting point and the ending point, a proportional lateral direction coordinate position proportional to the wire total length, and a proportional height direction coordinate position proportional to the wire total length.

### Claim 17 (depends on 12)

The three-dimensional diagram generation apparatus according to claim 12, wherein the controller groups the multiple wires into multiple groups composed of the wires having the starting points located on the same component, the ending points located on the same component, and the same thickness, and extracts the contour line in the vertical view image to generate the vertical view diagram of the object,
the bending parameter stored in the storage part is composed of multiple group-specific bending parameters defined for each of the groups, and
the controller respectively converts each vertical view diagram of each of the wires included in each group into multiple oblique view diagrams based on each of the group-specific bending parameters stored in the storage part, and repeatedly executes adjustment of each of the group-specific bending parameters of each of the groups and conversion of each vertical view diagram of the wires included in each of the groups into the oblique view diagrams until each of the converted oblique view diagrams of each of the wires included in each of the groups overlaps with each of the oblique view images of each of the wires included in each of the groups.

### Claim 18 (depends on 14)

The three-dimensional diagram generation apparatus according to claim 14, wherein the controller groups the multiple wires into multiple groups composed of the wires having the starting points located on the same component, the ending points located on the same component, and the same thickness, and generates each contour line of each component and each of the wires in the vertical view image drawn by the user by operating the input part as the vertical view diagram of the device,
the bending parameter stored in the storage part is composed of multiple group-specific bending parameters defined for each of the groups, and
the controller repeatedly
respectively converts each vertical view diagram of each of the wires included in each of the groups into multiple oblique view diagrams based on each of the group-specific bending parameters, and
then performs conversion of each vertical view diagram of each of the wires included in each of the groups into each of the oblique view diagrams based on each of the adjusted group-specific bending parameters input from the input part by the user, and displays each of the converted oblique view diagrams and each of the oblique view images on the display part by superimposing each of the converted oblique view diagrams and each of the oblique view images.

### Claim 19 (depends on 17)

The three-dimensional diagram generation apparatus according to claim 17, wherein the multiple wires grouped in one group respectively have multiple bending points between the starting point and the ending point,
the group-specific bending parameter is a three-dimensional coordinate position of each of the bending points that the wires grouped in one group have in common, and
the three-dimensional coordinate position of each of the bending points is a combination of a longitudinal direction coordinate position, a lateral direction coordinate position, and a height direction coordinate position in a coordinate system composed of a longitudinal direction axis extending from the starting point to the ending point of the wire in a plane of the reference surface, a lateral direction axis extending in a direction orthogonal to the longitudinal direction axis from the starting point of the wire in the plane of the reference surface, and a height direction axis extending in a vertical direction with respect to the reference surface through the starting point.

### Claim 20 (depends on 19)

The three-dimensional diagram generation apparatus according to claim 19, wherein the three-dimensional coordinate position of each of the bending points comprises a three-dimensional proportional coordinate position which is a combination of a proportional longitudinal direction coordinate position proportional to a wire total length between the starting point and the ending point, a proportional lateral direction coordinate position proportional to the wire total length, and a proportional height direction coordinate position proportional to the wire total length.

### Claim 21

A three-dimensional shape inspection apparatus for performing shape inspection on a three-dimensional object, the three-dimensional shape inspection apparatus comprising:
a master three-dimensional diagram generation part that generates a master three-dimensional diagram of a standard product of the object; and
an inspection part that compares a three-dimensional diagram of the object with the master three-dimensional diagram to perform inspection on the object,
wherein the master three-dimensional diagram generation part
acquires a standard vertical view image of the standard product imaged from vertically above and multiple standard oblique view images of the standard product imaged from multiple obliquely upper directions,
extracts a contour line in the standard vertical view image of the standard product to generate a standard vertical view diagram of the standard product,
respectively converts the standard vertical view diagram into multiple standard oblique view diagrams based on a shape parameter including height information of the standard product stored in a storage part, and repeatedly executes adjustment of the shape parameter and conversion of the standard vertical view diagram into each of the standard oblique view diagrams until each of the converted standard oblique view diagrams overlaps with each of the standard oblique view images, and
synthesizes the generated standard vertical view diagram and each of the standard oblique view diagrams overlapping with each of the standard oblique view images to generate the master three-dimensional diagram, and
the inspection part
acquires a vertical view image of the object imaged from vertically above and multiple oblique view images of the object imaged from multiple obliquely upper directions,
extracts a contour line from the vertical view image while referring to the master three-dimensional diagram to generate a vertical view diagram of the object,
respectively extracts a contour line from each of the oblique view images while referring to the master three-dimensional diagram to generate multiple oblique view diagrams of the object,
synthesizes the generated vertical view diagram and each of the oblique view diagrams to generate the three-dimensional diagram of the object, and
compares the generated three-dimensional diagram of the object with the master three-dimensional diagram to perform inspection on a three-dimensional shape of the object.

### Claim 22 (depends on 21)

The three-dimensional shape inspection apparatus according to claim 21, wherein the object and the standard product are devices composed of multiple components and multiple wires connecting between the components,
the master three-dimensional diagram generation part extracts each contour line of each component and each of the wires in the standard vertical view image to generate the standard vertical view diagram of the device, and
the shape parameter is height of each component from a reference surface, inclination of a surface of each component, and a bending parameter of each of the wires.

### Claim 23 (depends on 21)

The three-dimensional shape inspection apparatus according to claim 21, comprising:
a display part that displays images and diagrams; and
an input part for a user to input data,
wherein the master three-dimensional diagram generation part repeatedly
displays the standard vertical view image of the standard product on the display part, and generates a contour line in the standard vertical view image drawn by the user by operating the input part as the standard vertical view diagram, and
performs conversion of the standard vertical view diagram into each of the standard oblique view diagrams based on the adjusted shape parameter input from the input part by the user, and displays each of the converted standard oblique view diagrams and each of the standard oblique view images on the display part by superimposing each of the converted standard oblique view diagrams and each of the standard oblique view images.

### Claim 24 (depends on 23)

The three-dimensional shape inspection apparatus according to claim 23, wherein the object and the standard product are devices composed of multiple components and multiple wires connecting between the components,
the master three-dimensional diagram generation part generates each contour line of each component and each of the wires in the standard vertical view image drawn by the user by operating the input part as the standard vertical view diagram of the device, and
the shape parameter is height of each component from a reference surface, inclination of a surface of each component, and a bending parameter of each of the wires.

### Claim 25 (depends on 22)

The three-dimensional shape inspection apparatus according to claim 22, wherein each of the wires has multiple bending points between a starting point and an ending point,
the bending parameter is a three-dimensional coordinate position of each of the bending points of each of the wires, and
the three-dimensional coordinate position of each of the bending points is a combination of a longitudinal direction coordinate position, a lateral direction coordinate position, and a height direction coordinate position in a coordinate system composed of a longitudinal direction axis extending from the starting point to the ending point of the wire in a plane of the reference surface, a lateral direction axis extending in a direction orthogonal to the longitudinal direction axis from the starting point of the wire in the plane of the reference surface, and a height direction axis extending in a vertical direction with respect to the reference surface through the starting point.

### Claim 26 (depends on 25)

The three-dimensional shape inspection apparatus according to claim 25, wherein the three-dimensional coordinate position of each of the bending points comprises a three-dimensional proportional coordinate position which is a combination of a proportional longitudinal direction coordinate position proportional to a wire total length between the starting point and the ending point, a proportional lateral direction coordinate position proportional to the wire total length, and a proportional height direction coordinate position proportional to the wire total length.

### Claim 27 (depends on 22)

The three-dimensional shape inspection apparatus according to claim 22, wherein the master three-dimensional diagram generation part groups the multiple wires into multiple groups composed of the wires having the starting points located on the same component, the ending points located on the same component, and the same thickness, and extracts the contour line in the standard vertical view image of the standard product to generate the standard vertical view diagram of the standard product,
the bending parameter stored in the storage part is composed of multiple group-specific bending parameters defined for each of the groups, and
the master three-dimensional diagram generation part
respectively converts each standard vertical view diagram of each of the wires included in each group into multiple standard oblique view diagrams based on each of the group-specific bending parameters, and
repeatedly executes adjustment of each of the group-specific bending parameters of each of the groups and conversion of each standard vertical view diagram of the wires included in each of the groups into the standard oblique view diagrams until each of the converted standard oblique view diagrams of each of the wires included in each of the groups overlaps with each of the standard oblique view images of each of the wires included in each of the groups.

### Claim 28 (depends on 24)

The three-dimensional shape inspection apparatus according to claim 24, wherein the master three-dimensional diagram generation part groups the multiple wires into multiple groups composed of the wires having the starting points located on the same component, the ending points located on the same component, and the same thickness, and generates each contour line of each component and each of the wires in the standard vertical view image drawn by the user by operating the input part as the vertical view diagram of the standard product,
the bending parameter stored in the storage part is composed of multiple group-specific bending parameters defined for each of the groups, and
the master three-dimensional diagram generation part repeatedly
respectively converts each standard vertical view diagram of each of the wires included in each of the groups into multiple standard oblique view diagrams based on each of the group-specific bending parameters, and
then performs conversion of each standard vertical view diagram of each of the wires included in each of the groups into each of the standard oblique view diagrams based on each of the adjusted group-specific bending parameters input from the input part by the user, and displays each of the converted standard oblique view diagrams and each of the standard oblique view images on the display part by superimposing each of the converted standard oblique view diagrams and each of the standard oblique view images.

### Claim 29 (depends on 27)

The three-dimensional shape inspection apparatus according to claim 27, wherein the multiple wires grouped in one group respectively have multiple bending points between the starting point and the ending point,
the group-specific bending parameter is a three-dimensional coordinate position of each of the bending points that the wires grouped in one group have in common, and
the three-dimensional coordinate position of each of the bending points is a combination of a longitudinal direction coordinate position, a lateral direction coordinate position, and a height direction coordinate position in a coordinate system composed of a longitudinal direction axis extending from the starting point to the ending point of the wire in a plane of the reference surface, a lateral direction axis extending in a direction orthogonal to the longitudinal direction axis from the starting point of the wire in the plane of the reference surface, and a height direction axis extending in a vertical direction with respect to the reference surface through the starting point.

### Claim 30 (depends on 29)

The three-dimensional shape inspection apparatus according to claim 29, wherein the three-dimensional coordinate position of each of the bending points comprises a three-dimensional proportional coordinate position which is a combination of a proportional longitudinal direction coordinate position proportional to a wire total length between the starting point and the ending point, a proportional lateral direction coordinate position proportional to the wire total length, and a proportional height direction coordinate position proportional to the wire total length.