# US 2024/0003675 A1 · Measurement system, measurement device, measurement method, and measurement program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0003675 A1
- Document ID: US20240003675A1
- Status: Published application
- Kind: A1
- Application number: 18/251,322
- Filed: 2021-09-14
- Published: 2024-01-04
- Earliest claimed: 2020-11-12
- Assignees: OMRON Corporation
- Applicants: Omron Corporation
- Inventors: Xingdou Fu
- Web page: https://patentgrep.com/patent/US20240003675A1

## CPC

- G01B 11/005: {coordinate measuring machines}
- B25J 9/1697: {Vision controlled systems}
- G01B 11/24: for measuring contours or curvatures
- G01B 11/25: by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50})
- G06T 7/579: from motion
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 20/64: Three-dimensional [3D] objects
- G05B 2219/37567: 3-D vision, stereo vision, with two cameras
- G05B 2219/40053: Pick 3-D object from pile of objects
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30164: Workpiece; Machine component
- G06V 2201/06: Recognition of objects for industrial automation
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A measurement system and associated techniques can reduce the time for obtaining 3D data about a target object by measurement. The measurement system includes a 3D sensor installable on a robot to obtain, by measurement, 3D data indicating 3D coordinates of surface points on a surface of a target object, a displacement detector that detects a displacement of a joint of the robot, a drive that drives the joint of the robot, a sensor controller that controls the 3D sensor to obtain 3D data about the target object by measurement at one or more measurement points, an integrator that performs 3D integration of 3D data about the target object obtained by measurement before a first operation performed by the robot on the target object and 3D data about the target object obtained by measurement at the one or more measurement points after the first operation performed by the robot on the target object, and a robot controller that outputs, to the drive, a drive command for controlling a second operation performed by the robot based on 3D coordinates of the surface points on the surface of the target object indicated by 3D data obtained through the 3D integration.

## Claims

### Claim 1

A measurement system, comprising:
a three-dimensional sensor installable on a robot to obtain, by measurement, three-dimensional data indicating three-dimensional coordinates of surface points on a surface of a target object;
a displacement detector configured to detect a displacement of a joint of the robot;
a drive configured to drive the joint of the robot;
a sensor controller configured to control the three-dimensional sensor to obtain three-dimensional data about the target object by measurement at one or more measurement points;
an integrator configured to perform three-dimensional integration of three-dimensional data about the target object obtained by measurement before a first operation performed by the robot on the target object and three-dimensional data about the target object obtained by the sensor controller by measurement at the one or more measurement points after the first operation performed by the robot on the target object; and
a robot controller configured to output, to the drive, a drive command for controlling a second operation performed by the robot based on three-dimensional coordinates of the surface points on the surface of the target object indicated by three-dimensional data obtained through the three-dimensional integration.

### Claim 2 (depends on 1)

The measurement system according to claim 1, wherein
the integrator deletes, from the three-dimensional data about the target object obtained by measurement before the first operation performed by the robot on the target object, a portion subjected to the first operation to generate preliminary three-dimensional data and performs three-dimensional integration of the preliminary three-dimensional data and three-dimensional data about the target object obtained by the sensor controller by measurement at the one or more measurement points after the first operation performed by the robot on the target object.

### Claim 3 (depends on 1)

The measurement system according to claim 1, wherein
the three-dimensional sensor obtains, before an initial operation of a plurality of operations performed by the robot on the target object, initial three-dimensional data about the target object by measurement at a specific measurement point while the robot is stopped,
the displacement detector detects, before the initial operation, an initial displacement of the joint of the robot at the specific measurement point while the robot is stopped, and
the measurement system further comprises a transformer configured to calculate, based on the initial three-dimensional data and the initial displacement, transformation from a sensor coordinate system to a robot coordinate system and to transform, from the sensor coordinate system to the robot coordinate system, the three-dimensional coordinates of the surface points on the surface of the target object indicated by the three-dimensional data obtained through the three-dimensional integration.

### Claim 4 (depends on 3)

The measurement system according to claim 3, wherein
the robot controller controls, after the initial three-dimensional data is obtained by measurement and the initial displacement is detected, the robot to continuously perform the plurality of operations without stopping the robot.

### Claim 5 (depends on 1)

The measurement system according to claim 1, wherein
the sensor controller controls the three-dimensional sensor to obtain three-dimensional data about the target object by measurement at the one or more measurement points during driving of the joint of the robot.

### Claim 6 (depends on 5)

The measurement system according to claim 5, wherein
the three-dimensional sensor obtains three-dimensional data about the target object by measurement at the one or more measurement points at timing asynchronous with timing at which the displacement detector detects a displacement of the joint of the robot.

### Claim 7

A measurement device for receiving input information indicating a displacement of a joint of a robot output from a displacement detector and outputting a drive command to a drive configured to drive the joint of the robot, the measurement device comprising:
a three-dimensional sensor installable on the robot to obtain, by measurement, three-dimensional data indicating three-dimensional coordinates of surface points on a surface of a target object;
a sensor controller configured to control the three-dimensional sensor to obtain three-dimensional data about the target object by measurement at one or more measurement points;
an integrator configured to perform three-dimensional integration of three-dimensional data about the target object obtained by measurement before a first operation performed by the robot on the target object and three-dimensional data about the target object obtained by the sensor controller by measurement at the one or more measurement points after the first operation performed by the robot on the target object; and
a robot controller configured to output the drive command for controlling a second operation performed by the robot based on three-dimensional coordinates of the surface points on the surface of the target object indicated by three-dimensional data obtained through the three-dimensional integration.

### Claim 8

A measurement method implementable by a measurement device including a three-dimensional sensor installable on a robot to obtain, by measurement, three-dimensional data indicating three-dimensional coordinates of surface points on a surface of a target object, the measurement device being configured to receive input information indicating a displacement of a joint of the robot output from a displacement detector and to output a drive command to a drive configured to drive the joint of the robot, the measurement method comprising:
controlling the three-dimensional sensor to obtain three-dimensional data about the target object by measurement at one or more measurement points;
performing three-dimensional integration of three-dimensional data about the target object obtained by measurement before a first operation performed by the robot on the target object and three-dimensional data about the target object obtained by measurement at the one or more measurement points after the first operation performed by the robot on the target object; and
outputting the drive command for controlling a second operation performed by the robot based on three-dimensional coordinates of the surface points on the surface of the target object indicated by three-dimensional data obtained through the three-dimensional integration.

### Claim 9 (depends on 8)

A non-transitory computer-readable medium having stored therein computer executable instructions for execution by a computing device, the non-transitory computer-readable medium comprising:
a measurement program executable by a measurement device, the measurement device including a three-dimensional sensor installable on a robot to obtain, by measurement, three-dimensional data indicating three-dimensional coordinates of surface points on a surface of a target object;
wherein the measurement device is configured to receive input information indicating a displacement of a joint of the robot output from a displacement detector and to output a drive command to a drive configured to drive the joint of the robot; and
wherein the measurement program is configured to cause the measurement device to perform the method of claim 8.
