# US 2004/0080758 A1 · Three-dimensional visual sensor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2004/0080758 A1
- Document ID: US20040080758A1
- Status: Published application
- Kind: A1
- Application number: 10/689,721
- Filed: 2003-10-22
- Published: 2004-04-29
- Earliest claimed: 2002-10-23
- Assignees: Fanuc Corporation
- Inventors: Kazunori Ban; Ichiro Kanno
- Web page: https://patentgrep.com/patent/US20040080758A1

## CPC

- G01B 11/25: by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50})
- G06T 7/74: {involving reference images or patches}
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Slit light is projected onto the surface of an object, the reflected light is received using a video camera and analyzed by means of an image processor to thereby determine an equation of the surface onto which the slit light is projected. Then, an image is captured through normal image taking using the video camera and a straight line (line of sight) passing through the measuring point on the object and the center of the video camera is determined. Furthermore, parameters such as an affine transformation are determined by a comparison between the image of a characteristic area F of the object and a reference image. Then, the three-dimensional position and posture of the measuring point are determined from an intersection between the line of sight and the determined surface.

## Claims

### Claim 1

A three-dimensional sensor comprising:
pattern light projecting means for projecting slit light or spot light onto the surface of an object;
a camera that captures a two-dimensional image of the object;
means for determining a straight line which passes through a measuring point on the object and a specific point on said camera from the image of the object captured by said camera;
means for causing said camera to capture the slit light or spot light projected by said pattern light projecting means onto the surface of the object and determining the surface of the object on which said measuring point exists;
means for determining the three-dimensional position of said measuring point from the straight line determined by said means for determining a straight line and the surface determined by said means for determining a surface; and
means for calculating an amount of rotation of the object around said measuring point on the plane including said surface of the object by comparing the shape of the image of the entire object or part of the object captured by said camera with a prepared reference shape of the entire object or part of the object.

### Claim 2

A three-dimensional visual sensor which performs three-dimensional measurement of an object, comprising:
two-dimensional information acquiring means;
three-dimensional information acquiring means; and
information combining means,
wherein said two-dimensional information acquiring means determines the position of a measuring point of said object on a two-dimensional image including said object captured by a camera, compares a reference image including a characteristic area of the object with the image of said characteristic area in said two-dimensional image and determines parameter values that describe a transformation expressing geometrical deformation with respect to said reference image provided by mapping using said camera,
said three-dimensional information acquiring means receives the reflected light of the light projected by the projecting means onto said object by means of the light receiving means to acquire three-dimensional information on the inclination of the surface on which said measuring point of said object exists and/or distance from said camera to the surface, and
said information combining means combines the information acquired by said two-dimensional information acquiring means and the information acquired by said three-dimensional information acquiring means based on the calibration information of said camera and generates new three-dimensional information.

### Claim 3 (depends on 2)

The three-dimensional visual sensor according to claim 2, wherein said light is received at the same camera position as the camera position at which said two-dimensional image is captured.

### Claim 4 (depends on 2)

The three-dimensional visual sensor according to claim 2, wherein said camera also serves as said light receiving means.

### Claim 5 (depends on 4)

The three-dimensional visual sensor according to claim 4, wherein said camera is mounted in a robot and captures said two-dimensional information and said three-dimensional information at the same robot position, and further comprises transforming means for transforming the information acquired from said information combining means into information expressed on the coordinate system of said robot.

### Claim 6 (depends on 5)

The three-dimensional visual sensor according to claim 5, wherein said transforming means acquires position information of said robot from said robot and transforms the result obtained by said information combining means into one expressed on the coordinate system on said robot.

### Claim 7 (depends on 5)

The three-dimensional visual sensor according to claim 5, wherein said transforming means is provided on said robot and the result obtained by said information combining means is transferred to said robot.

### Claim 8 (depends on 2)

The three-dimensional visual sensor according to claim 2, wherein said information combining means comprises:
means for determining, in the three-dimensional space, a straight line which passes through the measuring point on said object and a specific point on said camera; and
means for determining, based on information on said straight line and the surface on which the measuring point on said object exists, an intersection between said surface and said straight line.

### Claim 9

A three-dimensional visual sensor which performs three-dimensional measurement of an object, comprising:
two-dimensional information acquiring means;
three-dimensional information acquiring means; and
information combining means,
wherein said two-dimensional information acquiring means determines the position of a measuring point of said object on a two-dimensional image including said object captured by a camera, compares a reference image including a characteristic area of said object with the image of said characteristic area in said two-dimensional image and determines parameter values that describe a transformation expressing geometrical deformation with respect to said reference image provided by mapping using said camera,
said three-dimensional information acquiring means receives the reflected light of the light projected by the projecting means onto said object by means of the light receiving means to acquire three-dimensional information on the inclination of the first surface which has a certain positional relationship with said measuring point on said object and/or distance from said camera to the surface, and
said information combining means combines the information acquired by said two-dimensional information acquiring means and the information acquired by said three-dimensional information acquiring means based on the calibration information of said camera and generates new three-dimensional information and further comprises:
means for determining a straight line in a three-dimensional space which passes through the measuring point on said object and a specific point on said camera;
means for determining, from the information on said first surface, information on a virtual second surface which has a certain positional relationship with said first surface and passes through the measuring point on said object; and
means for determining an intersection between said straight line and said second surface.

### Claim 10 (depends on 9)

The three-dimensional visual sensor according to claim 9, wherein said light is received at the same camera position as the camera position at which said two-dimensional image is captured.

### Claim 11 (depends on 9)

The three-dimensional visual sensor according to claim 9, wherein said camera also serves as said light receiving means.

### Claim 12 (depends on 10)

The three-dimensional visual sensor according to claim 10, wherein said camera is mounted on a robot and captures said two-dimensional information and said three-dimensional information at the same robot position and further comprises transforming means for transforming the information acquired from said information combining means into information expressed on the coordinate system of said robot.

### Claim 13 (depends on 12)

The three-dimensional visual sensor according to claim 12, wherein said transforming means acquires position information of said robot from said robot and transforms the result obtained by said information combining means into one expressed on the coordinate system on said robot.

### Claim 14 (depends on 12)

The three-dimensional visual sensor according to claim 12, wherein said transforming means is provided on said robot and the result obtained by said information combining means is transferred to said robot.

### Claim 15 (depends on 2, 9)

The three-dimensional visual sensor according to claim 2 or 9, wherein said transformation is an affine transformation.

### Claim 16 (depends on 2, 9)

The three-dimensional visual sensor according to claim 2 or 9, wherein said transformation is a perspective transformation.
