# US 2013/0258060 A1 · Information processing apparatus that performs three-dimensional shape measurement, information processing method, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0258060 A1
- Document ID: US20130258060A1
- Status: Published application
- Kind: A1
- Application number: 13/854,413
- Filed: 2013-04-01
- Published: 2013-10-03
- Earliest claimed: 2012-04-03
- Assignees: Canon Kabushiki Kaisha
- Applicants: Canon Kabushiki Kaisha
- Inventors: Daisuke Kotake
- Web page: https://patentgrep.com/patent/US20130258060A1

## CPC

- G01C 11/025: {by scanning the object}
- G01B 11/25: by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50})
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS

## Abstract

Information processing apparatus that performs three-dimensional shape measurement with high accuracy at high speed while taking into account lens distortion of a projection device. An image input unit of an information processing apparatus inputs image data of a measurement object photographed by a camera in a state where a predetermined pattern light is projected by a projector. An association unit calculates associations between coordinates on the image data of the measurement object and coordinates on image data of the predetermined pattern light. A three-dimensional coordinate calculation unit calculates a viewing vector of the camera from which lens distortion thereof has been eliminated and a viewing vector of the projector to which lens distortion thereof has been added. The calculation unit calculates coordinates of a point, in a three-dimensional space, of intersection between the camera viewing vector and the projector viewing vector, for each association.

## Claims

### Claim 1

An information processing apparatus comprising:
an input unit configured to input image data of a target object, onto which a predetermined pattern image has been projected by a projection apparatus and which has then been photographed by an imaging apparatus;
an association unit configured to provide associations between coordinates on the image data of the target object with coordinates on image data of the predetermined pattern image;
a first viewing vector-deriving unit configured to derive a viewing vector on the imaging apparatus-side which has been corrected with respect to a lens distortion of the imaging apparatus;
a second viewing vector-deriving unit configured to derive a viewing vector on the projection apparatus-side which has not been corrected with respect to a lens distortion of the projection apparatus; and
an intersection coordinate-deriving unit configured to derive coordinates of a point, in a three-dimensional space, of intersection between the viewing vector on the imaging apparatus-side which has been corrected with respect to the lens distortion of the imaging apparatus and the viewing vector on the projection apparatus-side which has not been corrected with respect to the lens distortion of the projection apparatus, for each of the associations provided by said association unit.

### Claim 2 (depends on 1)

The information processing apparatus according to claim 1, wherein said second viewing vector-deriving unit derives the viewing vector on the projection apparatus-side which has not been corrected with respect to the lens distortion of the projection apparatus, such that the viewing vector on the imaging apparatus-side and the viewing vector on the projection apparatus-side intersect in the three-dimensional space, and coordinates of a point of intersection between the viewing vector on the projection apparatus-side which has been corrected with respect to the lens distortion of the projection apparatus and an image plane of the projection apparatus coincide with the coordinates on the image data of the predetermined pattern image associated by said association unit.

### Claim 3 (depends on 1)

The information processing apparatus according to claim 1, wherein the predetermined pattern image is a striped pattern image.

### Claim 4 (depends on 3)

The information processing apparatus according to claim 3, wherein a direction of stripes of the striped pattern image is a horizontal direction or a vertical direction on the image data of the striped pattern image.

### Claim 5 (depends on 1)

The information processing apparatus according to claim 1, wherein the predetermined pattern image is a line pattern image.

### Claim 6 (depends on 5)

The information processing apparatus according to claim 5, wherein a direction of line of the line pattern image is a horizontal direction or a vertical direction on the image data of the line pattern image.

### Claim 7 (depends on 1)

The information processing apparatus according to claim 1, wherein the image data of the predetermined pattern consists of multiple patterns for a space coding method or a phase shift method.

### Claim 8

A information processing method executed by an information processing apparatus, comprising:
inputting image data of a target object, onto which a predetermined pattern image has been projected by a projection apparatus and which has then been photographed by an imaging apparatus;
providing associations between coordinates on the image data of the target object with coordinates on image data of the predetermined pattern image;
deriving a viewing vector on the imaging apparatus-side which has been corrected with respect to a lens distortion of the imaging apparatus;
deriving a viewing vector on the projection apparatus-side which has not been corrected with respect to a lens distortion of the projection apparatus; and
deriving coordinates of a point, in a three-dimensional space, of intersection between the viewing vector on the imaging apparatus-side which has been corrected with respect to the lens distortion of the imaging apparatus and the viewing vector on the projection apparatus-side which has not been corrected with respect to the lens distortion of the projection apparatus, for each of the provided associations.

### Claim 9

A non-transitory storage medium storing a program for causing a computer to function as each unit of an information processing apparatus, said program comprising:
inputting image data of a target object, onto which a predetermined pattern image has been projected by a projection apparatus and which has then been photographed by an imaging apparatus;
providing associations between coordinates on the image data of the target object with coordinates on image data of the predetermined pattern image;
deriving a viewing vector on the imaging apparatus-side which has been corrected with respect to a lens distortion of the imaging apparatus;
deriving a viewing vector on the projection apparatus-side which has not been corrected with respect to a lens distortion of the projection apparatus; and
deriving coordinates of a point, in a three-dimensional space, of intersection between the viewing vector on the imaging apparatus-side which has been corrected with respect to the lens distortion of the imaging apparatus and the viewing vector on the projection apparatus-side which has not been corrected with respect to the lens distortion of the projection apparatus, for each of the provided associations.

### Claim 10

An information processing apparatus comprising:
a projection unit configured to project a pattern image which varies in luminance in a predetermined direction onto a target object;
an imaging unit configured to photograph an image of the target object onto which the pattern image has been projected; and
a deriving unit configured to derive a distance from said projection unit or said imaging unit to the target object, by associating coordinates on the photographed image and coordinates of the pattern image in said projection unit,
wherein said deriving unit includes:
a first correction unit configured to correct the coordinates on the photographed image based on a distortion parameter of said imaging unit, which is determined in advance, and
a second correction unit configured to correct at least one coordinate component on the pattern image, based on the distortion parameter of said projection unit, which is determined in advance, the corrected coordinates on the photographed image, and constraint conditions determined from an arrangement of said projection unit and said imaging unit.

### Claim 11

An information processing method of an information processing apparatus including a projection unit configured to project a pattern image which varies in luminance in a predetermined direction onto a target object, and an imaging unit configured to photograph an image of the target object onto which the pattern image has been projected,
the information processing method comprising:
deriving a distance from the projection unit or the imaging unit to the target object, by associating coordinates on the photographed image and coordinates of the pattern image in the projection unit;
correcting the coordinates on the photographed image based on a distortion parameter of the imaging unit, which is determined in advance; and
correcting at least one coordinate component on the pattern image, based on the distortion parameter of the projection unit, which is determined in advance, the corrected coordinates on the photographed image, and constraint conditions determined from an arrangement of the projection unit and the imaging unit.

### Claim 12

A non-transitory storage medium storing a program for causing a computer to function as each unit of an information processing apparatus including a projection unit configured to project a pattern image which varies in luminance in a predetermined direction onto a target object, and an imaging unit configured to photograph an image of the target object onto which the pattern image has been projected, said program comprising:
deriving a distance from the projection unit or the imaging unit to the target object, by associating coordinates on the photographed image and coordinates of the pattern image in the projection unit;
correcting the coordinates on the photographed image based on a distortion parameter of the imaging unit, which is determined in advance; and
correcting at least one coordinate component on the pattern image, based on the distortion parameter of the projection unit, which is determined in advance, the corrected coordinates on the photographed image, and constraint conditions determined from an arrangement of the projection unit and the imaging unit.
