# US 10,455,139 B2 · Image processing device and image processing method for calculating distance information to a subject

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,455,139 B2
- Document ID: US10455139B2
- Status: Granted
- Kind: B2
- Application number: 15/667,513
- Filed: 2017-08-02
- Published: 2017-11-16
- Granted: 2019-10-22
- Earliest claimed: 2016-02-29
- Assignees: Panasonic Corporation
- Applicants: Panasonic Corporation
- Inventors: Khang Nguyen; Takashi Kawamura; Shinsaku Hiura
- Web page: https://patentgrep.com/patent/US10455139B2

## CPC

- H04N 23/959: by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics
- G01C 3/08: Use of electric radiation detectors
- G01C 3/32: by focusing the object, e.g. on a ground glass screen
- G02B 7/365: {by analysis of the spatial frequency components of the image}
- G02B 7/38: measured at different points on the optical axis {, e.g. focussing on two or more planes and comparing image data}
- G06T 7/571: from focus
- H04N 19/62: by frequency transforming in three dimensions [3D] (H04N19/63 takes precedence)
- H04N 23/80: Camera processing pipelines; Components thereof
- H04N 1/3216: {using Fourier transforms}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Image processing device is image processing device that uses a plurality of images respectively having focusing positions different from each other to calculate distance information to a subject, and includes frequency converter, amplitude extractor, and distance information calculator. Frequency converter converts the plurality of images into frequency. Amplitude extractor extracts an amplitude component out of a phase component and the amplitude component of a coefficient obtained by converting the plurality of images into frequency. Distance information calculator calculates the distance information, by using lens blur data and only the amplitude component extracted by amplitude extractor out of the phase component and the amplitude component of the coefficient.

## Claims

### Claim 1

An image processing device that uses a plurality of images respectively having focusing positions different from each other to calculate distance information to a subject, the image processing device comprising:
a frequency converter that converts the plurality of images into frequency;
an amplitude extractor that extracts an amplitude component out of a phase component and the amplitude component of a coefficient obtained by converting the plurality of images into frequency; and
a distance information calculator that calculates the distance information, by using lens blur data and only the amplitude component extracted by the amplitude extractor out of the phase component and the amplitude component of the coefficient, and outputs the calculated distance information for controlling a lens system of a camera in capturing of images.,
wherein the distance information calculator includes:
a cost calculator that calculates a cost of a distance assumed for the plurality of images, based on a depth calculation formula, by using the lens blur data and the amplitude component extracted by the amplitude extractor, an image deviation being applied to one of the plurality of images and introduced into the depth calculation formula; and
a distance determiner that determines the distance information, by using the cost of the distance calculated by the cost calculator.

### Claim 2 (depends on 1)

The image processing device according to claim 1, further comprising:
an area divider that divides one image of the plurality of images into a plurality of first small areas; and
an area searcher that searches for a second small area corresponding to each of the plurality of first small areas, for another image of the plurality of images,
wherein the frequency converter converts an image of each of the plurality of first small areas, and an image of the second small area corresponding to each of the plurality of first small areas into frequency,
the amplitude extractor extracts the amplitude component, for each of the plurality of first small areas, and the second small area corresponding to each of the plurality of first small areas, and
the distance information calculator calculates the distance information, for each of the plurality of first small areas.

### Claim 3 (depends on 2)

The image processing device according to claim 2, wherein
the cost calculator calculates a representative cost of a distance assumed for each of the plurality of first small areas, and
the distance determiner determines representative distance information for each of the plurality of first small areas, by using the representative cost of the distance calculated by the cost calculator.

### Claim 4 (depends on 1)

The image processing device according to claim 1, wherein
the distance information calculator calculates the distance information for each pixel of the plurality of images.

### Claim 5

An image processing method that uses a plurality of images respectively having focusing positions different from each other to calculate distance information to a subject,
the image processing method comprising:
converting the plurality of images into frequency;
extracting an amplitude component out of a phase component and the amplitude component of a coefficient obtained by converting the plurality of images into frequency; and
calculating the distance information, by using lens blur data and only the amplitude component extracted out of the phase component and the amplitude component of the coefficient,
wherein the calculating of the distance information includes:
calculating a cost of a distance assumed for the plurality of images, based on a depth calculation formula, by using the lens blur data and the amplitude component extracted, an image deviation being applied to one of the plurality of images and introduced into the depth calculation formula; and
determining the distance information, by using the cost of the distance calculated.

### Claim 6

An image processing device that uses a plurality of images respectively having focusing positions different from each other to calculate distance information to a subject, the image processing device comprising:
a frequency converter that converts the plurality of images into frequency;
an amplitude extractor that extracts an amplitude component out of a phase component and the amplitude component of a coefficient obtained by converting the plurality of images into frequency;
a distance information calculator that calculates the distance information, by using lens blur data and only the amplitude component extracted by the amplitude extractor out of the phase component and the amplitude component of the coefficient, and outputs the calculated distance information for controlling a lens system of a camera in capturing of images;
an area divider that divides one image of the plurality of images into a plurality of first small areas; and
an area searcher that searches for a second small area corresponding to each of the plurality of first small areas, for another image of the plurality of images,
wherein the frequency converter converts an image of each of the plurality of first small areas, and an image of the second small area corresponding to each of the plurality of first small areas into frequency,
the amplitude extractor extracts the amplitude component, for each of the plurality of first small areas, and the second small area corresponding to each of the plurality of first small areas, and
the distance information calculator calculates the distance information, for each of the plurality of first small areas,
wherein the distance information calculator includes:
a cost calculator that calculates a cost of a distance assumed for the plurality of images, based on a depth calculation formula, by using the lens blur data and the amplitude component extracted by the amplitude extractor, an image deviation being applied to one of the plurality of images and introduced into the depth calculation formula; and
a distance determiner that determines the distance information, by using the cost of the distance calculated by the cost calculator.