# US 11,310,411 B2 · Distance measuring device and method of controlling distance measuring device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,310,411 B2
- Document ID: US11310411B2
- Status: Granted
- Kind: B2
- Application number: 16/802,282
- Filed: 2020-02-26
- Published: 2020-06-18
- Granted: 2022-04-19
- Earliest claimed: 2016-08-30
- Assignees: Sony Semiconductor Solutions Corporation
- Applicants: Sony Semiconductor Solutions Corporation
- Inventors: Mitsuharu Ohki
- Web page: https://patentgrep.com/patent/US11310411B2

## CPC

- H04N 23/672: {based on the phase difference signals}
- G01C 3/06: Use of electric means to obtain final indication
- G01S 7/4914: of detector arrays, e.g. charge-transfer gates
- G01S 7/493: Extracting wanted echo signals
- G01S 17/36: with phase comparison between the received signal and the contemporaneously transmitted signal
- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- G01S 17/93: for anti-collision purposes
- G06T 5/70: Denoising; Smoothing
- G06T 7/55: from multiple images
- G06T 7/557: from light fields, e.g. from plenoptic cameras
- H04N 23/20: for generating image signals from infrared radiation only
- H04N 25/771: comprising storage means other than floating diffusion
- G06T 2207/10048: Infrared image
- G06T 2207/20028: Bilateral filtering
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Image quality of a depth map is improved in a distance measuring device for receiving intermittent light. In a pixel array unit, a plurality of pixels for receiving predetermined intermittent light is arranged in a two-dimensional lattice pattern. A continuous light image data generating unit generates continuous light image data in which a plurality of pieces of pixel data indicating a luminance of predetermined continuous light is arranged in a two-dimensional lattice pattern on the basis of received light data of each of the plurality of pixels. A depth map generating unit generates a depth map in which distance information corresponding to each of the plurality of pixels is arranged, from the received light data and the continuous light image data.

## Claims

### Claim 1

A distance measuring device comprising:
a pixel array including a plurality of pixels which receives predetermined intermittent light and is arranged in a two-dimensional lattice pattern, wherein the plurality of pixels is configured to output a first signal according to a charge accumulated in a first period, a second signal according to a charge accumulated in a second period, a third signal according to a charge accumulated in a third period, and a fourth signal according to a charge accumulated in a fourth period; and
processing circuitry configured to generate a depth map according to a distance d and a luminance ir,
wherein the distance d is obtained by the following formula:
d=(c/4πf)×tan−1{(q3−q4)/(q1−q2)}
wherein the luminance ir is obtained by the following formula:
ir=(q1+q2+q3+q4)/2
wherein q1 is the first signal, q2 is the second signal, q3 is the third signal, q4 is the fourth signal, c is the speed of light, and f is frequency and wherein the processing circuitry is configured to generate a high resolution depth map using an upsampling filter to process the distance d of a low resolution depth map and the luminance ir of low resolution infrared image data.

### Claim 2

A distance measuring method performed by processing circuitry, the method comprising:
receiving, from a pixel array including a plurality of pixels which receives predetermined intermittent light and is arranged in a two-dimensional lattice pattern, a first signal according to a charge accumulated in a first period, a second signal according to a charge accumulated in a second period, a third signal according to a charge accumulated in a third period, and a fourth signal according to a charge accumulated in a fourth period; and
generating a depth map according to a distance d and a luminance ir,
wherein the distance d is obtained by the following formula:
d=(c/4πf)×tan−1{(q3−q4)/(q1−q2)}
wherein the luminance ir is obtained by the following formula:
ir=(q1+q2+q3+q4)/2
and wherein q1 is the first signal, q2 is the second signal, q3 is the third signal, q4 is the fourth signal, c is the speed of light, and f is frequency, including generating a high resolution depth map using an upsampling filter to process the distance d of a low resolution depth map and the luminance ir of low resolution infrared image data.

### Claim 3

A non-transitory computer readable medium containing instructions that, when executed by processing circuitry, perform a distance measuring method comprising:
receiving, from a pixel array including a plurality of pixels which receives predetermined intermittent light and is arranged in a two-dimensional lattice pattern, a first signal according to a charge accumulated in a first period, a second signal according to a charge accumulated in a second period, a third signal according to a charge accumulated in a third period, and a fourth signal according to a charge accumulated in a fourth period; and
generating a depth map according to a distance d and a luminance ir, wherein the distance d is obtained by the following formula:
d=(c/4πf)×tan−1{(q3−q4)/(q1−q2)}
wherein the luminance ir is obtained by the following formula:
ir=(q1+q2+q3+q4)/2
and wherein q1 is the first signal, q2 is the second signal, q3 is the third signal, q4 is the fourth signal, c is the speed of light, and f is frequency, including generating a high resolution depth map using an upsampling filter to process the distance d of a low resolution depth map and the luminance ir of low resolution infrared image data.