US 2017/0365068 A1 · COMBINING LIGHT-FIELD DATA WITH ACTIVE DEPTH DATA FOR DEPTH MAP GENERATION

Abstract

Depths of one or more objects in a scene may be measured with enhanced accuracy through the use of a light-field camera and a depth sensor. The light-field camera may capture a light-field image of the scene. The depth sensor may capture depth sensor data of the scene. Light-field depth data may be extracted from the light-field image and used, in combination with the sensor depth data, to generate a depth map indicative of distance between the light-field camera and one or more objects in the scene. The depth sensor may be an active depth sensor that transmits electromagnetic energy toward the scene; the electromagnetic energy may be reflected off of the scene and detected by the active depth sensor. The active depth sensor may have a 360° field of view; accordingly, one or more mirrors may be used to direct the electromagnetic energy between the active depth sensor and the scene.

Claims28

Dependent claims

A method for measuring depths of at least one object in a scene, the method comprising:

with a light-field camera, capturing a light-field image of the scene;
with a depth sensor, capturing depth sensor data of the scene;
extracting light-field depth data from the captured light-field image;
combining the extracted light-field depth data with the captured depth sensor data to generate a depth map indicative of distance between the light-field camera and each object in the scene;
and
outputting the generated depth map.

3 more elements

A non-transitory computer-readable medium for measuring depths of at least one object in a scene, comprising instructions stored thereon, that when executed by a processor, perform the steps of:

receiving a light-field image of a scene, wherein the light-field image has been captured with a light-field camera;
receiving depth sensor data of the scene, wherein the depth sensor data has been captured with a depth sensor;
extracting light-field depth data from the captured light-field image;
combining the extracted light-field depth data with the captured depth sensor data to generate a depth map indicative of distance between the light-field camera and each object in the scene;
and
outputting the generated depth map.

3 more elements

A system for measuring depths of at least one object in a scene, the system comprising:

a light-field camera configured to capture a light-field image of the scene;
a depth sensor configured to capture depth sensor data of the scene;
and
a processor, communicatively coupled to the light-field camera and the depth sensor, configured to:
extract light-field depth data from the captured light-field image;
and
combine the extracted light-field depth data with the captured depth sensor data to generate a depth map indicative of distance between the light-field camera and each object in the scene;
and
an output device, communicatively coupled to the processor, configured to output the generated depth map.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application

15/635,894Filed 28 Jun 2017

Priority

15 Apr 2015

Published

21 Dec 2017

Term ends

Not estimatedThe source records don’t support an estimate.

Classification11 codes

H04N 13/232
Using a single two-dimensional [2D] image sensor: using fly-eye lenses, e.g. arrangements of circular lenses
G01S 17/08
Systems determining position data of a target: for measuring distance only
G01S 17/42
Systems determining position data of a target: simultaneous measurement of distance and other co-ordinates
G01S 17/86
Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems: combinations of lidar systems…

Family

Application62/148,055Us provisional application · filed 15 Apr 2015

Application62/200,804Us provisional application · filed 4 Aug 2015

Application14/834,924Continuation in part · filed 25 Aug 2015

Application62/305,917Us provisional application · filed 9 Mar 2016

Application15/098,674Continuation in part · filed 14 Apr 2016

Application62/359,022Us provisional application · filed 6 Jul 2016

Application15/451,831Continuation in part · filed 7 Mar 2017

Application15/635,894Continuation in part

US 10,565,734 B2Related patent

US 11,328,446 B2Related patent

US 2016/0309065 A1Published application

US 2017/0059305 A1Published application

US 2017/0243373 A1Published application

US 2017/0365068 A1This patent

From the US filing history. Foreign family members aren’t listed.

Combining light-field data with active depth data for depth map generation

Life of this patent

Combining light-field data with active depth data for depth map generation

Life of this patent

Abstract

Claims28