US 10,122,994 B2 · Object reconstruction from dense light fields via depth from gradients

Abstract

The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.

Claims20

Dependent claims

A method for object reconstruction, the method comprising:

receiving a light field represented by a plurality of images, wherein each image is at a different viewpoint, and wherein the plurality of images are associated with an object;
determining a depth for a color edge of the object in an image of the plurality of images;
propagating the depth for the color edge to one or more points around the color edge based on a type of the color edge, wherein a direction of propagation depends on whether the type of color edge is texture or silhouette;
generating a depth map for the image using the depth for the color edge and the depth for the one or more points;
and
generating a mesh for the object based on the depth map.

3 more elements

A non-transitory computer-readable storage medium storing a plurality of instructions executable by one or more processors, the plurality of instructions when executed by the one or more processors cause the one or more processors to:

receive a light field represented by a plurality of images, wherein each image is at a different viewpoint, and wherein the plurality of images are associated with an object;
determine a depth for a color edge of the object in an image of the plurality of images;
propagate the depth for the color edge to one or more points around the color edge based on a type of the color edge, wherein a direction of propagation depends on whether the type of color edge is texture or silhouette;
generate a depth map for the image using the depth for the color edge and the depth for the one or more points;
and
generate a mesh for the object based on the depth map.

3 more elements

A system comprising:

one or more processors;
and
a non-transitory computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to:
receive a light field represented by a plurality of images, wherein each image is at a different viewpoint, and wherein the plurality of images are associated with an object;
determine a depth for a color edge of the object in an image of the plurality of images;
propagate the depth for the color edge to one or more points around the color edge based on a type of the color edge, wherein a direction of propagation depends on whether the type of color edge is texture or silhouette;
generate a depth map for the image using the depth for the color edge and the depth for the one or more points;
and
generate a mesh for the object based on the depth map.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application

15/799,831Filed 31 Oct 2017 · continuation

Priority

11 Nov 2016

Published

US 2018/0137674 A117 May 2018

Granted

6 Nov 2018

Term ends

Not estimatedThe source records don’t support an estimate.

Classification15 codes

H04N 13/271
Image signal generators: wherein the generated image signals comprise depth maps or disparity maps
G06T 7/557
From multiple images: from light fields, e.g. from plenoptic cameras
G06T 7/564
From multiple images: from contours
G06T 7/593
From multiple images: from stereo images

Family

Application62/421,178Us provisional application · filed 11 Nov 2016

Application15/799,663Parent of this continuation · filed 31 Oct 2017

US 2018/0137674 A1Pre-grant publication · published 17 May 2018

US 10,887,581 B2Related patent

US 2018/0139436 A1Published application

US 10,122,994 B2This patent

From the US filing history. Foreign family members aren’t listed.

Object reconstruction from dense light fields via depth from gradients

Life of this patent

Object reconstruction from dense light fields via depth from gradients

Life of this patent

Abstract

Claims20