US 10,097,777 B2 · Depth map from multi-focal plane images

Abstract

A system for generating a depth map for an object in a three-dimensional (3D) scene includes an image capture sensor and a processor. The image capture sensor is configured to capture a plurality of images of the object at a plurality of different focal planes. The processor is configured to calculate a plurality of variance values for a plurality of image locations for each image captured by the image capture sensor. The processor is also configured to determine a peak variance value for the plurality of image locations based on the calculated variance values associated with the same image location for each of the plurality of images of the object at the plurality of different focal planes. The processor is also configured to generate the depth map for the object based on the determined peak variance value for each image location and the plurality of different focal planes.

Claims20

Dependent claims

A system for generating a depth map for an object in a three-dimensional (3D) scene, comprising:

an image capture sensor configured to capture a plurality of images of the object at a plurality of different focal planes;
a liquid lens associated with the image capture sensor for changing the focal plane for each image captured by the image capture sensor;
and
a processor configured to:
calculate a plurality of variance values for a plurality of image locations for each image captured by the image capture sensor;
determine a peak variance value for the plurality of image locations based on the calculated variance values associated with the same image location for each of the plurality of images of the object at the plurality of different focal planes;
generate the depth map for the object based on the determined peak variance value for each image location and the plurality of different focal planes;
and
determine the focal plane for a respective image of the plurality of images of the object based on a voltage being applied to the liquid lens and associate the determined focal plane for the respective image with the calculated variance values for the respective image.

6 more elements

A method for generating a depth map for an object in a three-dimensional (3D) scene, comprising:

capturing, with an image capture sensor, a plurality of images of the object at a plurality of different focal planes, each image of the plurality of images comprising a matrix of pixels;
calculating a plurality of variance values for a plurality of image locations for each image of the plurality of images of the object at the plurality of different focal planes;
determining a peak variance value for each image location of the plurality of image locations based on the calculated variance values associated with the same image location in each image for the plurality of images of the object at the different focal planes;
determining a depth value for each image location of the plurality of image locations based on the determined peak variance value for each image location of the plurality of image locations;
generating the depth map for the object based on the determined depth value for each image location of the plurality of image locations;
changing the focal plane of a liquid lens associated with the image capture sensor for each image captured by the image capture sensor;
and
determining the focal plane for a respective image of the plurality of images of the object based on a voltage being applied to the liquid lens.

5 more elements

Dependencies are inferred from the claim text.

Depth map from multi-focal plane images

Life of this patent

Depth map from multi-focal plane images

Life of this patent

Abstract

Claims20