US 9,542,750 B2Granted

Method, apparatus and computer program product for depth estimation of stereo images

AssigneeNokia Technologies Oy

InventorsVeldandi Muninder, Soumik Ukil, Krishna Annasagar Govindarao

Life of this patent

Estimating the term…

Loading patent…

Abstract

In an example embodiment, a method, apparatus and computer program product are provided. The method includes facilitating receipt of primary depth map and plurality of color segments, the primary depth map associated with a first image and a second image that are stereoscopic pair of images of a scene, and the color segments associated with the first image. The method includes generating plurality of disparity planes based on the color segments and the primary depth map. The method includes determining aggregated cost volume between pixels of the first and second images for the disparity planes. The method includes assigning plane labels corresponding to the disparity planes to pixels of the first and second images based on the aggregated cost volume, an individual pixel being assigned a plane label. The method includes generating secondary depth map based on the plane labels assigned to the pixels of the first and second images.

Claims21

Dependent claims

A method comprising:

facilitating receipt of a primary depth map and a plurality of color segments, the primary depth map being associated with a first image and a second image of a scene, the first image and the second image being a stereoscopic pair of images of the scene, and the plurality of color segments being associated with the first image;
generating, by a processor, a plurality of disparity planes for the scene based on the plurality of color segments and the primary depth map, the plurality of disparity planes associated with a plurality of plane labels;
determining, by the processor, an aggregated cost volume between pixels of the first image and pixels of the second image for the plurality of disparity planes;
assigning, by the processor, plane labels to pixels of at least one image of the first image and the second image based on the aggregated cost volume, wherein an individual pixel of the pixels is assigned a plane label of the plane labels;
generating, by the processor, a secondary depth map based on the plane labels assigned to the pixels of the at least one image, wherein generating the secondary depth map comprises refining the secondary depth map by assigning plane labels to one or more occlusion pixels based on plane labels of neighboring pixels of the one or more occlusion pixels;
and
determining, by the processor, a dominant disparity plane from the secondary depth map.

4 more elements

An apparatus comprising:

at least one processor;
and
at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least perform:
facilitate receipt of a primary depth map and a plurality of color segments, the primary depth map being associated with a first image and a second image of a scene, the first image and the second image being a stereoscopic pair of images of the scene, and the plurality of color segments being associated with the first image;
generate a plurality of disparity planes for the scene based on the plurality of color segments and the primary depth map, the plurality of disparity planes associated with a plurality of plane labels;
determine an aggregated cost volume between pixels of the first image and pixels of the second image for the plurality of disparity planes;
assign plane labels to pixels of at least one image of the first image and the second image based on the aggregated cost volume, wherein an individual pixel of the pixels is assigned a plane label of the plane labels;
generate a secondary depth map based on the plane labels assigned to the pixels of the at least one image, wherein the apparatus is caused to generate the secondary depth map by refining the secondary depth map by assigning plane labels to one or more occlusion pixels based on plane labels of neighboring pixels of the one or more occlusion pixels;
and
determine a dominant disparity plane from the secondary depth map.

7 more elements

A computer program product comprising at least one non-transitory computer-readable storage medium, the at least one non-transitory computer-readable storage medium comprising a set of instructions, which, when executed by one or more processors, cause an apparatus to at least perform:

facilitate receipt of a primary depth map and a plurality of color segments, the primary depth map being associated with a first image and a second image of a scene, the first image and the second image being a stereoscopic pair of images of the scene, and the plurality of color segments being associated with the first image;
generate a plurality of disparity planes for the scene based on the plurality of color segments and the primary depth map, the plurality of disparity planes associated with a plurality of plane labels;
determine an aggregated cost volume between pixels of the first image and pixels of the second image for the plurality of disparity planes;
assign plane labels to pixels of at least one image of the first image and the second image based on the aggregated cost volume, wherein an individual pixel of the pixels is assigned a plane label of the plane labels;
generate a secondary depth map based on the plane labels assigned to the pixels of the at least one image, wherein the apparatus is caused to generate the secondary depth map by refining the secondary depth map by assigning plane labels to one or more occlusion pixels based on plane labels of neighboring pixels of the one or more occlusion pixels, and
determine a dominant disparity plane from the secondary depth map.

3 more elements

Dependencies are inferred from the claim text.