US 7,444,013 B2Granted

Method and apparatus for recovering depth using multi-plane stereo and spatial propagation

AssigneeSTMicroelectronics S.r.l.

InventorGeorge Q. Chen

Life of this patent

Estimating the term…

Loading patent…

Abstract

An image processing system recovers 3-D depth information for pixels of a base image representing a view of a scene. The system detects a plurality of pixels in a base image that represents a first view of a scene. The system the determines 3-D depth of the plurality of pixels in the base image by matching correspondence to a plurality of pixels in a plurality of images representing a plurality of views of the scene. The system then traces pixels in a virtual piecewise continuous depth surface by spatial propagation starting from the detected pixels in the base image by using the matching and corresponding plurality of pixels in the plurality of images to create the virtual piecewise continuous depth surface viewed from the base image, each successfully traced pixel being associated with a depth in the scene viewed from the base image.

Claims29

Dependent claims

A method of recovering depth information for pixels of a base image representing a view of a scene, the method comprising the steps of:

detecting a plurality of pixels in a base image that represents a first view of a scene, wherein the first view is one of at least three relative views of a scene;
determining 3-D depth of the plurality of pixels in the base image by contemporaneously matching image features between three or more relative images of a scene by matching correspondence to a plurality of pixels in three or more relative images representing the at least three relative views of the scene, wherein the three or more images are captured relative to each other and are of the same scene, and wherein each of the three or more images includes at least an overlapping portion of the scene relative to each other, wherein the at least three relative views of the scene are situated in a non-linear arrangement relative to each other and are further oriented in non-parallel planes relative to each other;
and
tracing pixels in a virtual piecewise continuous depth surface by spatial propagation starting from the detected pixels in the base image by using the matching and corresponding plurality of pixels in the three or more relative images to create the virtual piecewise continuous depth surface viewed from the base image, each successfully traced pixel being associated with a depth in the scene viewed from the base image.

1 more element

An image processing system comprising:

a memory;
a controller/processor electrically coupled to the memory;
an image matching module, electrically coupled to the controller/processor and to the memory, for providing a plurality of seed pixels that represent 3-D depth of the plurality of pixels in the base image view of a scene by contemporaneously matching image features between three or more relative images of a scene by matching correspondence to a plurality of pixels in three or more relative images representing a plurality of relative views of the scene, wherein the three or more relative images are captured relative to each other and are of the same scene, and wherein each of the three or more images includes at least an overlapping portion of the scene relative to each other, wherein the base image view is one of at least three relative views of the scene, the at least three relative views of the scene being situated in a non-linear arrangement and are further oriented in non-parallel planes relative to each other;
and
a propagation module, electrically coupled to the controller/processor and to the memory, for tracing pixels in a virtual piecewise continuous depth surface by spatial propagation starting from the provided plurality of seed pixels in the base image by using the matching and corresponding plurality of pixels in the three or more relative images to create the virtual piecewise continuous depth surface viewed from the base image, each successfully traced pixel being associated with a depth in the scene viewed from the base image.

2 more elements

A non-transistory computer readable medium including computer instructions for a 3-D image reconstruction computer system, the computer instructions comprising instructions for:

detecting a plurality of pixels in a base image that represents a first view of a scene, wherein the first view is one of at least three relative views of a scene;
determining 3-D depth of the plurality of pixels in the base image by contemporaneously matching image features between three or more relative images of a scene by matching correspondence to a plurality of pixels in three or more relative images representing the at least three relative views of the scene, wherein the three or more images are captured relative to each other and are of the same scene, and wherein each of the three or more images includes at least an overlapping portion of the scene relative to each other, wherein the at least three relative views of the scene are situated in a non-linear arrangement relative to each other and are further oriented in non-parallel planes relative to each other;
and
tracing pixels in a virtual piecewise continuous depth surface by spatial propagation starting from the detected pixels in the base image by using the matching and corresponding plurality of pixels in the three or more relative images to create the virtual piecewise continuous depth surface viewed from the base image, each successfully traced pixel being associated with a depth in the scene viewed from the base image.

1 more element

Dependencies are inferred from the claim text.