Stereo reconstruction employing a layered approach and layer refinement techniques

A system for extracting 3D structure from plural, stereo, 2D images of a scene, comprising:

• a general purpose computing device;
• a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules to,
• compute from the 2D images input to the computing device a representation of the scene comprising plural layers, each of which is characterized by estimated layer parameters at least including the layer's orientation and position in the form of a plane equation, and per-pixel color and per-pixel boolean opacity in the form of a sprite image,
• refine the estimates of the plane equation and sprite image for each layer, said refining comprising,
• characterizing the opacities of each pixel of each estimated sprite image as real values representing the degree of transparency of the pixel,
• synthesizing sprite images representing each input image from the plural layers,
• comparing the synthesized sprite images to the actual input images,
• revising estimates for the sprite image of each layer by deriving the sprite image for each layer, which will produce the minimum degree of difference between the synthesized sprite images and the input images, and
• re-estimating the plane equations for each layer based on the revised sprite image estimates,
• the computer thereafter outputting the layer parameters to define the 3D structure of the scene.

A system for extracting 3D structure from plural, stereo, 2D images of a scene, comprising:

• a general purpose computing device;
• a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules to,
• compute from the 2D images input to the computing device a representation of the scene comprising plural layers, each of which is characterized by estimated layer parameters including the layer's orientation and position in the form of a plane equation, per-pixel color and per-pixel boolean opacity in the form of a sprite image and a per-pixel depth in the form of a residual depth map defining the distance each pixel associated with the layer is offset from that layer,
• refine the estimates of the plane equation, sprite image and residual depth maps for each layer,
• the computer thereafter outputting the layer parameters to define the 3D structure of the scene.

A computer-readable memory for causing a computer to extract 3D structure from plural, stereo, 2D images of a scene, comprising:

• a computer-readable storage medium;
• and
• a computer program comprising program modules stored in the storage medium, wherein the storage medium is so configured by the computer program that it causes the computer to,
• compute from the 2D images input to the computer a representation of the scene comprising plural layers, each of which is characterized by estimated layer parameters at least including the layer's orientation and position in the form of a plane equation, and per-pixel color and per-pixel boolean opacity in the form of a sprite image, and
• refine the estimates of the plane equation and sprite image for each layer, said refining comprising,
• characterizing the opacities of each pixel of each estimated sprite image as real values representing the degree of transparency of the pixel,
• synthesizing sprite images representing each input image from the plural layers,
• comparing the synthesized sprite images to the actual input images,
• revising estimates for the sprite image of each layer by deriving the sprite image for each layer, which will produce the minimum degree of difference between the synthesized sprite images and the input images, and
• re-estimating the plane equations for each layer based on the revised sprite image estimates.

A computer-readable memory for causing a computer to extract 3D structure from plural, stereo, 2D images of a scene, comprising:

• a computer-readable storage medium;
• and
• a computer program comprising program modules stored in the storage medium, wherein the storage medium is so configured by the computer program that it causes the computer to,
• compute from the 2D images input to the computer a representation of the scene comprising plural layers, each of which is characterized by estimated layer parameters including the layer's orientation and position in the form of a plane equation, per-pixel color and per-pixel boolean opacity in the form of a sprite image and a per-pixel depth in the form of a residual depth map defining the distance each pixel associated with the layer is offset from that layer, and
• refine the estimates of the plane equation, sprite image and residual depth maps for each layer.