Depth map generation from a monoscopic image based on combined depth cues
AssigneeGoogle Inc.
InventorsChen Wu, Debargha Mukherjee, Meng Wang
Life of this patent
Estimating the term…
Abstract
A combined depth map is generated for a monoscopic image based on a weighted combination of a color depth map, a spatial depth map, and a motion depth map for the image, each describing the depth of pixels in the image relative to the image plane. The color depth map determines the depth of pixels according to their color, the spatial depth map determines the depth of pixels according to their location in the image, and the motion depth map determines the depth of pixels according to their motion in the image. Each of the depth maps is associated with a weight that is used to generate the weighted combination. The weights are adaptive to account for variation between different monoscopic images.
Claims18
A method for generating a depth map of an image, the method comprising:
- accessing the image, the image comprising a plurality of pixels, wherein each pixel has a color and a location in the image;
- determining a color depth map for the image based upon the colors of the pixels in the image;
- determining a spatial depth map for the image based upon the locations of pixels and historical depth information for pixels in similar locations in a plurality of other images;
- determining a motion depth map for the image based upon pixel motion in the image;
- determining a color depth map weight, a spatial depth map weight, and a motion depth map weight, the spatial depth map weight determined based on historical depth variance information for the pixels in similar locations in a plurality of other images, the historical depth variance information describing a variance of the historical depth information;
- and
- generating a combined depth map for the image from a combination of the color depth map weighted by the color depth map weight, the spatial depth map weighted by the spatial depth map weight, and the motion depth map weighted by the motion depth map weight.
A non-transitory computer-readable storage medium storing computer program instructions for generating a depth map of an image, the computer program instructions executable to perform steps comprising:
- accessing the image, the image comprising a plurality of pixels, wherein each pixel has a color and a location in the image;
- determining a color depth map for the image based upon the colors of the pixels in the image;
- determining a spatial depth map for the image based upon the locations of pixels and historical depth information for pixels in similar locations in a plurality of other images;
- determining a motion depth map for the image based upon pixel motion in the image;
- determining a color depth map weight, a spatial depth map weight, and a motion depth map weight, the spatial depth map weight determined based on historical depth variance information for the pixels in similar locations in a plurality of other images, the historical depth variance information describing a variance of the historical depth information;
- and
- generating a combined depth map for the image from a combination of the color depth map weighted by the color depth map weight, the spatial depth map weighted by the spatial depth map weight, and the motion depth map weighted by the motion depth map weight.
A method for generating a depth map of an image, the method comprising:
- accessing the image, the image comprising a plurality of pixels, wherein each pixel has a color and a location in the image;
- determining a color depth map for the image based upon the colors of the pixels in the image;
- determining a spatial depth map for the image based upon the locations of pixels and historical depth information for pixels in similar locations in a plurality of other images;
- determining a motion depth map for the image based upon pixel motion in the image;
- determining a color depth map weight, a spatial depth map weight, and a motion depth map weight, the motion depth map weight determined based on a percentage of pixels in the image having local motion;
- and
- generating a combined depth map for the image from a combination of the color depth map weighted by the color depth map weight, the spatial depth map weighted by the spatial depth map weight, and the motion depth map weighted by the motion depth map weight.
Dependencies are inferred from the claim text.