Example-based two-dimensional to three-dimensional image conversion method, computer readable medium therefor, and system

An image conversion method of converting an input two-dimensional image into three-dimensional image content, the method comprising the steps of:

• selecting one of a plurality of example background images of an image database as a matched background image;
• performing a rectification operation to rectify the matched background image, so that the rectified matched background image and the input two-dimensional image are located on the same reference plane, to obtain a rectified background image according to the matched background image and the input two-dimensional image;
• performing a segmentation operation on the input two-dimensional image to obtain a segmented image, which comprises a plurality of patches;
• performing a foreground and background detection operation to separate the patches of the segmented image into a foreground patch group and a background patch group according to the input two-dimensional image, the rectified background image, and the segmented image;
• and
• performing a depth estimation operation to set a depth value corresponding to each of the patches of the background patch group and the foreground patch group and to obtain a depth map of the input two-dimensional image according to depth information provided by the rectified background image,
• wherein the three-dimensional image content is based on the input two-dimensional image and the depth map.

An image conversion method of converting an input two-dimensional image into three-dimensional image content, the method comprising the steps of:

• selecting one of a plurality of example background images of an image database as a matched background image, wherein content of the matched background image, among the other the example background images, most closely approximates content of the input two-dimensional image;
• performing a rectification operation to rectify the matched background image, such that the rectified matched background image and the input two-dimensional image are located on the same reference plane, to obtain a rectified background image according to the matched background image and the input two-dimensional image;
• performing a segmentation operation on the input two-dimensional image to obtain a segmented image, which comprises a plurality of patches, wherein each of the patches respectively represents correspondingly homogeneous pixels included in the input two-dimensional image;
• performing a foreground and background detection operation according to the input two-dimensional image, the rectified background image and the segmented image, wherein the step of performing the foreground and background detection operation comprises:
• performing a subtraction operation on the input two-dimensional image and the rectified background image;
• and
• performing a foreground and background classification according to a result of the subtraction operation and the segmented image in order to classify the patches of the segmented image as a foreground or a background, and separating the patches of the segmented image into a foreground patch group and a background patch group;
• performing an object detection operation on the foreground patch group to determine whether there is any patch of the foreground patch group satisfying at least one object feature, and performing a boundary smoothing process on each patch satisfying the at least one object feature to obtain an adjusted foreground patch group, which is based on the patches which are processed by the boundary smoothing process;
• and
• performing a depth estimation operation according to depth information provided by the rectified background image in order to assign a depth value corresponding to each of the patches of the background patch group and the adjusted foreground patch group and to obtain a depth map of the input two-dimensional image accordingly,
• wherein the three-dimensional image content is based on the input two-dimensional image and the depth map.

An image conversion system for converting an input two-dimensional image into three-dimensional image content, the system comprising:

• an image matching unit for determining a matched background image according to an image database, wherein the matched background image most closely approximates the input two-dimensional image in content,;
• a rectifying unit for obtaining a rectified background image, which is on a same image-capturing plane as that of the input two-dimensional image, according to the matched background image and the input two-dimensional image;
• a segmenting unit for performing a segmentation operation on the input two-dimensional image to obtain a segmented image, which comprises a plurality of patches;
• a foreground and background detecting unit for separating the patches of the segmented image into a foreground patch group and a background patch group by performing a subtraction operation on the input two-dimensional image and the rectified background image according to the input two-dimensional image, the rectified background image and the segmented image;
• and
• a depth estimating unit for setting a depth value corresponding to each of the patches of the background patch group and the foreground patch group to obtain a depth map of the input two-dimensional image according to depth information provided by the rectified background image,
• wherein the three-dimensional image content is based on the input two-dimensional image and the depth map.

An image conversion system for converting a two-dimensional image into three-dimensional image content, the system comprising:

• a memory device;
• and
• a microprocessor, coupled to the memory device, for reading commands stored in the memory device to:
• determine a matched background image, which most closely approximates content of the two-dimensional image, according to a digital image database;
• perform a rectification operation according to the matched background image and the two-dimensional image to obtain a rectified background image, wherein the rectified background image and the two-dimensional image are on a same image-capturing plane;
• perform a segmentation operation on the two-dimensional image to obtain a segmented image, wherein the segmented image comprises a plurality of patches, and each of the patches respectively represents a portion of pixels in the two-dimensional image correspondingly having similarity;
• perform a subtraction operation on the two-dimensional image and the rectified background image according to the two-dimensional image, the rectified background image and the segmented image, and to perform a foreground and background detection operation to separate the patches of the segmented image into a foreground patch group and a background patch group;
• and
• perform a depth estimation operation to set a depth value corresponding to each of the patches of the background patch group and the foreground patch group to obtain a depth map of the two-dimensional image according to depth information provided by the rectified background image,
• wherein the three-dimensional image content is based on the two-dimensional image and the depth map.