Method, apparatus, and device for synthesizing virtual viewpoint images

An apparatus for synthesizing virtual viewpoint images, comprising:

• an acquiring module configured for acquiring an original image set;
• a matching module configured for performing stereo matching between original images in the original image set and prestored images to obtain a first disparity map set;
• an optimizing module configured for performing optimization processing to each first disparity map in the first disparity map set according to a preset optimization rule to obtain a second disparity map set;
• a determining module configured for determining whether each second disparity map in the second disparity map set has a third disparity map with deep tomography to obtain a first determination result;
• a segmenting module configured for:
• if the first determination result indicates existence of the third disparity map, segmenting a portion with deep tomography in the third disparity map to obtain a foreground disparity map and a background disparity map which correspond to the portion with deep tomography in the third disparity map;
• a first mapping module configured for mapping, as first virtual viewpoints, the second disparity map without deep tomography, the foreground disparity map, the background disparity map, and the portion without deep tomography in the third disparity map to obtain a first virtual viewpoint image set;
• and
• a fusing module configured for fusing first virtual viewpoint images in the first virtual viewpoint image set to obtain a to-be-synthesized viewpoint image.

A device for synthesizing virtual viewpoint images, comprising:

• at least one processor;
• and
• a memory in communication connection with the at least one processor;
• wherein, the memory stores one or more instructions that may be executed by the at least one processor;
• and
• the one or more instructions are executed by the at least one processor, such that the at least one processor is configured for
• acquiring an original image set;
• performing stereo matching between original images in the original image set and prestored images to obtain a first disparity map set;
• performing optimization processing to each first disparity map in the first disparity map set according to a preset optimization rule to obtain a second disparity map set;
• determining whether each second disparity map in the second disparity map set has a third disparity map with deep tomography to obtain a first determination result;
• if the first determination result indicates existence of the third disparity map, segmenting a portion with deep tomography in the third disparity map to obtain a foreground disparity map and a background disparity map which correspond to the portion with deep tomography in the third disparity map;
• mapping the second disparity map without deep tomography, the foreground disparity map, the background disparity map, and the portion without deep tomography in the third disparity map according to a first virtual viewpoint to obtain a first virtual viewpoint image set;
• and
• fusing first virtual viewpoint images in the first virtual viewpoint image set to obtain a to-be-synthesized viewpoint image.

A non-transitory computer readable memory medium on which one or more instructions are stored, wherein the one or more instructions, when being executed, are configured for

• acquiring an original image set;
• performing stereo matching between original images in the original image set and prestored images to obtain a first disparity map set;
• performing optimization processing to each first disparity map in the first disparity map set according to a preset optimization rule to obtain a second disparity map set;
• determining whether each second disparity map in the second disparity map set has a third disparity map with deep tomography to obtain a first determination result;
• if the first determination result indicates existence of the third disparity map, segmenting a portion with deep tomography in the third disparity map to obtain a foreground disparity map and a background disparity map which correspond to the portion with deep tomography in the third disparity map;
• mapping the second disparity map without deep tomography, the foreground disparity map, the background disparity map, and the portion without deep tomography in the third disparity map according to a first virtual viewpoint to obtain a first virtual viewpoint image set;
• and
• fusing first virtual viewpoint images in the first virtual viewpoint image set to obtain a to-be-synthesized viewpoint image.