Method and system for processing multiview videos for view synthesis using skip and direct modes

A method for processing a plurality of multiview videos of a scene, in which each video is acquired by a corresponding camera arranged at a particular pose, and in which a view of each camera overlaps with the view of at least one other camera, comprising the steps of:

• obtaining side information for synthesizing a particular view of the multiview videos;
• synthesizing a synthesized reference picture from at least one input video selected from the plurality of multiview videos and the side information, wherein the synthesized reference picture corresponds to a single pose different than the input video;
• maintaining a reference picture list for each current frame of each of the plurality of multiview videos, wherein the reference picture list indexes temporal reference pictures and spatial reference pictures of the plurality of multiview videos and the synthesized reference picture and wherein the temporal reference pictures are associated with different time instants and the spatial reference pictures are associated with a same time instant;
• and
• predicting each current frame corresponding to the single pose of the synthesized reference picture according to reference pictures indexed by the associated reference picture list, wherein the predicting uses a synthetic skip mode and a synthetic direct mode based on the synthesized reference picture, and wherein the side information is inferred from an earliest synthesized reference picture in the reference picture list.