Method and apparatus for frame interpolation
AssigneeVestel Elektronik Sanayi ve Ticaret A. S.
InventorsOsman Serdar Gedik, Abdullah Aydin Alatan
Life of this patent
Estimating the term…
Abstract
The invention generally provides a method and apparatus for up-converting the frame rate of a digital video signal, the method comprising: receiving a digital video signal containing a first frame and a second frame; finding in one of the received frames, matches for objects in the other of the received frames; utilizing 3 dimensional position data in respect of the objects within the frames to determine 3 dimensional movement matrices for the matched objects; using the 3 dimensional movement matrices, determining the position of the objects in a temporally intermediate frame and thereby generating an interpolated frame, temporally between the first and second frame.
Claims24
A method of up-converting a frame rate of a digital video signal, the method comprising:
- receiving a digital video signal containing a plurality of temporally spaced apart image frames in one view of a multiview set, the plurality of temporally spaced apart image frames include two consecutive frames comprising a first frame and a second frame;
- finding in one of the received frames, a match for an object in the other of the received frames;
- utilizing three-dimensional (3D) data in respect of the matched object within the received frames to determine 3D movement matrices for the matched object;
- using the 3D movement matrices, determining a temporally intermediate 3D position of the matched object with respect to the received frames to enable an interpolated frame to be generated temporally between the first and second frame;
- and
- projecting the 3D position of the matched object onto any two-dimensional view of the multiview set to thereby generate the interpolated frame at a desired time instant and view;
- wherein finding a match for an object comprises segmenting a moving object by obtaining the differences between the two consecutive frames by calculating pixel-wise absolute differences between depth, D, and texture, C, frames at time instants t and t+1:
- CD(i,j)=abs(C(i,j,t,n)−C(i,j,t+1,n))
- DD(i,j)=abs(D(i,j,t,n)−D(i,j,t+1,n))
- and using these frames calculating a global segmentation map.
A method of up-converting the frame rate of a digital video signal, the method comprising:
- receiving a digital video signal containing a plurality of temporally spaced apart image frames in one view of a multiview set, the plurality of temporally spaced apart image frames include two consecutive frames comprising a first frame and a second frame, each frame having an associated time and comprising one or more three-dimensional (3D) objects having a position;
- finding in one of the received frames, matches for the one or more objects in the other of the received frames;
- utilizing 3D data in respect of the one or more matched objects within the received frames to determine 3D movement matrices for the one or more matched objects;
- using the 3D movement matrices and the 3D position of the one or more matched objects in one of the received frames, interpolating a 3D position of the one or more matched objects at the time of the other received frame;
- comparing the interpolated 3D position at the time of the other received frame and the 3D position of the one or more matched objects in the other received frame;
- modifying the 3D movement matrices based on the comparison;
- and
- using the modified 3D movement matrices, determining the position of the one or more matched objects in three dimensions in a temporally intermediate frame to enable an interpolated frame in any two-dimensional view of the multiview set to be generated temporally between the first and second frame;
- wherein finding a match for an object comprises segmenting a moving object by obtaining the differences between the two consecutive frames by calculating pixel-wise absolute differences between depth, D, and texture, C, frames at time instants t and t+1:
- CD(i,j)=abs(C(i,j,t,n)−C(i,j,t+1,n))
- DD(i,j)=abs(D(i,j,t,n)−D(i,j,t+1,n))
- and using these frames calculating a global segmentation map.
Dependencies are inferred from the claim text.