Back to results
US 12,568,184 B2Granted
Techniques to generate interpolated video frames
AssigneeNVIDIA Corporation
InventorsFitsum Reda, Karan Sapra, Robert Thomas Pottorff, Shiqiu Liu, Andrew Tao, Bryan Christopher Catanzaro
Life of this patent
Estimating the term…
Abstract
Apparatuses, systems, and techniques to generate interpolated video frames. In at least one embodiment, an interpolated video frame is generated based, at least in part, on a first set of pixel data sampled from a first video frame, and a second set of pixel data sampled from a second video frame based, at least in part, on a set of forward pointing motion vectors from the first video frame to the second video frame.
Claims35
A processor, comprising:
- one or more circuits to:
- sample a first set of pixel data from a first video frame;
- sample a second set of pixel data from a second video frame based, at least in part, on a set of forward pointing motion vectors from the first video frame to a pixel location in the second video frame, wherein the first video frame and the second video frame were captured at different times;
- determine weight factors to be applied to the first and second sets of pixel data by:
- generating an occlusion mask and a dis-occlusion mask;
- for pixels not affected by the occlusion mask or the dis-occlusion mask, determining the weight factors based at least in part on first and second amounts of time between an intermediate video frame and the first and second video frames, respectively;
- and
- for pixels affected by the occlusion mask or the dis-occlusion mask, selecting the weight factors from a set of predetermined values;
- and
- generate the intermediate video frame at least in part by applying the weight factors to the first set of pixel data and the second set of pixel data.
7 more elements
A non-transitory computer-readable storage medium having stored thereon a set of instructions which, if performed by one or more processors, causes the one or more processors to at least:
- sample a first set of pixel data from a first video frame;
- sample a second set of pixel data from a second video frame based, at least in part, on a set of forward pointing motion vectors from the first video frame to a pixel location in the second video frame, wherein the first and second video frames were captured at different times;
- determine weight factors to be applied to the first and second sets of pixel data;
- generate an occlusion mask and a dis-occlusion mask, for pixels not affected by the occlusion mask or the dis-occlusion mask, the weight factors to be based at least in part on first and second amounts of time between an intermediate video frame and the first and second video frames, respectively, and for pixels affected by the occlusion mask or the dis-occlusion mask, the weight factors to be selected from a set of predetermined values;
- and
- generate the intermediate video frame by applying the weight factors to the first set of pixel data and the second set of pixel data.
3 more elements
A method, comprising:
- sampling a first set of pixel data from a first video frame;
- sampling a second set of pixel data from a second video frame based, at least in part, on a set of forward pointing motion vectors from the first video frame to a pixel location in the second video frame, wherein the first and second video frames were captured at different times;
- generating an occlusion mask and a dis-occlusion mask;
- for pixels not affected by the occlusion mask or the dis-occlusion mask, determining weight factors based at least in part on first and second amounts of time between an intermediate video frame and the first and second video frames, respectively;
- for pixels affected by the occlusion mask or the dis-occlusion mask, selecting the weight factors from a set of predetermined values;
- and
- generating the intermediate video frame at least in part by applying the weight factors to the first set of pixel data and the second set of pixel data.
4 more elements
A system, comprising:
- one or more processors to:
- sample a first set of pixel data from a first video frame;
- sample a second set of pixel data from a second video frame based, at least in part, on a set of forward pointing motion vectors from the first video frame to a pixel location in the second video frame, wherein the first video frame and the second video frame were captured at different times;
- determine weight factors to be applied to the first and second sets of pixel data;
- generate an occlusion mask and a dis-occlusion mask, for any pixels not affected by the occlusion mask or the dis-occlusion mask, the weight factors to be based at least in part on first and second amounts of time between an intermediate video frame and the first and second video frames, respectively, and for any pixels affected by the occlusion mask or the dis-occlusion mask, the weight factors to be selected from a set of predetermined values;
- and
- generate the intermediate video frame at least in part by applying the weight factors to the first set of pixel data and the second set of pixel data.
4 more elements
Dependencies are inferred from the claim text.