US 2013/0170551 A1Published application

Halo Reduction in Frame-Rate-Conversion Using Hybrid Bi-Directional Motion Vectors for Occlusion/Disocclusion Detection

AssigneeHong Kong Applied Science and Technology Research Institute Company Limited

InventorsXuejiao LIU, King Hung CHIU, Peng LUO, Tim Ka Lung WONG

Life of this patent

Estimating the term…

Loading patent…

Abstract

A frame-rate converter reduces halo artifacts along edges of moving objects. Halo artifacts occur on interpolated frames where a moving object covers and uncovers pixels along its edges. Motion estimation among three original frames produces hybrid direction motion vectors that are bi-directional for background and objects, but are unidirectional for covered and uncovered regions, since motion vectors with large matching errors are deleted. Covered regions in the interpolated frame are detected as intersecting only a forward but no backward hybrid motion vector. Bi-directional motion estimation from the hybrid motion vectors of two original frames produces refined motion vectors for the interpolated frame. Refined motion vectors in the covered regions are deleted and replaced with hybrid motion vectors from the original frames. Hybrid motion vectors from the original frames are assigned to the critical covered regions rather than using interpolated vectors in the covered regions, reducing halo artifacts.

Claims23

Dependent claims

A halo-reducing frame rate converter comprising:

a forward motion vector generator, receiving a current frame of graphics data and a next frame of graphics data, and generating forward motion vectors for the current frame, the forward motion vectors each pointing to a best-matching block in the next frame;
a backward motion vector generator, receiving the current frame of graphics data and generating backward motion vectors, the backward motion vectors each pointing to a best matching block in an earlier frame in a sequence of frames;
wherein the forward motion vectors each have a block-matching error to the best-matching block in the next frame;
wherein the backward motion vectors each have a block-matching error to the best-matching block in the earlier frame;
a hybrid direction motion vector generator, receiving the forward motion vectors for the current frame, and receiving backward motion vectors for the current frame, the hybrid direction motion vector generator generating hybrid direction motion vectors for the current frame that include the forward motion vectors and the backward motion vectors, wherein forward motion vectors having block-matching errors above a threshold are deleted from the hybrid direction motion vectors for the current frame, wherein backward motion vectors having block-matching errors above the threshold are deleted from the hybrid direction motion vectors for the current frame;
a covered region detector that projects a forward trajectory for each forward motion vector in the hybrid direction motion vectors for a prior frame, and that projects a backward trajectory for each backward motion vector in the hybrid direction motion vectors for the current frame, the covered region detector indicating a covered region in an interpolated frame between the prior frame and the current frame, wherein the covered region has at least one forward trajectory but not any backward trajectory intersecting the covered region;
an interpolated motion vector refiner, that generates refined motion vectors for the interpolated frame by bi-directional motion estimation using the hybrid direction motion vectors for the prior frame and for the current frame;
wherein the refined motion vectors include faulty motion vectors in the covered region;
and
a motion vector corrector that assigns hybrid direction motion vectors from the prior frame to replace the faulty motion vectors in the covered region detected by the covered region detector to generate corrected motion vectors for the interpolated frame,
whereby faulty motion vectors in the covered region are replaced by hybrid direction motion vectors from an original frame after the covered region is detected by intersection of least one forward trajectory but not any backward trajectory.

8 more elements

A method for reducing halo artifacts when interpolating video frames comprising:

generating forward motion vectors for a current frame using motion estimation from the current frame to a next frame in a series of frames of encoded video;
generating backward motion vectors for the current frame using motion estimation from the current frame back to a prior frame;
generating hybrid direction motion vectors for the current frame by including the forward motion vectors for the current frame that have errors that are less than an error threshold, and by including the backward motion vectors for the current frame that have errors that are less than the error threshold;
wherein an interpolated frame is between the prior frame and the current frame, the interpolated frame having no video data in an original video that includes the prior frame, the current frame, and the next frame;
detecting covered regions by projecting hybrid direction motion vectors that point forward from the prior frame to intersect the interpolated frame at first points, and by projecting hybrid direction motion vectors that point backward from the current frame to intersect the interpolated frame at second points, and by identifying covered regions as blocks having first points but not having any second points;
refining hybrid direction motion vectors for the prior frame and for the current frame to generate refined motion vectors for the interpolated frame;
replacing the refined motion vectors for blocks in the covered regions with hybrid direction motion vectors for corresponding blocks from the prior frame to generate corrected motion vectors for the interpolated frame;
and
using the corrected motion vectors for the interpolated frame to fetch blocks from the prior frame and from the current frame to generate blocks of video data for the interpolated frame,
whereby covered regions are assigned motion vectors from original frames while other regions use refined motion vectors.

7 more elements

A frame interpolator comprising:

forward motion estimating means for generating forward motion vectors for a current frame using motion estimation from the current frame to a next frame in a series of frames of encoded video;
backward motion estimating means for generating backward motion vectors for the current frame using motion estimation from the current frame back to a prior frame;
hybrid direction motion vector generating means for generating hybrid direction motion vectors for the current frame by including the forward motion vectors for the current frame that have errors that are less than an error threshold, and by including the backward motion vectors for the current frame that have errors that are less than the error threshold;
wherein an interpolated frame is between the prior frame and the current frame, the interpolated frame having no video data in an original video that includes the prior frame, the current frame, and the next frame;
covered region detect means for detecting covered regions by projecting hybrid direction motion vectors that point forward from the prior frame to intersect the interpolated frame at first points, and by projecting hybrid direction motion vectors that point backward from the current frame to intersect the interpolated frame at second points, and by identifying covered regions as blocks having first points but not having any second points;
refine means for refining hybrid direction motion vectors for the prior frame and for the current frame to generate refined motion vectors for the interpolated frame;
assignment means for replacing the refined motion vectors for blocks in the covered regions with hybrid direction motion vectors for corresponding blocks from the prior frame to generate corrected motion vectors for the interpolated frame;
and
motion compensate means for using the corrected motion vectors for the interpolated frame to fetch blocks from the prior frame and from the current frame to generate blocks of video data for the interpolated frame,
whereby covered regions are assigned motion vectors from original frames while other regions use refined motion vectors.

7 more elements

Dependencies are inferred from the claim text.