Digital image processing error concealment method

#The present invention relates to digital image processing error concealment, and in particular, to occlusion handling for motion compensated interpolated frames.

#In a digital image processing system, images are represented as binary data (discrete values represented as zeros and ones). Changing images, such as video or graphics, are typically broken down into a stream of frames. Each frame may include a number of static objects represented by an array of pixels. Different values for different pixels correspond to a wide range of colors (e.g., RGB or YUV) across each frame to represent different visual objects in the frame. In a video or graphics system, for example, the objects may change location from frame to frame, thereby creating the appearance of movement.

#FIG. 1 illustrates an image 101 with a moving object 102. Object 102 may move across the image with a velocity, v, which specifies both the direction and speed of the object's movement across the image. As objects move across an image from frame to frame, leading edges of an object may cover some background images and trailing edges of the object may uncover other background images. This is illustrated in FIG. 2, which shows the movement of an image across multiple frames. In FIG. 2, an object 102 changes location between frames 201A and 201B. In this example, object 102 moves horizontally a distance, d, between frames. As illustrated in frame 201C, a first region 211 is occluded by the leading (or forward) edge of the object, and a second region 210 is revealed by the trailing (or rear) edge of the object.

#FIG. 1 illustrates an image with a moving object.

#FIG. 2 illustrates the movement of an image across multiple frames.

#FIG. 3 illustrates an occlusion region and associated error.

#Described herein are techniques for concealing errors in digital images. In the following description, for purposes of explanation, numerous examples and specific details are set forth in order to provide a thorough understanding of the present invention. It will be evident, however, to one skilled in the art that the present invention as defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

#Features and advantages of the present invention include a method for error concealment in digital images with moving objects. In one embodiment, the location of regions with high error are determined and more accurate replacement pixels are generated based on an in-painting technique described below. For example, in-painting uses the spatial and temporal pixels around an error region to estimate the pixels with the lowest error to be used as replacement pixels. The method described herein may be implemented in an image processing system, for example, as described in more detail below.

#FIG. 3 illustrates an occlusion region and associated error. Generally, when a foreground object moves onto the background, a motion estimator using information from the successive frames may not find a good match in the region that undergoes occlusion. In this example, occlusion region 211 may result in an error region 220, where pixel values may need to be replaced to improve the image quality. While the disclosure below is described in the context of occlusion, it is to be understood that the techniques described herein may also be applied to a region that undergoes uncovering. As described in more detail below, embodiments of the present invention may use motion vector profiles to decide whether to copy information from the foreground or from the background to correct for occlusion errors. Image information from the spatial neighborhood may be used to replace information in the error region and improve image quality.