Back to results
US 7,606,313 B2Granted
System, method, and apparatus for error concealment in coded video signals
AssigneeIttiam Systems (P) Ltd.
InventorsArvind Raman, Murali Babu Muthukrishnan
Life of this patent
Estimating the term…
Abstract
Efficient techniques are provided to detect and correct channel errors found in an encoded video signal. In one example embodiment, header information in a video frame is analyzed to detect channel errors in the video frame. The video frame is then corrected for detected channel errors by isolating the detected channel errors to a few macroblocks in the video frame to reduce data loss and improve video quality.
Claims22
A method comprising:
- detecting a channel error by locating a damaged macroblock (Px, y) in multiple macroblocks of a video frame using header information by a video decoder;
- and
- isolating the detected channel error to a few macroblocks around the located damaged macroblock to reduce data loss and improve video quality, wherein isolating the detected channel error comprises:
- estimating the damaged macroblock by using undamaged macroblocks substantially surrounding a boundary of the damaged macroblock in the video frame by the video decoder, wherein the undamaged macroblocks include 1 to 4 undamaged macroblocks (Px, y−1, P x+1, y, Px, y+1, and Px−1, y) and wherein estimating the damaged macroblock includes estimating the damaged macroblock by using a weighted linear interpolation of the undamaged macroblocks surrounding the damaged macroblock;
- and
- replacing the damaged macroblock with the estimated damaged macroblock to conceal the error in the damaged macroblock by the video decoder.
3 more elements
A method comprising:
- detecting an error by locating a damaged macroblock (Px, y)in multiple macroblocks in a video frame using header information, global information, and/or video packet information in the video frame by a video decoder;
- estimating a pixel value for each pixel in the damaged macroblock by computing a weighted sum of the associated pixel values in each of the undamaged macroblocks surrounding the damaged macroblock by the video decoder, wherein, in estimating, the undamaged macroblocks surrounding the damaged macroblock include about 1 to 4 undamaged macroblocks (Px, y−1, Px+1, y, Px, y+1, and Px−1, y);
- and
- copying the estimated pixel value of each pixel in the damaged macroblock to conceal the error in the damaged macroblock by the video decoder.
1 more element
A method comprising:
- detecting an error by locating a damaged macroblock in multiple macroblocks in a video frame using header information, global information, and/or video packet information in the video frame by a video decoder;
- estimating a pixel value for each pixel in the damaged macroblock by computing a weighted sum of the associated pixel values in each of the undamaged macroblocks surrounding the damaged macroblock by the video decoder;
- and
- copying the estimated damaged macroblock to conceal the error in the damaged macroblock by the video decoder,
- wherein estimating the pixel value of each pixel in the damaged macroblock by computing a weighted sum of the associated pixels in each of the undamaged macroblocks surrounding the damaged macroblock comprises:
- checking each of the macroblocks surrounding the damaged macroblock for an error in the current video frame;
- if there is an error, then choosing undamaged macroblocks based on the detected error and computing a weighted sum of absolute differences between pixel values in the chosen undamaged macroblocks surrounding the damaged macroblock in the current frame and associated pixel values in associated macroblocks in a previous video frame;
- if there is no error, then choosing all macroblocks surrounding the damaged macroblock as the undamaged macroblocks and computing a weighted sum of absolute differences between pixel values in the chosen macroblocks surrounding the damaged macroblock and associated macroblocks in the previous video frame;
- determining a number of chosen undamaged macroblocks used in computing the weighted sum of absolute differences;
- computing a sum value by dividing the computed weighted sum of the absolute differences by the number of chosen undamaged macroblocks;
- comparing the computed sum value with a threshold value;
- if the computed sum value is less than or equal to the threshold value, then using a spatial interpolation to estimate the damaged macroblock to conceal the error;
- and
- if the computed sum value is greater than the threshold value, then copying an associated undamaged macroblock in a previous video frame to conceal the error.
11 more elements
A method comprising:
- detecting a channel error by locating a damaged macroblock in multiple macroblocks in a current video frame using header information, global information, and/or video packet information in the video frame by a video decoder;
- reconstructing the damaged macroblock by estimating a motion vector of the damaged macroblock using motion vectors of undamaged macroblocks surrounding the damaged macroblock by the video decoder, wherein reconstruction of the damaged macroblock by estimating the motion vector of the damaged macroblock using the motion vectors of the undamaged macroblocks surrounding the damaged macroblock comprises:
- estimating the motion vector of the damaged macroblock in the current video frame;
- estimating a motion vector of a macroblock located substantially adjacent and above the damaged macroblock;
- estimating a motion vector of a macroblock located substantially below the damaged macroblock;
- estimating a motion vector of a macroblock located substantially adjacent, above, and left of the damaged macroblock;
- checking for error in the macroblock located substantially below the damaged macroblock;
- and
- if there is an error in the macroblock located substantially below the damaged macroblock, then estimating the motion vector of the macroblock located substantially below the damaged macroblock by assigning a motion vector of a macroblock located about 2 rows below the damaged macroblock;
- and
- copying the reconstructed damaged macroblock to conceal the error in the damaged macroblock by the video decoder.
8 more elements
A method comprising:
- detecting a channel error by locating a damaged macroblock in multiple macroblocks in a current video frame using header information, global information, and/or video packet information in the video frame by a video decoder;
- reconstructing the damaged macroblock by estimating a motion vector of the damaged macroblock using motion vectors of undamaged macroblocks surrounding the damaged macroblock by the video decoder, wherein reconstruction of the damaged macroblock by estimating the motion vector of the damaged macroblock using the motion vectors of the undamaged macroblocks surrounding the damaged macroblock comprises:
- estimating the motion vector of the damaged macroblock in the current video frame;
- estimating a motion vector of a macroblock located substantially adjacent and above the damaged macroblock;
- estimating a motion vector of a macroblock located substantially below the damaged macroblock;
- estimating a motion vector of a macroblock located substantially adjacent, above, and left of the damaged macroblock;
- checking for error in the macroblock located substantially below the damaged macroblock;
- if there is an error in the macroblock located substantially below the damaged macroblock, then estimating the motion vector of the damaged macroblock as the motion vector of the macroblock located substantially adjacent and above the damaged macroblock;
- if there is error in the associated macroblock located substantially below the damaged macroblock, then estimating the motion vector of the macroblock located substantially below the damaged macroblock by assigning a motion vector of a macroblock located about 2 rows below the damaged macroblock, and then computing a motion distance by computing a sum of absolute difference between the motion vectors of the macroblock located substantially adjacent and above the damaged macroblock and the macroblock located substantially below the damaged macroblock;
- if the computed motion distance is less than a motion threshold value, then estimating the motion of the damaged macroblock using the equation:
- motion vector of the damaged macroblock=((α)*motion vector of the macroblock located substantially adjacent and above the damaged macroblock+(1-α)*motion vector of the macroblock located substantially below the damaged macroblock);
- if the computed motion distance is greater than or equal to the motion threshold value, then computing a first motion vector difference by computing a sum of absolute difference between the motion vectors of the macroblock located substantially adjacent, above, and left of the damaged macroblock and the macroblock located substantially adjacent and above the damaged macroblock, and further computing a second motion vector difference by computing a sum of absolute difference between the motion vectors of the macroblock located substantially adjacent, above, and left of the damaged macroblock and the macroblock located substantially below the damaged macroblock;
- if the first motion vector difference is less than or equal to the second motion vector difference, then estimating the motion vector of the damaged macroblock as the motion vector of the macroblock located substantially adjacent and above the damaged macroblock;
- and
- if the first motion vector difference is greater than the second motion vector difference, then estimating the motion vector of the damaged macroblock as the motion vector of the macroblock located substantially below the damaged macroblock;
- and
- copying the reconstructed damaged macroblock to conceal the error in the damaged macroblock by the video decoder.
14 more elements
An apparatus comprising:
- a header decoding module parses a video frame to get header information and multiple macroblocks;
- an error recovery module detects a channel error by locating a damaged macroblock (Px, y) in the multiple macroblocks using the header information;
- and
- a spatial data error concealment module estimates the damaged macroblock by using undamaged macroblocks substantially surrounding a boundary of the damaged macroblock and replaces the damaged macroblock with the estimated damaged macroblock to conceal the channel error in the damaged macroblock, wherein the undamaged macroblocks include 1 to 4 undamaged macroblocks (Px, y−1, Px+1, y, Px, y+1, and Px−1, y), and wherein the spatial data error concealment module estimates the damaged macroblock by using a weighted linear interpolation of the undamaged macroblocks surrounding the damaged macroblock.
1 more element
A video decoder comprising:
- a header decoding module to parse a video frame to get header information, video packet information, and multiple macroblocks;
- an error recovery module to detect an error in a damaged macroblock (Px, y) in the multiple macroblocks of the video frame using the header information and/or the video packet information;
- and
- a spatial data error concealment module to estimate a pixel value for each pixel in the damaged macroblock by computing a weighted sum of associated pixel values in each of undamaged macroblocks surrounding the damaged macroblock, wherein the spatial data error concealment module to copy the estimated pixel value of each pixel in the damaged macroblock to conceal the error in the damaged macroblock, wherein the undamaged macroblocks surrounding the damaged macroblock include about 1 to 4 undamaged macroblocks (Px, y−1, Px+1, y, Px, y+1, and/or Px−1, y) substantially surrounding the damaged macroblock.
1 more element
An apparatus for decoding a coded video signal comprising:
- a header decoding module parses a video frame to get header information and multiple macroblocks;
- an error recovery module detects a channel error by locating a damaged macroblock in the multiple macroblocks using the header information, global information, and/or video packet information in the video frame;
- a spatial data error concealment module obtains motion vectors of undamaged macroblocks surrounding the damaged macroblock, wherein the spatial error concealment module estimates a motion vector of the damaged macroblock using the motion vectors of undamaged macroblocks, wherein the spatial data error concealment module reconstructs the damaged macroblock using the estimated damaged macroblock, wherein the spatial data error concealment module estimates the motion vector of the damaged macroblock using the motion vectors of the undamaged macroblocks located in about two rows that are substantially adjacent and below the damaged macroblock, and wherein reconstruction of the damaged macroblock by estimating the motion vector of the damaged macroblock comprises:
- estimating the motion vector of the damaged macroblock in the current video frame;
- estimating a motion vector of a macroblock located substantially adjacent and above the damaged macroblock;
- estimating a motion vector of a macroblock located substantially below the damaged macroblock;
- and
- estimating a motion vector of a macroblock located substantially adjacent, above, and left of the damaged macroblock;
- and
- copying the reconstructed damaged macroblock to conceal the error in the damaged macroblock.
7 more elements
An article comprising:
- a storage medium encoded with a computer program that, when executed by a computing platform, result in execution of a method comprising:
- detecting a channel error by locating a damaged macroblock (Px, y) in multiple macroblocks of a video frame using header information;
- estimating the damaged macroblock by using undamaged macroblocks surrounding the damaged macroblock in the video frame wherein the undamaged macroblocks include 1 to 4 undamaged macroblocks (Px, y−1, Px+1, y, Px, y+1, and Px−1, y), and wherein estimating the damaged macroblock includes estimating the damaged macroblock by using a weighted linear interpolation of the undamaged macroblocks surrounding the damaged macroblock;
- and
- replacing the damaged macroblock with the estimated damaged macroblock to conceal the error in the damaged macroblock.
2 more elements
A system comprising:
- a bus;
- a processor coupled to the bus;
- a memory coupled to the processor;
- a network interface coupled to the processor and the memory;
- and
- a video decoder coupled to the network interface comprising:
- a header decoding module parses a video frame to get header information and multiple macroblocks;
- an error recovery module coupled to the header decoding module detects a channel error by locating a damaged macroblock (Px, y) in the multiple macroblocks using the header information;
- and
- a spatial data error concealment module coupled to the error recovery module estimates the damaged macroblock by using undamaged macroblocks surrounding the damaged macroblock and replaces the damaged macroblock with the estimated damaged macroblock to conceal the channel error in the damaged macroblock, wherein the undamaged macroblocks include 1 to 4 undamaged macroblocks (Px, y−1, Px+1, y, Px, y+1, and Px−1, y), and wherein the spatial data error concealment module estimates the damaged macroblock by using a weighted linear interpolation of the undamaged macroblocks surrounding the damaged macroblock.
7 more elements
Dependencies are inferred from the claim text.