Architecture and method for fine granularity scalable video coding

An apparatus for robust fine granularity scalability video encoding, comprising:

• a base layer encoder receiving an original video image and a base layer reference image for generating a base layer bit stream, wherein said base layer encoder includes:
• a first motion estimator receiving said original video image and said base layer reference image for estimating a motion vector;
• a first motion compensator receiving said base layer reference image and said motion vector for generating a base layer predicted image;
• a first subtraction unit subtracting said base layer predicted image from said original video image and generating a motion compensated base layer frame difference image;
• a first discrete cosine transform unit transforming said motion compensated base layer frame difference image into motion compensated base layer difference coefficients;
• a quantizer quantizing said motion compensated base layer difference coefficients into quantized base layer difference coefficients;
• a first variable length coding unit receiving said motion vector and said quantized base layer difference coefficients for generating said base layer bit stream;
• a dequantizer receiving said quantized base layer difference coefficients and generating said coded base layer discrete cosine transform coefficients;
• a first inverse discrete cosine transform unit receiving said coded base layer discrete cosine transform coefficients and generating a decoded base layer difference image;
• a first summation unit adding said decoded base layer difference image to said base layer predicted image and generating an unclipped base layer reconstructed image;
• and
• a first clipping unit receiving said unclipped base layer reconstructed image and generating said base layer reconstructed image;
• and
• a base layer frame buffer for buffering said base layer reconstructed image;
• an enhancement layer encoder receiving a video image, a motion estimation vector, a base layer reconstructed image, and coded base layer discrete cosine transform coefficients for generating an enhancement layer bit stream, wherein said enhancement layer encoder includes:
• a second motion compensator receiving a motion vector and a buffered enhancement layer reconstructed image for generating an enhancement layer predicted image;
• a second subtraction unit subtracting said enhancement layer predicted image from a video image and generating a motion compensated enhancement layer difference image;
• a second discrete cosine transform unit transforming said motion compensated enhancement layer difference image into motion compensated enhancement layer difference coefficients;
• a third subtraction unit subtracting said coded base layer discrete cosine transform coefficients or interpolated base layer discrete cosine transform coefficients from said motion compensated enhancement layer difference coefficients and generating first residue coefficients;
• a fourth subtraction unit subtracting said coded base layer discrete cosine transform coefficients or said interpolated base layer discrete cosine transform coefficients from said motion compensated base layer difference coefficients and generating second residue coefficients;
• a bitplane coding unit receiving said first or second residue coefficients and generating enhancement layer difference coefficients;
• a second variable length coding unit receiving said enhancement layer difference coefficients and generating an enhancement layer bit stream, said variable length coding unit having an input for receiving an enhancement layer motion vector;
• a second summation unit adding said coded base layer discrete cosine transform coefficients or said interpolated base layer discrete cosine transform coefficients to first β bitplanes of said enhancement layer difference coefficients for generating enhancement layer partial difference coefficients;
• a second inverse discrete cosine transform unit transforming said enhancement layer partial difference coefficients into an enhancement layer partial difference image;
• a third summation unit adding said enhancement layer partial difference image to said enhancement layer predicted image and generating an unclipped enhancement layer partially reconstructed image;
• a second clipping unit receiving said unclipped enhancement layer partially reconstructed image and generating an enhancement layer partially reconstructed image;
• a fifth subtraction unit subtracting said base layer reconstructed image or an interpolated base layer reconstructed image from said enhancement layer partially reconstructed image and generating an enhancement layer reconstructed difference image;
• a first scaler scaling said enhancement layer reconstructed difference image by a factor α and generating a scaled enhancement layer reconstructed difference image;
• a fourth summation unit adding said base layer reconstructed image or said interpolated base layer reconstructed image to said scaled enhancement layer reconstructed difference image and generating an enhancement layer reconstructed image;
• and
• an enhancement layer frame buffer for buffering said enhancement layer reconstructed image;
• and an encoder spatial scalability module having a plurality of decimators, a plurality of interpolators and a plurality of switches;
• wherein said base layer reference image is generated from said enhancement layer encoder or said base layer encoder, said base layer reconstructed image is generated from said base layer encoder, and said encoder spatial scalability module controls if said base layer reference image is decimated or not when said base layer reference image is generated from said enhancement layer encoder and controls if said base layer reconstructed image is interpolated or not.