US 5,970,173 AGranted

Image compression and affine transformation for image motion compensation

AssigneeMicrosoft Corporation

InventorsMing-Chieh Lee, Wei-ge Chen

Life of this patent

Estimating the term…

Loading patent…

Abstract

A transformation method provides a multi-dimensional affine transformation for representing motion between corresponding image components of successive video image frames. The multi-dimensional affine transformations can represent complex motion that includes any or all of translation, rotation, magnification, and shear. The transformation method of this invention includes determining motion transformations between corresponding pixels in the image components of the first and second video image frames. From the motion transformations between corresponding pixels, multi-dimensional affine motion transformations between the corresponding image components are determined. This transformation method increases the accuracy with which complex motion is represented and results in fewer compression or encoding errors in comparison to conventional methods, thereby increasing compression efficiency.

Claims32

Dependent claims

In a method of encoding, in a compressed format, information within a video image frame sequence having first and second video image frames, a method of determining quantized multi-dimensional motion transformations between corresponding image components of the first and second video image frames, comprising:

determining multi-dimensional affine motion transformations between representations of the corresponding image components on the first and second video image frames;
and
quantizing the multi-dimensional affine motion transformations between the corresponding image components, wherein the quantizing step includes for each of the components;
selecting reference pixel coordinates within each component in the second video image frame, including selecting a number of reference pixel coordinates to encode motion for an image component depending on complexity of motion of pixels in the image component, wherein two reference pixel coordinates are encoded for rotation and magnification;
and three reference pixel coordinates are encoded for shear;
applying a multi-dimensional affine motion transformation to the selected reference coordinates within each component to find corresponding pixel coordinates in the first video image frame;
and
encoding for transmission or storage the reference pixel coordinates and the relative positions of the corresponding pixel coordinates so that a motion transformation can be derived from the reference pixel coordinates and the corresponding pixel coordinates during decoding operations;
wherein the encoding step includes independently encoding image components using the selected number of reference pixels such that the number of reference pixels encoded per image component vary depending on the complexity of the motion of the pixels within the image component.

6 more elements

In a method of encoding, in a compressed format, information within a video sequence having first and second video image frames, each frame including an arbitrarily-shaped object therein, an improvement comprising:

(a) identifying a plurality of multi-pixel image components in the first and second frames which encompass the arbitrarily-shaped object;
(b) performing dense motion estimation processes to generate a plurality of dense motion vectors for each of said plural multi-pixel image components in the second frame, said plurality of dense motion vectors representing motion of individual pixels in said multi-pixel image components between the first and second video image frames;
and
(c) from said dense motion vectors, determining multi-dimensional motion transformations between the first and second video image frames for each of said plural multi-pixel image components in the second frame;
(d) selecting reference pixel coordinates for the multi-pixel image components in the second frame, including selecting a number of reference pixels to encode motion for an image component depending on complexity of motion of pixels in the image component, wherein two reference pixels are encoded for rotation and magnification;
and three reference pixels are encoded for shear;
(e) applying a multi-dimensional motion transformation to the selected reference coordinates to find corresponding pixel coordinates in the first frame;
and
(f) encoding for transmission or storage the reference pixel coordinates and relative positions of the corresponding pixel coordinates so that transform coefficients can be derived from the reference pixel coordinates and the corresponding pixel coordinates during decoding operations;
wherein the encoding seep includes independently encoding image components using the selected number of reference pixels such that the number of reference pixels encoded per image component vary depending on the complexity of the motion of the pixels within the image component.

7 more elements

In a method of encoding affine transformation data relating to motion of pixels between first and second image frames, an improvement comprising:

deriving affine coefficients that approximate motion of pixels within a transformation block for each block in a group of transformation blocks of the second image frame;
selecting reference pixel coordinates in each of the transformation blocks of the second image frame, including selecting a number of reference pixel coordinates to encode motion for an image component depending on complexity of motion of pixels in the image component;
transforming the selected reference pixel coordinates for each of the transformation blocks with the derived affine coefficients for the block to find corresponding pixel coordinates in the first video image frame for each of the blocks;
and
representing the affine coefficients of the transformation blocks of the second image frame by encoding coordinate data of the reference pixels in each of the transformation blocks and encoding coordinate data of corresponding pixel coordinates of each of the transformation blocks, including converting the coordinate data to integer format, wherein truncation errors associated with representation of the affine coefficients are avoided and the affine coefficients are quantized by encoding the coefficients with the converted pixel coordinate data from which the coefficients can be derived during decoding operations
wherein the encoding step includes independently encoding image components using the selected number of reference pixels such that the number of reference pixels encoded per image component vary depending on the complexity of the motion of the pixels within the image component.

3 more elements

determining multi-dimensional affine motion transformations between representations of the corresponding image components in the first and second video image frames;
quantizing the multi-dimensional affine motion transformations between the corresponding image components, including:
a) selecting reference pixel coordinates in the second video image frame;
b) applying a multi-dimensional affine motion transformation to the selected reference coordinates to find corresponding pixel coordinates in the first video image frame;
and
c) encoding for transmission or storage the reference pixel coordinates and relative positions of the corresponding pixel coordinates so that transform coefficients can be derived from the reference pixel coordinates and the corresponding pixel coordinates during decoding operations;
and the method further including:
quantizing the multi-dimensional affine motion transformations between the corresponding image components selectively according to the dimensions of the multi-dimensional affine motion transformations;
selecting a number of reference pixels to encode motion for an image component depending on complexity of motion of pixels in the image component, wherein only one reference pixel is encoded for translation, two reference pixels are encoded for rotation and magnification;
and three reference pixels are encoded for shear;
and
independently encoding image components using the selected number of reference pixels such that the number of reference pixels encoded per image component vary depending on the complexity of the motion of the pixels within the image component.

9 more elements

(a) identifying a plurality of multi-pixel image components in the first and second frames which encompass the arbitrarily-shaped object;
(b) performing dense motion estimation processes to generate a plurality of dense motion vectors for each of said plural multi-pixel image components in the second frame, said plurality of dense motion vectors representing motion of individual pixels in said multi-pixel image components between the first and second video image frames;
(c) from said dense motion vectors, determining multi-dimensional motion transformations between the first and second video image frames for each of said plural multi-pixel image components in the second frame;
(d) selecting reference pixel coordinates for the multi-pixel image components in the second frame;
(e) applying a multi-dimensional motion transformation to the selected reference coordinates to find corresponding pixel coordinates in the first frame;
and
(f) encoding for transmission or storage the reference pixel coordinates and relative positions of the corresponding pixel coordinates so that transform coefficients can be derived from the reference pixel coordinates and the corresponding pixel coordinates during decoding operations;
(g) representing the motion transformations by specifying locations of at least two pixels in the first image frame, and at least two corresponding pixels in the second image frame, wherein errors associated with truncation of transformation coefficients are avoided;
(h) representing said motion transformations selectively according to the dimensions of the multi-dimensional motion transformations;
(i) selecting a number of reference pixels to encode motion for an image component depending on complexity of motion of pixels in the image component, wherein two reference pixels are encoded for rotation and magnification;
and three reference pixels are encoded for shear;
and
(j) independently encoding image components using the selected number of reference pixels such that the number of reference pixels encoded per image component vary depending on the complexity of the motion of the pixels within the image component.

10 more elements

Dependencies are inferred from the claim text.