# US 9,438,910 B1 · Affine motion prediction in video coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,438,910 B1
- Document ID: US9438910B1
- Status: Granted
- Kind: B1
- Application number: 14/204,100
- Filed: 2014-03-11
- Granted: 2016-09-06
- Earliest claimed: 2014-03-11
- Assignees: Google Inc.
- Applicants: Google Inc.
- Inventors: Jingning Han; Hui Su; Yaowu Xu
- Web page: https://patentgrep.com/patent/US9438910B1

## CPC

- H04N 19/52: by predictive encoding
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/124: Quantisation
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/184: the unit being bits, e.g. of the compressed video stream
- H04N 19/54: using feature points or meshes
- H04N 19/61: in combination with predictive coding
- H04N 19/615: using motion compensated temporal filtering [MCTF]
- H04N 19/91: Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A block of a video frame may be predicted and encoded using a local affine motion prediction scheme. At least three points within the video frame are selected, each point being adjacent to a respective corner of the current block to be encoded. For each point, a motion search is performed for a template block centered about the point within a reference frame to find a template reference block. The template reference block has a center point that forms an anchor point pair with the corresponding point adjacent to the current block. Using coordinates of the anchor point pairs, affine coefficients for an affine transformation are generated. Using the affine transformation, a prediction block for the current block is generated and used to encode the current block within a bitstream. Providing the local affine motion prediction scheme can improve the prediction of certain blocks with complex motion, increasing compression.

## Claims

### Claim 1

A method of encoding a video frame, comprising:
selecting at least three points within the video frame, each point of the at least three points adjacent to a respective corner of a current block to be encoded, the current block being one of a plurality of blocks of the video frame;
for each point of the at least three points, performing a motion search for a template block centered about the point within a reference frame to find a template reference block, the template reference block having a center point forming an anchor point pair with the point of the at least three points;
using coordinates of the anchor point pairs, generating affine coefficients for an affine transformation;
using the affine transformation, generating a prediction block for the current block; and
encoding, using a processor, the current block within an encoded bitstream using the prediction block.

### Claim 2 (depends on 1)

The method of claim 1 wherein each template block comprises a square block in contact with a respective corner of the current block and aligned with at least one edge of the current block.

### Claim 3 (depends on 1)

The method of claim 1 wherein encoding the current block comprises:
generating a residual block as a difference between pixel values of the current block and pixel values of the prediction block;
applying a transform to the residual block to generate a plurality of transform coefficients;
quantizing the plurality of transform coefficients to generate a plurality of quantized transform coefficients; and
entropy coding the plurality of quantized transform coefficients for inclusion within the encoded bitstream.

### Claim 4 (depends on 3)

The method of claim 3 wherein applying the transform to the residual block comprises applying a two-dimensional Asymmetrical Discrete Sine Transform to the residual block.

### Claim 5 (depends on 1)

The method of claim 1 wherein generating the affine coefficients comprises solving six linear equations using the coordinates of only three anchor point pairs.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
selecting the only three anchor point pairs by omitting a fourth point of the at least three points whose template reference block results in a highest prediction error as a result of the motion search.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
calculating a prediction error for each template block based on the template reference block found for the template block in the motion search; and
selecting three points of the at least three points based on the template blocks having the lowest prediction errors; and wherein generating the affine coefficients comprises solving six linear equations using the coordinates of three anchor point pairs including the three points.

### Claim 8 (depends on 1)

The method of claim 1 wherein:
selecting the at least three points within the video frame comprises selecting four points;
generating the affine coefficients for the affine transformation comprises generating affine coefficients for different combinations of the anchor point pairs based on the four points to produce a plurality of affine transformations;
generating the prediction block for the current block comprises generating a plurality of prediction blocks, each prediction block generate using one of the plurality of affine transformations; and
encoding the current block using the prediction block comprises encoding the current block using one of the plurality of prediction blocks.

### Claim 9 (depends on 8)

The method of claim 8 wherein encoding the current block using one of the plurality of prediction blocks comprises encoding the current block using the one of the plurality of prediction blocks resulting in a lowest rate-distortion value.

### Claim 10 (depends on 1)

The method of claim 1 wherein encoding the current block using the prediction block comprises encoding a residual of the current block and the prediction block using a two-dimensional Asymmetrical Discrete Sine Transform.

### Claim 11

An apparatus for encoding a video frame, comprising:
a memory; and
a processor configured to execute instructions stored in the memory to:
select at least three points within the video frame, each point of the at least three points adjacent to a respective corner of a current block to be encoded, the current block being one of a plurality of blocks of the video frame;
for each point of the at least three points, perform a motion search for a template block centered about the point within a reference frame to find a template reference block, the template reference block having a center point forming an anchor point pair with the point of the at least three points;
using coordinates of the anchor point pairs, generate affine coefficients for an affine transformation;
using the affine transformation, generate a prediction block for the current block; and
encode the current block within an encoded bitstream using the prediction block.

### Claim 12 (depends on 11)

The apparatus of claim 11 wherein each template block comprises a square block in contact with a respective corner of the current block and aligned with at least one edge of the current block.

### Claim 13 (depends on 11)

The apparatus of claim 11 wherein the processor is configured to encode the current block by:
generating a residual block as a difference between pixel values of the current block and pixel values of the prediction block;
applying a transform to the residual block to generate a plurality of transform coefficients;
quantizing the plurality of transform coefficients to generate a plurality of quantized transform coefficients; and
entropy coding the plurality of quantized transform coefficients for inclusion within the encoded bitstream.

### Claim 14 (depends on 13)

The apparatus of claim 13 wherein the processor is configured to apply the transform to the residual block by applying a two-dimensional Asymmetrical Discrete Sine Transform to the residual block.

### Claim 15 (depends on 11)

The apparatus of claim 11 wherein the processor is configured to generate a single set of the affine coefficients by solving six linear equations using the coordinates of only three anchor point pairs.

### Claim 16 (depends on 15)

The apparatus of claim 15 wherein the processor is configured to select the only three anchor point pairs to generate the single set by:
omitting a fourth point of the at least three points whose template reference block results in a highest prediction error as a result of the motion search.

### Claim 17 (depends on 11)

The apparatus of claim 11 wherein the processor is configured to:
calculate a prediction error for each template block based on the template reference block found for the template block in the motion search;
select three points of the at least three points based on the template blocks having the lowest prediction errors; and
generate the affine coefficients by solving six linear equations using the coordinates of three anchor point pairs including the three points.

### Claim 18 (depends on 11)

The apparatus of claim 11 wherein the processor is configured to:
select the at least three points within the video frame by selecting four points;
generate the affine coefficients for the affine transformation by generating affine coefficients for different combinations of the anchor point pairs based on the four points to produce a plurality of affine transformations;
generate the prediction block for the current block by generating a plurality of prediction blocks, each prediction block generate using one of the plurality of affine transformations; and
encode the current block using the prediction block by encoding the current block using one of the plurality of prediction blocks.

### Claim 19 (depends on 18)

The apparatus of claim 18 wherein the processor is configured to encode the current block using one of the plurality of prediction blocks by encoding the current block using the one of the plurality of prediction blocks resulting in a lowest rate-distortion value.

### Claim 20 (depends on 19)

The apparatus of claim 19 wherein the processor is configured to encode the current block by encoding a residual of the current block and the one of the plurality of prediction blocks using a two-dimensional Asymmetrical Discrete Sine Transform.