US 10,652,581 B1 · Entropy coding in image and video compression using machine learning

Abstract

Machine learning is used to refine a probability distribution for entropy coding video or image data. A probability distribution is determined for symbols associated with a video block (e.g., quantized transform coefficients, such as during encoding, or syntax elements from a bitstream, such as during decoding), and a set of features is extracted from video data associated with the video block and/or neighbor blocks. The probability distribution and the set of features are then processed using machine learning to produce a refined probability distribution. The video data associated with a video block are entropy coded according to the refined probability distribution. Using machine learning to refine the probability distribution for entropy coding minimizes the cross-entropy loss between the symbols to entropy code and the refined probability distribution.

Claims13

Dependent claims

A method for decoding an encoded video block from a bitstream, the method comprising:

deriving syntax elements representative of the encoded video block from the bitstream;
determining a probability distribution for the syntax elements;
identifying, based on one or both of the syntax elements or other video data associated with the encoded video block, differences in pixel values of the encoded video block along one or more different directions;
computing a normalized histogram of oriented gradients based on the one or more different directions of the differences in pixel values;
producing a refined probability distribution by updating the probability distribution based on the normalized histogram of oriented gradients;
entropy decoding the syntax elements according to the refined probability distribution to produce quantized transform coefficients;
dequantizing the quantized transform coefficients to produce transform coefficients;
inverse transforming the transform coefficients to produce a prediction residual;
reconstructing the prediction residual to produce a decoded video block;
and
outputting the decoded video block to an output video stream.

8 more elements

An apparatus for decoding an encoded video block from a bitstream, the apparatus comprising:

a memory;
and
a processor configured to execute instructions stored in the memory to:
determine a probability distribution for syntax elements representative of the encoded video block, the syntax elements derived from the bitstream;
identify, based on one or both of the syntax elements or other video data associated with the encoded video block, differences in pixel values of the encoded video block along one or more different directions;
process the probability distribution and the differences in pixel values using machine learning to produce a refined probability distribution, including by:
computing a normalized histogram of oriented gradients based on the one or more different directions of the differences in pixel values;
and
updating the probability distribution based on the normalized histogram of oriented gradients;
entropy decode the syntax elements according to the refined probability distribution to produce quantized transform coefficients;
further decode the quantized transform coefficients to produce a decoded video block;
and
output the decoded video block to an output video stream.

10 more elements

Dependencies are inferred from the claim text.

Entropy coding in image and video compression using machine learning

Life of this patent

Entropy coding in image and video compression using machine learning

Life of this patent

Abstract

Claims13