# US 11,310,501 B2 · Efficient use of quantization parameters in machine-learning models for video coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,310,501 B2
- Document ID: US11310501B2
- Status: Granted
- Kind: B2
- Application number: 16/868,729
- Filed: 2020-05-07
- Published: 2020-08-27
- Granted: 2022-04-19
- Earliest claimed: 2018-09-18
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Claudionor Coelho; Dake He; Aki Kuusela; Shan Li
- Web page: https://patentgrep.com/patent/US11310501B2

## CPC

- H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
- H04N 19/124: Quantisation
- G06N 3/048: Activation functions
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06T 9/002: {using neural networks}
- H04N 19/126: Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
- H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567)
- H04N 19/164: Feedback from the receiver or from the transmission channel
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/96: Tree coding, e.g. quad-tree coding
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Encoding an image block using a quantization parameter includes presenting, to an encoder that includes a machine-learning model, the image block and a value derived from the quantization parameter, where the value is a result of a non-linear function using the quantization parameter as input, where the non-linear function relates to a second function used to calculate, using the quantization parameter, a Lagrange multiplier that is used in a rate-distortion calculation, and where the machine-learning model is trained to output mode decision parameters for encoding the image block; obtaining the mode decision parameters from the encoder; and encoding, in a compressed bitstream, the image block using the mode decision parameters.

## Claims

### Claim 1

A method for encoding an image block using a quantization parameter, comprising:
presenting, to an encoder that includes a machine-learning model, the image block and a value derived from the quantization parameter,
wherein the value is a result of a non-linear function that uses the quantization parameter as input,
wherein the non-linear function relates to a second function used to calculate, using the quantization parameter, a Lagrange multiplier that is used in a rate-distortion calculation,
wherein the machine-learning model is trained to output mode decision parameters for encoding the image block, and
wherein each training datum used to train the machine-learning model includes:
a training block;
training mode decision parameters for encoding the training block;
a training value derived from a training quantization parameter;
first samples of a top neighboring block of the image block; and
second samples of a left neighboring block of the image block;
obtaining the mode decision parameters from the machine-learning model; and
encoding, in a compressed bitstream, the image block using the mode decision parameters.

### Claim 2 (depends on 1)

The method of claim 1, wherein the image block is a block of an intra-predicted image.

### Claim 3 (depends on 1)

The method of claim 1, wherein the mode decision parameters are indicative of a quad-tree partitioning of the image block.

### Claim 4 (depends on 1)

The method of claim 1, wherein the machine-learning model is a convolutional neural-network model.

### Claim 5 (depends on 1)

The method of claim 1, wherein the non-linear function is approximated by linear segments.

### Claim 6 (depends on 1)

The method of claim 1, wherein the non-linear function is an exponential function of the quantization parameter.

### Claim 7 (depends on 6)

The method of claim 6, wherein the exponential function is of a form cQP, where c is a constant and QP is the quantization parameter.

### Claim 8 (depends on 1)

The method of claim 1, wherein the non-linear function is a quadratic function of the quantization parameter.

### Claim 9 (depends on 8)

The method of claim 8, wherein the quadratic function is of a form QPα, where QP is the quantization parameter and α is a positive integer that is greater or equal to 2.

### Claim 10 (depends on 1)

The method of claim 1, wherein the non-linear function is of a same type as the second function.

### Claim 11

An apparatus for encoding an image block using a quantization parameter, comprising:
a processor that:
calculates a non-linear value of the quantization parameter using the quantization parameter as an input to a non-linear function, wherein the non-linear function is similar to a function of the quantization parameter that is used to calculate a Lagrange multiplier that is used in a rate-distortion calculation;
presents, to a machine-learning model, the image block and the non-linear value, wherein the machine-learning model is trained to output mode decision parameters for encoding the image block;
obtains the mode decision parameters from the machine-learning model; and
encodes, in a compressed bitstream, the image block using the mode decision parameters indicative of a quad-tree partitioning of the image block.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the image block is a block of an intra-predicted image.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein each training datum used to train the machine-learning model includes:
a training block;
training mode decision parameters for encoding the training block;
a training value derived from a training quantization parameter;
first samples of a top neighboring block of the image block; and
second samples of a left neighboring block of the image block.

### Claim 14 (depends on 11)

The apparatus of claim 11, wherein the non-linear function is approximated by linear segments.

### Claim 15 (depends on 11)

The apparatus of claim 11, wherein the non-linear function is an exponential function of a form cQP, where c is a constant and QP is the quantization parameter.

### Claim 16 (depends on 11)

The apparatus of claim 11, wherein the non-linear function being a quadratic function of a form QPα, where QP is the quantization parameter and α is a positive integer that is greater or equal to 2.

### Claim 17

A method for decoding an image block, comprising:
receiving, in a compressed bitstream, an indication of a quad-tree partitioning of the image block into sub-blocks, wherein an encoder determined the quad-tree partitioning of the image block using a machine-learning model that is trained by:
using training data as inputs to train the machine-learning model, each training datum includes
a training block,
mode decision parameters used for encoding the training block, and
a non-linear value that is derived by using a quantization parameter as input to a non-linear function,
wherein the quantization parameter is used for encoding the training block,
and
wherein the non-linear function relates to a function used to calculate, using the quantization parameter, a multiplier that is used in a rate-distortion calculation; and
decoding the image block using the indication of the quad-tree partitioning of the image block.

### Claim 18 (depends on 17)

The method of claim 17, wherein the non-linear function is of a same type as the function, and the function is used by the encoder for determining the multiplier used in rate-distortion calculation.