Level decision in rate distortion optimized quantization

A computing device that encodes video data, the computing device comprising one or more processors that are configured to:

• determine an initial quantized level for a coefficient of a coefficient block;
• determine whether the coefficient is less than the product of the initial quantized level and a quantization step size value;
• and
• in response to determining that the coefficient is less than the product of the initial quantized level and the quantization step size value:
• determine rate-distortion costs of quantizing the coefficient to be the initial quantized level and the initial quantized level minus one;
• and
• determine an actual quantized level for the coefficient based on the calculated rate-distortion costs.

A computing device comprising:

• means for determining an initial quantized level for a coefficient of a coefficient block;
• means for determining whether the coefficient is less than the product of the initial quantized level and a quantization step size value;
• means for determining, in response to determining that the coefficient is less than the product of the initial quantized level and the quantization step size value, rate-distortion costs of quantizing the coefficient to be the initial quantized level and the initial quantized level minus one;
• and
• means for determining, in response to determining that the coefficient is less than the product of the initial quantized level and the quantization step size value, an actual quantized level for the coefficient based on the calculated rate-distortion costs.

A computer-readable storage medium that stores instructions that, when executed by one or more processors of a computing device, configure the computing device to:

• determine an initial quantized level for a coefficient of a coefficient block;
• determine whether the coefficient is less than the product of the initial quantized level and a quantization step size value;
• and
• in response to determining that the coefficient is less than the product of the initial quantized level and the quantization step size value:
• determine rate-distortion costs of quantizing the coefficient to be the initial quantized level and the initial quantized level minus one;
• and
• determine an actual quantized level for the coefficient based on the calculated rate-distortion costs.