# US 2009/0257489 A1 · Rate-distortion quantization for context-adaptive variable length coding (cavlc)

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2009/0257489 A1
- Document ID: US20090257489A1
- Status: Published application
- Kind: A1
- Application number: 12/338,838
- Filed: 2008-12-18
- Published: 2009-10-15
- Earliest claimed: 2008-04-11
- Assignees: QUALCOMM Incorporated
- Inventors: Marta Karczewicz; Peisong Chen; Yan Ye
- Web page: https://patentgrep.com/patent/US20090257489A1

## CPC

- H04N 19/124: Quantisation
- H04N 19/126: Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In general, this disclosure provides techniques for quantization of the coefficients of video blocks in a manner that can achieve a desirable balance of rate and distortion. The described techniques may analyze a plurality of quantization levels associated with each individual coefficient to select the quantization level for the individual coefficients that results in a lowest coding cost. Since CAVLC does not encode each coefficient independently, the techniques may compute the coding costs for each of the candidate quantization levels associated with the individual coefficients based on quantization levels selected for previously quantized coefficients and estimated (or predicted) quantization levels for subsequent coefficients of a coefficient vector. The quantization levels for each of the coefficients are selected based on computed coding costs to obtain a set of quantized coefficients that minimize a rate-distortion model.

## Claims

### Claim 1

A method of quantizing at least a portion of coefficients of a video block in a video coding process, the method comprising:
determining, for each of the portion of the coefficients of the video block, coding costs for a plurality of candidate quantization levels associated with the respective coefficient; and
selecting one of the plurality of the candidate quantization levels for each of the coefficients based on the coding costs to obtain a set of quantized coefficients.

### Claim 2 (depends on 1)

The method of claim 1,
wherein determining the coding costs comprises computing a coding cost for each of the candidate quantization levels associated with the respective coefficient based on the respective candidate quantization level, previously selected quantization levels associated with previously quantized coefficients and predicted quantization levels associated with subsequent non-quantized coefficients; and
wherein selecting the one of the plurality of candidate quantization levels comprises selecting, for the respective coefficients, the one of the plurality of candidate quantization levels corresponding with a lowest coding cost.

### Claim 3 (depends on 1)

The method of claim 1,
wherein determining the coding costs comprises computing a plurality of coding costs for each of the candidate quantization levels associated with the respective coefficient based on surviving paths associated with a previous coefficient, the respective candidate quantization level associated with a currently coefficient, and predicted quantization levels associated with subsequent non-quantized coefficients, wherein the surviving paths represent sequences of quantization decisions for coefficients preceding the current coefficient in the block;
wherein selecting the one of the plurality of candidate quantization levels comprises selecting a surviving path for each of the candidate quantization levels associated with the current coefficient corresponding to a lowest coding cost.

### Claim 4 (depends on 3)

The method of claim 3, further comprising selecting, for a last coefficient of the video block, one of the surviving paths with a lowest coding cost from the plurality of surviving paths of the candidate quantization levels associated with the last coefficient.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining the coding costs comprises computing coding costs using a Lagrangian cost function associated with a rate-distortion model.

### Claim 6 (depends on 1)

The method of claim 1, further comprising calculating at least one of the plurality of candidate quantization levels of respective individual coefficients based on an absolute value of the respective individual coefficient of the video block, a quantization parameter, and a quantization matrix.

### Claim 7 (depends on 6)

The method of claim 6, wherein the plurality of candidate quantization levels include quantization level zero.

### Claim 8 (depends on 7)

The method of claim 7, further comprising skipping determination of coding costs for the plurality of candidate quantization levels if a current one of the quantized coefficients is closer to the quantization level zero than quantization level one.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
performing context adaptive variable length coding (CAVLC) on the set of quantized coefficients to entropy code the set of quantized coefficients into a bitstream; and
outputting the encoded bitstream.

### Claim 10

A apparatus for quantizing at least a portion of coefficients of a video block in a video coding process, the apparatus comprising:
a quantization unit that:
determines, for each of the portion of the coefficients of the video block, coding costs for a plurality of candidate quantization levels associated with the respective coefficient; and
selects one of the plurality of the candidate quantization levels for each of the coefficients based on the coding costs to obtain a set of quantized coefficients.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the quantization unit:
computes a coding cost for each of the candidate quantization levels associated with the respective coefficient based on the respective candidate quantization level, previously selected quantization levels associated with previously quantized coefficients and predicted quantization levels associated with subsequent non-quantized coefficients; and
selects, for the respective coefficients, the one of the plurality of candidate quantization levels corresponding with a lowest coding cost.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein the quantization unit:
computes a plurality of coding costs for each of the candidate quantization levels associated with the respective coefficient based on surviving paths associated with a previous coefficient, the respective candidate quantization level associated with a currently coefficient, and predicted quantization levels associated with subsequent non-quantized coefficients, wherein the surviving paths represent sequences of quantization decisions for coefficients preceding the current coefficient in the block;
selects a surviving path for each of the candidate quantization levels associated with the current coefficient corresponding to a lowest coding cost.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the quantization unit selects, for a last coefficient of the video block, one of the surviving paths with a lowest coding cost from the plurality of surviving paths of the candidate quantization levels associated with the last coefficient.

### Claim 14 (depends on 10)

The apparatus of claim 10, wherein the quantization unit computes coding costs using a Lagrangian cost function associated with a rate-distortion model.

### Claim 15 (depends on 10)

The apparatus of claim 10, wherein the quantization unit calculates at least one of the plurality of candidate quantization levels of respective individual coefficients based on an absolute value of the respective individual coefficient of the video block, a quantization parameter, and a quantization matrix.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the plurality of candidate quantization levels include quantization level zero.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the quantization unit skips determination of coding costs for the plurality of candidate quantization levels if a current one of the quantized coefficients is closer to the quantization level zero than quantization level one.

### Claim 18 (depends on 10)

The apparatus of claim 10, further comprising:
an entropy encoder that performs context adaptive variable length coding (CAVLC) on the set of quantized coefficients to entropy code the set of quantized coefficients into a bitstream; and
a transmitter to transmit the encoded bitstream.

### Claim 19 (depends on 10)

The apparatus of claim 10, wherein the apparatus is incorporated within a wireless communication device.

### Claim 20 (depends on 10)

The apparatus of claim 10, wherein the apparatus comprises an integrated circuit.

### Claim 21

A computer-readable medium comprising instructions that upon execution cause a device to quantize at least a portion of coefficients of a video block in a video coding process, wherein the instructions cause the device to:
determine, for each of the portion of the coefficients of the video block, coding costs for a plurality of candidate quantization levels associated with the respective coefficient; and
select one of the plurality of the candidate quantization levels for each of the coefficients based on the coding costs to obtain a set of quantized coefficients.

### Claim 22 (depends on 21)

The computer-readable medium of claim 21,
wherein instructions that cause the device to determine the coding costs comprise instructions that cause the device to compute a coding cost for each of the candidate quantization levels associated with the respective coefficient based on the respective candidate quantization level, previously selected quantization levels associated with previously quantized coefficients and predicted quantization levels associated with subsequent non-quantized coefficients; and
wherein instructions that cause the device to select the one of the plurality of candidate quantization levels comprise instructions that cause the device to select, for the respective coefficients, the one of the plurality of candidate quantization levels corresponding with a lowest coding cost.

### Claim 23 (depends on 21)

The computer-readable medium of claim 21,
wherein instructions that cause the device to determine the coding costs comprises instructions that cause the device to compute a plurality of coding costs for each of the candidate quantization levels associated with the respective coefficient based on surviving paths associated with a previous coefficient, the respective candidate quantization level associated with a currently coefficient, and predicted quantization levels associated with subsequent non-quantized coefficients, wherein the surviving paths represent sequences of quantization decisions for coefficients preceding the current coefficient in the block;
wherein instructions that cause the device to select the one of the plurality of candidate quantization levels comprises instructions that cause the device to select a surviving path for each of the candidate quantization levels associated with the current coefficient corresponding to a lowest coding cost.

### Claim 24 (depends on 23)

The computer-readable medium of claim 23, further comprising instructions that cause the device to select, for a last coefficient of the video block, one of the surviving paths with a lowest coding cost from the plurality of surviving paths of the candidate quantization levels associated with the last coefficient.

### Claim 25 (depends on 21)

The computer-readable medium of claim 21, wherein instructions that cause the device to determine the coding costs comprises instructions that cause the device to compute coding costs using a Lagrangian cost function associated with a rate-distortion model.

### Claim 26 (depends on 21)

The computer-readable medium of claim 21, further comprising instructions that cause the device to calculate at least one of the plurality of candidate quantization levels of respective individual coefficients based on an absolute value of the respective individual coefficient of the video block, a quantization parameter, and a quantization matrix.

### Claim 27 (depends on 26)

The computer-readable medium of claim 26, wherein the plurality of candidate quantization levels include quantization level zero.

### Claim 28 (depends on 27)

The computer-readable medium of claim 27, further comprising instructions that cause the device to skip determination of coding costs for the plurality of candidate quantization levels if a current one of the quantized coefficients is closer to the quantization level zero than quantization level one.

### Claim 29 (depends on 21)

The computer-readable medium of claim 21, further comprising instructions that cause the device to:
perform context adaptive variable length coding (CAVLC) on the set of quantized coefficients to entropy code the set of quantized coefficients into a bitstream; and
output the encoded bitstream.

### Claim 30

A device for quantizing at least a portion of coefficients of a video block in a video coding process, the device comprising:
means for determining, for each of the portion of the coefficients of the video block, coding costs for a plurality of candidate quantization levels associated with the respective coefficient; and
means for selecting one of the plurality of the candidate quantization levels for each of the coefficients based on the coding costs to obtain a set of quantized coefficients.

### Claim 31 (depends on 30)

The device of claim 30,
wherein the determining means computes a coding cost for each of the candidate quantization levels associated with the respective coefficient based on the respective candidate quantization level, previously selected quantization levels associated with previously quantized coefficients and predicted quantization levels associated with subsequent non-quantized coefficients; and
wherein the selecting means selects, for the respective coefficients, the one of the plurality of candidate quantization levels corresponding with a lowest coding cost.

### Claim 32 (depends on 30)

The device of claim 30,
wherein the determining means computes a plurality of coding costs for each of the candidate quantization levels associated with the respective coefficient based on surviving paths associated with a previous coefficient, the respective candidate quantization level associated with a currently coefficient, and predicted quantization levels associated with subsequent non-quantized coefficients, wherein the surviving paths represent sequences of quantization decisions for coefficients preceding the current coefficient in the block;
wherein the selecting means selects a surviving path for each of the candidate quantization levels associated with the current coefficient corresponding to a lowest coding cost.

### Claim 33 (depends on 32)

The device of claim 32, wherein the selecting means selects, for a last coefficient of the video block, one of the surviving paths with a lowest coding cost from the plurality of surviving paths of the candidate quantization levels associated with the last coefficient.

### Claim 34 (depends on 30)

The device of claim 30, wherein the determining means computes coding costs using a Lagrangian cost function associated with a rate-distortion model.

### Claim 35 (depends on 30)

The device of claim 30, further comprising means for calculating at least one of the plurality of candidate quantization levels of respective individual coefficients based on an absolute value of the respective individual coefficient of the video block, a quantization parameter, and a quantization matrix.

### Claim 36 (depends on 35)

The device of claim 35, wherein the plurality of candidate quantization levels include quantization level zero.

### Claim 37 (depends on 36)

The device of claim 36, further comprising means for skipping determination of coding costs for the plurality of candidate quantization levels if a current one of the quantized coefficients is closer to the quantization level zero than quantization level one.

### Claim 38 (depends on 30)

The device of claim 30, further comprising:
means for performing context adaptive variable length coding (CAVLC) on the set of quantized coefficients to entropy code the set of quantized coefficients into a bitstream; and
means for outputting the encoded bitstream.
