# US 2024/0267529 A1 · Context-aware quantization for high-performance video encoding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0267529 A1
- Document ID: US20240267529A1
- Status: Published application
- Kind: A1
- Application number: 18/107,295
- Filed: 2023-02-08
- Published: 2024-08-08
- Earliest claimed: 2023-02-08
- Assignees: NVIDIA Corporation
- Applicants: NVIDIA Corporation
- Inventors: Jianjun Chen; Junan Chen; Yonghai Wu; Yongmao Tang; Xinan Lu
- Web page: https://patentgrep.com/patent/US20240267529A1

## CPC

- H04N 19/14: Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence)
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567)
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/18: the unit being a set of transform coefficients
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Disclosed are apparatuses, systems, and techniques for efficient real-time codec encoding of video files. In one embodiment, the techniques include generating a block of predicted pixels that approximates a block of source pixels of an image frame and representing a difference between the block of source pixels and the block of predicted pixels via a plurality of transformation coefficients (TCs). The techniques further include evaluating TCs using statistical data for neighborhoods of the TCs to select an action for a respective TC, including adjusting the respective TC or maintaining the respective TC.

## Claims

### Claim 1

A method comprising:
generating a block of predicted pixels that approximates a block of source pixels of an image frame;
representing a difference between the block of source pixels and the block of predicted pixels via a plurality of transformation coefficients (TCs);
evaluating each TC of at least a subset of the plurality of TCs using statistical data for a neighborhood of TCs associated with a respective TC, to select an action that is performed on the respective TC, the action comprising:
adjusting the respective TC, or
maintaining the respective TC; and
generating a compressed representation of the image frame using the evaluated subset of the plurality of TCs.

### Claim 2 (depends on 1)

The method of claim 1, wherein the block of predicted pixels comprises pixels predicted using at least one of:
reference pixels of one or more reference image frames different from the image frame, or
reconstructed pixels of the image frame.

### Claim 3 (depends on 1)

The method of claim 1, wherein representing the difference between the block of source pixels and the block of predicted pixels via the plurality of TCs comprises applying a discrete linear transformation to the difference.

### Claim 4 (depends on 1)

The method of claim 1, wherein adjusting the respective TC is responsive to a first cost value associated with an adjustment of the respective TC being less than a second cost value associated with maintaining the respective TC.

### Claim 5 (depends on 1)

The method of claim 1, wherein evaluating the respective TC comprises:
accessing the statistical data for each TC of the neighborhood of TCs;
virtually replacing one or more TCs of the neighborhood of TCs with TCs based at least on the statistical data; and
evaluating the respective TC using a cost function computed with the virtually replaced one or more TCs of the neighborhood of TCs.

### Claim 6 (depends on 5)

The method of claim 5, wherein virtually replacing the one or more TCs of the neighborhood of TCs is with a most likely historical TC modification of the one or more TCs of the neighborhood of TCs indicated by the statistical data.

### Claim 7 (depends on 1)

The method of claim 1, wherein adjusting the respective TC comprises at least one of:
decrementing the respective TC; or
replacing the respective TC with a zero value.

### Claim 8 (depends on 1)

The method of claim 1, wherein two or more of the subset of the plurality of TCs are evaluated in parallel.

### Claim 9

A system comprising:
a memory device to store a block of source pixel values of an image frame; and
one or more circuits communicatively coupled to the memory device, the one or more circuits to:
generate a block of predicted pixels that approximates the block of source pixels of the image frame;
represent a difference between the block of source pixels and the block of predicted pixels via a plurality of transformation coefficients (TCs);
evaluate each TC of at least a subset of the plurality of TCs using statistical data for a neighborhood of TCs associated with a respective TC, to select an action that is performed on the respective TC, wherein the action comprises:
adjusting of the respective TC, or
maintaining the respective TC; and
generate a compressed representation of the image frame using the evaluated subset of the plurality of TCs.

### Claim 10 (depends on 9)

The system of claim 9, wherein the block of predicted pixels comprises pixels predicted using at least one of:
reference pixels of one or more reference image frames different from the image frame, or
reconstructed pixels of the image frame.

### Claim 11 (depends on 9)

The system of claim 9, wherein to represent the difference between the block of source pixels and the block of predicted pixels via the plurality of TCs, the one or more circuits are to apply a discrete linear transformation to the difference.

### Claim 12 (depends on 9)

The system of claim 9, wherein adjusting the respective TC is responsive to a first cost value associated with an adjustment of the respective TC being less than a second cost value associated with maintaining the respective TC.

### Claim 13 (depends on 9)

The system of claim 9, wherein to evaluate the respective TC, the one or more circuits are to:
access the statistical data for each TC of the neighborhood of TCs;
virtually replace one or more TCs of the neighborhood of TCs with TCs based at least on the statistical data; and
evaluate the respective TC using a cost function computed with the virtually replaced one or more TCs of the neighborhood of TCs.

### Claim 14 (depends on 13)

The system of claim 13, wherein to virtually replace the one or more TCs of the neighborhood of TCs, the one or more circuits are to select a most likely historical TC modification of the one or more TCs of the neighborhood of TCs indicated by the statistical data.

### Claim 15 (depends on 9)

The system of claim 9, wherein adjusting the respective TC comprises at least one of:
decrementing the respective TC; or
replacing the respective TC with a zero value.

### Claim 16 (depends on 9)

The system of claim 9, wherein two or more of the subset of the plurality of TCs are evaluated in parallel.

### Claim 17

A system comprising:
a memory device to store a block of source pixel values of an image frame; and
one or more circuit groups communicatively coupled to the memory device, the one or more circuit groups comprising:
a first circuit group to:
generate a block of predicted pixels that approximates the block of source pixels of the image frame; and
a second circuit group communicatively coupled to the first circuit group, the second circuit group to:
represent a difference between the block of source pixels and the block of predicted pixels via a plurality of transformation coefficients (TCs);
evaluate each TC of at least a subset of the plurality of TCs using statistical data for a neighborhood of TCs associated with a respective TC, to select an action that is performed on the respective TC, wherein the action comprises:
adjusting of the respective TC, or
maintaining the respective TC; and
generate a compressed representation of the image frame using the evaluated subset of the plurality of TCs.

### Claim 18 (depends on 17)

The system of claim 17, wherein to evaluate the respective TC, the second circuit group is to:
access the statistical data for each TC of the neighborhood of TCs;
virtually replace one or more TCs of the neighborhood of TCs with TCs based at least on the statistical data; and
evaluate the respective TC using a cost function computed with the virtually replaced one or more TCs of the neighborhood of TCs.

### Claim 19 (depends on 17)

The system of claim 17, wherein adjusting the respective TC comprises at least one of:
decrementing the respective TC; or
replacing the respective TC with a zero value.

### Claim 20 (depends on 17)

The system of claim 17, wherein the second circuit group is to evaluate two or more of the subset of the plurality of TCs in parallel.
