# US 2023/0068408 A1 · Parallel processing of video frames during video encoding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0068408 A1
- Document ID: US20230068408A1
- Status: Published application
- Kind: A1
- Application number: 17/451,964
- Filed: 2021-10-22
- Published: 2023-03-02
- Earliest claimed: 2021-09-02
- Assignees: NVIDIA Corporation
- Applicants: NVIDIA Corporation
- Inventors: Jianjun Chen; Wei Feng; Xi He; Sangeun Han; Ranga Ramanujam Srinivasan
- Web page: https://patentgrep.com/patent/US20230068408A1

## CPC

- H04N 19/436: using parallelised computational arrangements
- H04N 19/103: Selection of coding mode or of prediction mode
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/14: Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence)
- H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567)
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Disclosed are techniques for compressing data of an image. Intermediate pixels may be determined. Each location of the image may be associated with a block of a plurality of blocks of a first size and a block of a plurality of blocks of a second size. For each block of the first size and of the second size, a first cost for a first mode and a second cost for a second mode may be determined in parallel using the intermediate pixels. A final mode and a final block size may be selected for each location of the image using the first cost and the second cost for each of a respective block of the first size and a respective block of the second size associated with a corresponding location. Final pixels may be determined, and a representation of the image may be obtained based on the final pixels.

## Claims

### Claim 1

A method for compressing data of an image, the method comprising:
determining intermediate pixels;
associating each location of the image with a block of a plurality of blocks of a first size and a block a plurality of blocks of a second size;
determining, in parallel, using the intermediate pixels, for each block of the first size and for each block of the second size, a first cost for a first mode and a second cost for a second mode;
selecting, for each location of the image, i) a final mode from at least the first mode and the second mode and ii) a final block size from at least the first size and the second size, using the first cost and the second cost for each of a respective block of the first size and a respective block of the second size associated with a corresponding location;
determining final pixels based on the selected final mode and the selected final block size for each location of the image; and
obtaining, based on the final pixels, a representation of the image.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining the intermediate pixels comprises:
partitioning the image into a plurality of blocks of a third size;
processing, sequentially, each block of the third size, to determine a third cost for the first mode and a fourth cost for the second mode, wherein determining the third cost and the fourth cost uses a set of intermediate pixels obtained for a previously processed block;
selecting, for each block of the third size, an intermediate mode based on a comparison of the third cost for the first mode and the fourth cost for the second mode; and
obtaining, using the selected intermediate mode, a set of intermediate pixels for a corresponding block of the third size.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining the intermediate pixels comprises:
partitioning the image into a plurality of blocks of a third size; and
selecting, for each of the blocks of the third size, a corresponding block of source pixels from the image.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining the first cost for the first mode and the second cost for the second mode comprises, for each of the first mode and the second mode:
generating a temporary block prediction based on a respective mode and a set of intermediate pixels; and
calculating a cost of the respective mode based on a comparison of the temporary block prediction and a corresponding block of source pixels of the image.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining the final pixels comprises, for each location of the image:
generating an initial predicted block based on the selected final mode, the selected final block size, and a set of previously determined final pixels associated with one or more neighbor locations; and
deriving a final pixel block by transforming the initial predicted block using a compression algorithm.

### Claim 6 (depends on 5)

The method of claim 5, wherein the compression algorithm is based on one of AV1, HEVC, VVC, or VP9 encoding standards.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first mode is one of a plurality of intra-prediction modes and the second mode is an inter-prediction mode.

### Claim 8 (depends on 7)

The method of claim 7, wherein the inter-prediction mode approximates:
the respective block of the first size with a first reference block of the first size in a first reference image; and
the respective block of the second size with a second reference block of the second size in a second reference image.

### Claim 9

A system comprising:
a memory; and
a processor, coupled to the memory, to perform operations comprising:
determining intermediate pixels for an image;
associating each location of the image with a block of a plurality of blocks of a first size and a block a plurality of blocks of a second size;
determining, in parallel, using the intermediate pixels, for each block of the first size and for each block of the second size, a first cost for a first mode and a second cost for a second mode;
selecting, for each location of the image, i) a final mode from at least the first mode and the second mode and ii) a final block size from at least the first size and the second size, using the first cost and the second cost for each of a respective block of the first size and a respective block of the second size associated with a corresponding location;
determining final pixels based on the selected final mode and the selected final block size for each location of the image; and
obtaining, based on the final pixels, a representation of the image.

### Claim 10 (depends on 9)

The system of claim 9, wherein determining the intermediate pixels comprises:
partitioning the image into a plurality of blocks of a third size;
processing, sequentially, each block of the third size, to determine a third cost for the first mode and a fourth cost for the second mode, wherein determining the third cost and the fourth cost uses a set of intermediate pixels obtained for a previously processed block;
selecting, for each block of the third size, an intermediate mode based on a comparison of the third cost for the first mode and the fourth cost for the second mode; and
obtaining, using the selected intermediate mode, a set of intermediate pixels for a corresponding block of the third size.

### Claim 11 (depends on 9)

The system of claim 9, wherein determining the first cost for the first mode and the second cost for the second mode comprises, for each of the first mode and the second mode:
generating a temporary block prediction based on a respective mode and a set of intermediate pixels; and
calculating a cost of the respective mode based on a comparison of the temporary block prediction and a corresponding block of source pixels of the image.

### Claim 12 (depends on 9)

The system of claim 9, wherein determining the final pixels comprises, for each location of the image:
generating an initial predicted block based on the selected final mode, the selected final block size, and a set of previously determined final pixels associated with one or more neighbor locations; and
deriving a final pixel block by transforming the initial predicted block using a compression algorithm.

### Claim 13 (depends on 12)

The system of claim 12, wherein the compression algorithm is based on one of AV1, HEVC, VVC, or VP9 encoding standards.

### Claim 14 (depends on 9)

The system of claim 9, wherein the first mode is one of a plurality of intra-prediction modes and the second mode is an inter-prediction mode.

### Claim 15

A non-transitory computer-readable medium comprising instructions, which when executed by a processor cause the processor to perform operations comprising:
determining intermediate pixels for an image;
associating each location of the image with a block of a plurality of blocks of a first size and a block a plurality of blocks of a second size;
determining, in parallel, using the intermediate pixels, for each block of the first size and for each block of the second size, a first cost for a first mode and a second cost for a second mode;
selecting, for each location of the image, i) a final mode from at least the first mode and the second mode and ii) a final block size from at least the first size and the second size, using the first cost and the second cost for each of a respective block of the first size and a respective block of the second size associated with a corresponding location;
determining final pixels based on the selected final mode and the selected final block size for each location of the image; and
obtaining, based on the final pixels, a representation of the image.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein determining the intermediate pixels comprises:
partitioning the image into a plurality of blocks of a third size;
processing, sequentially, each block of the third size, to determine a third cost for the first mode and a fourth cost for the second mode, wherein determining the third cost and the fourth cost uses a set of intermediate pixels obtained for a previously processed block;
selecting, for each block of the third size, an intermediate mode based on a comparison of the third cost for the first mode and the fourth cost for the second mode; and
obtaining, using the selected intermediate mode, a set of intermediate pixels for a corresponding block of the third size.

### Claim 17 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein determining the first cost for the first mode and the second cost for the second mode comprises, for each of the first mode and the second mode:
generating a temporary block prediction based on a respective mode and a set of intermediate pixels; and
calculating a cost of the respective mode based on a comparison of the temporary block prediction and a corresponding block of source pixels of the image.

### Claim 18 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein determining the final pixels comprises, for each location of the image:
generating an initial predicted block based on the selected final mode, the selected final block size, and a set of previously determined final pixels associated with one or more neighbor locations; and
deriving a final pixel block by transforming the initial predicted block using a compression algorithm.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the compression algorithm is based on one of AV1, HEVC, VVC, or VP9 encoding standards.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the first mode is one of a plurality of intra-prediction modes and the second mode is an inter-prediction mode.
