# US 2015/0103909 A1 · Multi-threaded video encoder

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0103909 A1
- Document ID: US20150103909A1
- Status: Published application
- Kind: A1
- Application number: 14/322,350
- Filed: 2014-07-02
- Published: 2015-04-16
- Earliest claimed: 2013-10-14
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Umesh Kumar Pandey; Xinping Zhang; Aditya Bhuvanagiri; Kim-Chyan Gan; Santhosh Kumar Gunna; Adithya Prakash; Aravind Korlepara; Jayant Ingale; Arjun Sitaram; Revathy Shunmugam
- Web page: https://patentgrep.com/patent/US20150103909A1

## CPC

- H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
- H04N 19/109: among a plurality of temporal predictive coding modes
- H04N 19/11: among a plurality of spatial predictive coding modes
- H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567)
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/19: using optimisation based on Lagrange multipliers
- H04N 19/436: using parallelised computational arrangements
- H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109)
- H04N 19/593: involving spatial prediction techniques
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The techniques of this disclosure relate to video encoding and include using an inter mode determination for neighboring blocks, rather than the final prediction mode determination for the neighboring block, when determining an inter mode for a current block. In this way, inter mode and intra mode estimation may be separated and performed in different stages of a multi-threaded parallel video encoding implementation. In addition, this disclosure also proposes generating sub-pixel values in a third stage of the multi-threaded parallel video encoding implementation at a frame level, rather than for each macroblock during inter mode estimation process for that macroblock.

## Claims

### Claim 1

A method of encoding video data, the method comprising:
determining an inter-prediction mode for a current macroblock of a frame of video data based on a neighbor motion vector predictor and a neighbor inter-prediction mode from one or more neighboring blocks, wherein the inter-prediction mode for the current macroblock is determined without considering a neighbor final prediction mode determined for the one or more neighboring blocks;
determining an intra prediction mode for the current macroblock;
determining a final prediction mode for the current macroblock from one of the determined inter-prediction mode and the determined intra prediction mode; and
performing a prediction process on the current macroblock using the final prediction mode.

### Claim 2 (depends on 1)

The method of claim 1, wherein the determined inter-prediction mode is a best inter-prediction mode identified by rate-distortion optimization process, and wherein the determined intra prediction mode is a best intra prediction mode identified by the rate-distortion optimization process.

### Claim 3 (depends on 1)

The method of claim 1, the determining the inter-prediction mode comprising determining the inter-prediction mode for all macroblocks in the frame of video data in a first processing stage, and
the determining the intra prediction mode comprising determining the intra prediction mode for all macroblocks in the frame of video data in a second processing stage, wherein the second processing stage occurs after the first processing stage.

### Claim 4 (depends on 3)

The method of claim 3, the determining the final prediction mode comprising determining the final prediction mode for all macroblocks in the frame of video data in the second processing stage, and
the performing the prediction process comprising performing the prediction process for all macroblocks in the frame of video data in the second processing stage.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
performing transformation and quantization, inverse transformation, inverse quantization, and boundary strength calculation for all macroblocks in the frame of video data in the second stage of processing.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
performing deblocking and sub-pel plane generation on reconstructed blocks of the frame of video data in a third stage of processing, wherein the third stage of processing occurs after the second stage of processing.

### Claim 7 (depends on 6)

The method of claim 6, wherein the first processing stage, the second processing stage, and the third processing stage use a batch-server mode of processing.

### Claim 8 (depends on 7)

The method of claim 7, wherein the batch-server mode of processing for the first processing stage, the second processing stage, and the third processing stage uses n software threads.

### Claim 9 (depends on 8)

The method of claim 8, wherein n is 3.

### Claim 10 (depends on 8)

The method of claim 8, wherein the n software threads use k digital signal processor threads, wherein n is greater than or equal to k.

### Claim 11

An apparatus configured to encode video data, the apparatus comprising:
a video memory configured to store video data; and
a video encoder operatively coupled to the video memory, the video encoder configured to:
determine an inter-prediction mode for a current macroblock of a frame of video data based on a neighbor motion vector predictor and a neighbor inter-prediction mode from one or more neighboring blocks, wherein the inter-prediction mode for the current macroblock is determined without considering a neighbor final prediction mode determined for the one or more neighboring blocks;
determine an intra prediction mode for the current macroblock;
determine a final prediction mode for the current macroblock from one of the determined inter-prediction mode and the determined intra prediction mode; and
perform a prediction process on the current macroblock using the final prediction mode.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the determined inter-prediction mode is a best inter-prediction mode identified by rate-distortion optimization process, and wherein the determined intra prediction mode is a best intra prediction mode identified by the rate-distortion optimization process.

### Claim 13 (depends on 11)

The apparatus of claim 11, wherein the video encoder is further configured to determine the inter-prediction mode for all macroblocks in the frame of video data in a first processing stage, and
determine the intra prediction mode for all macroblocks in the frame of video data in a second processing stage, wherein the second processing stage occurs after the first processing stage.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the video encoder is further configured to determine the final prediction mode for all macroblocks in the frame of video data in the second processing stage, and
perform the prediction process for all macroblocks in the frame of video data in the second processing stage.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the video encoder is further configured to:
perform transformation and quantization, inverse transformation, inverse quantization, and boundary strength calculation for all macroblocks in the frame of video data in the second stage of processing.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the video encoder is further configured to:
perform deblocking and sub-pel plane generation on reconstructed blocks of the frame of video data in a third stage of processing, wherein the third stage of processing occurs after the second stage of processing.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the video encoder is further configured to perform the first processing stage, the second processing stage, and the third processing stage using a batch-server mode of processing.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the video encoder is further configured to use the batch-server mode of processing for the first processing stage, the second processing stage, and the third processing stage by using n software threads.

### Claim 19 (depends on 18)

The apparatus of claim 18, wherein n is 3.

### Claim 20 (depends on 18)

The apparatus of claim 18, wherein the n software threads use k digital signal processor threads, wherein n is greater than or equal to k.

### Claim 21

An apparatus configured to encode video data, the apparatus comprising:
means for determining an inter-prediction mode for a current macroblock of a frame of video data based on a neighbor motion vector predictor and a neighbor inter-prediction mode from one or more neighboring blocks, wherein the inter-prediction mode for the current macroblock is determined without considering a neighbor final prediction mode determined for the one or more neighboring blocks;
means for determining an intra prediction mode for the current macroblock;
means for determining a final prediction mode for the current macroblock from one of the determined inter-prediction mode and the determined intra prediction mode; and
means for performing a prediction process on the current macroblock using the final prediction mode.

### Claim 22 (depends on 21)

The apparatus of claim 21, the means for determining the inter-prediction mode comprising means for determining the inter-prediction mode for all macroblocks in the frame of video data in a first processing stage, and
the means for determining the intra prediction mode comprising means for determining the intra prediction mode for all macroblocks in the frame of video data in a second processing stage, wherein the second processing stage occurs after the first processing stage.

### Claim 23 (depends on 22)

The apparatus of claim 22, the means for determining the final prediction mode comprising means for determining the final prediction mode for all macroblocks in the frame of video data in the second processing stage, and
the means for performing the prediction process comprising means for performing the prediction process for all macroblocks in the frame of video data in the second processing stage.

### Claim 24 (depends on 23)

The apparatus of claim 23, further comprising:
means for performing deblocking and sub-pel plane generation on reconstructed blocks of the frame of video data in a third stage of processing, wherein the third stage of processing occurs after the second stage of processing.

### Claim 25 (depends on 24)

The apparatus of claim 24, wherein the first processing stage, the second processing stage, and the third processing stage use a batch-server mode of processing.

### Claim 26

A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to encode video data to:
determine an inter-prediction mode for a current macroblock of a frame of video data based on a neighbor motion vector predictor and a neighbor inter-prediction mode from one or more neighboring blocks, wherein the inter-prediction mode for the current macroblock is determined without considering a neighbor final prediction mode determined for the one or more neighboring blocks;
determine an intra prediction mode for the current macroblock;
determine a final prediction mode for the current macroblock from one of the determined inter-prediction mode and the determined intra prediction mode; and
perform a prediction process on the current macroblock using the final prediction mode.

### Claim 27 (depends on 26)

The non-transitory computer-readable storage medium of claim 26, wherein the instructions further cause the one or more processors to determine the inter-prediction mode for all macroblocks in the frame of video data in a first processing stage, and
determine the intra prediction mode for all macroblocks in the frame of video data in a second processing stage, wherein the second processing stage occurs after the first processing stage.

### Claim 28 (depends on 27)

The non-transitory computer-readable storage medium of claim 27, wherein the instructions further cause the one or more processors to determine the final prediction mode for all macroblocks in the frame of video data in the second processing stage, and
perform the prediction process for all macroblocks in the frame of video data in the second processing stage.

### Claim 29 (depends on 28)

The non-transitory computer-readable storage medium of claim 28, wherein the instructions further cause the one or more processors to:
perform deblocking and sub-pel plane generation on reconstructed blocks of the frame of video data in a third stage of processing, wherein the third stage of processing occurs after the second stage of processing.

### Claim 30 (depends on 29)

The non-transitory computer-readable storage medium of claim 29, wherein the instructions further cause the one or more processors to perform the first processing stage, the second processing stage, and the third processing stage using a batch-server mode of processing.
