# US 2011/0235713 A1 · Method for adaptively performing video decoding, and associated adaptive complexity video decoder and adaptive audio/video playback system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2011/0235713 A1
- Document ID: US20110235713A1
- Status: Published application
- Kind: A1
- Application number: 12/830,419
- Filed: 2010-07-05
- Published: 2011-09-29
- Earliest claimed: 2010-03-25
- Assignees: Mediatek Inc.
- Inventors: Fang-Yi Hsieh; Jian-Liang Lin
- Web page: https://patentgrep.com/patent/US20110235713A1

## CPC

- H04N 21/234363: {by altering the spatial resolution, e.g. for clients with a lower screen resolution}
- H04N 19/13: Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/156: Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
- H04N 19/44: Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
- H04N 19/48: using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data (motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513)
- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- H04N 19/61: in combination with predictive coding
- H04N 21/43637: {involving a wireless protocol, e.g. Bluetooth®, RF or wireless LAN [IEEE 802.11] (arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for adaptively performing video decoding includes: performing decoding complexity management based upon bit stream information of an input bit stream, in order to determine whether to reduce decoding complexity of at least one component of a plurality of components within an adaptive complexity video decoder; and selectively reducing decoding complexity of a portion of components within the adaptive complexity video decoder. An associated adaptive complexity video decoder and an associated adaptive audio/video playback system are also provided. In particular, the adaptive complexity video decoder includes a plurality of components and a decoding complexity manager. When needed, the decoding complexity manager delays audio playback of audio information.

## Claims

### Claim 1

A method for adaptively performing video decoding, comprising:
performing decoding complexity management based upon bit stream information of an input bit stream of an adaptive complexity video decoder, in order to determine whether to reduce decoding complexity of at least one component of a plurality of components within the adaptive complexity video decoder; and
selectively reducing decoding complexity of a portion of components within the adaptive complexity video decoder.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of components comprises:
a fast variable length decoding (VLD) and inverse quantization module arranged to perform fast VLD and inverse quantization on the input bit stream to generate inverse quantization results;
an inverse transform unit arranged to perform inverse transform on the inverse quantization results to generate inverse transform results;
a motion compensation module arranged to perform motion compensation according to the input bit stream and generate associated prediction outputs, wherein the motion compensation module comprises:
a temporal prediction unit arranged to perform temporal prediction to generate at least a portion of the prediction outputs; and
a spatial prediction unit arranged to perform spatial prediction to generate at least a portion of the prediction outputs;
an arithmetic unit arranged to sum up the inverse transform results and the prediction outputs to generate compensated outputs;
a reconstructed frame output unit arranged to generate a plurality of reconstructed frames according to the compensated outputs, wherein the spatial prediction unit performs spatial prediction according to a current reconstructed frame of the reconstructed frames; and
a frame storage arranged to temporarily store at least one portion of the reconstructed frames, wherein the temporal prediction unit performs temporal prediction according to the at least one portion of the reconstructed frames.

### Claim 3 (depends on 1)

The method of claim 1, wherein the plurality of components comprises:
a display buffer arranged to temporarily store a plurality of image frames;
wherein the step of performing decoding complexity management based upon the bit stream information of the input bit stream of the adaptive complexity video decoder further comprises:
performing decoding complexity management based upon a display buffer status of the display buffer.

### Claim 4 (depends on 1)

The method of claim 1, wherein the step of performing decoding complexity management based upon the bit stream information of the input bit stream of the adaptive complexity video decoder further comprises:
generating at least one control signal in order to control decoding complexity of the portion of components within the adaptive complexity video decoder.

### Claim 5 (depends on 1)

The method of claim 1, wherein the plurality of components comprises:
a display time adjustment driver, wherein under control of the display time adjustment driver, image frames temporarily stored in a display buffer are retrieved as output frames for being displayed at time points of audio playback of associated audio information, respectively.

### Claim 6 (depends on 2)

The method of claim 2, wherein the temporal prediction unit operates in accordance with a resolution of the reconstructed frames, rather than a resolution of a plurality of original frames represented by the input bit stream, in order to reduce complexity of performing temporal prediction.

### Claim 7 (depends on 2)

The method of claim 2, wherein the spatial prediction unit operates in accordance with a resolution of the reconstructed frames, rather than a resolution of a plurality of original frames represented by the input bit stream, in order to reduce complexity of performing spatial prediction; and the spatial prediction unit further estimates at least a portion of partial information that has been omitted for use of performing spatial prediction.

### Claim 8 (depends on 2)

The method of claim 2, wherein the inverse transform unit operates in accordance with a resolution of the reconstructed frames, rather than a resolution of a plurality of original frames represented by the input bit stream, in order to reduce complexity of performing the inverse transform; and the inverse transform unit omits a portion of calculations of the inverse transform.

### Claim 9 (depends on 2)

The method of claim 2, wherein the reconstructed frame output unit operates in accordance with a resolution of the reconstructed frames, rather than a resolution of a plurality of original frames represented by the input bit stream, in order to reduce complexity of generating the reconstructed frames.

### Claim 10 (depends on 2)

The method of claim 2, wherein the fast VLD and inverse quantization module operates in accordance with a resolution of the reconstructed frames, rather than a resolution of a plurality of original frames represented by the input bit stream, in order to reduce complexity of performing fast VLD and inverse quantization; and the fast VLD and inverse quantization module utilizes a lookup table comprising a main table and at least one sub-table during decoding, and a probability of utilizing the main table is greater than that of the at least one sub-table.

### Claim 11 (depends on 1)

The method of claim 1, wherein the step of performing decoding complexity management based upon the bit stream information of the input bit stream of the adaptive complexity video decoder further comprises:
performing decoding complexity management based upon at least one parameter representing processing capability of the adaptive complexity video decoder.

### Claim 12 (depends on 1)

The method of claim 1, wherein the bit stream information of the input bit stream comprises one or a combination of a bit rate, a resolution, and a frame type.

### Claim 13 (depends on 1)

The method of claim 1, wherein the bit stream information of the input bit stream comprises a derivation from one or a combination of motion vectors, residuals, and bit stream length.

### Claim 14

An adaptive complexity video decoder, comprising:
a plurality of components arranged to perform operations related to video decoding; and
a decoding complexity manager arranged to perform decoding complexity management based upon bit stream information of an input bit stream, in order to determine whether to reduce decoding complexity of at least one component within the adaptive complexity video decoder;
wherein the decoding complexity manager selectively reduces decoding complexity of a portion of components within the adaptive complexity video decoder.

### Claim 15 (depends on 14)

The adaptive complexity video decoder of claim 14, wherein the plurality of components comprises:
a fast variable length decoding (VLD) and inverse quantization module arranged to perform fast VLD and inverse quantization on the input bit stream to generate inverse quantization results;
an inverse transform unit arranged to perform inverse transform on the inverse quantization results to generate inverse transform results;
a motion compensation module arranged to perform motion compensation according to the input bit stream and generate associated prediction outputs, wherein the motion compensation module comprises:
a temporal prediction unit arranged to perform temporal prediction to generate at least a portion of the prediction outputs; and
a spatial prediction unit arranged to perform spatial prediction to generate at least a portion of the prediction outputs;
an arithmetic unit arranged to sum up the inverse transform results and the prediction outputs to generate compensated outputs;
a reconstructed frame output unit arranged to generate a plurality of reconstructed frames according to the compensated outputs, wherein the spatial prediction unit performs spatial prediction according to a current reconstructed frame of the reconstructed frames; and
a frame storage arranged to temporarily store at least one portion of the reconstructed frames, wherein the temporal prediction unit performs temporal prediction according to the at least one portion of the reconstructed frames.

### Claim 16 (depends on 14)

The adaptive complexity video decoder of claim 14, wherein the plurality of components comprises:
a display buffer arranged to temporarily store a plurality of image frames;
wherein the decoding complexity manager is arranged to perform decoding complexity management based upon a display buffer status of the display buffer.

### Claim 17 (depends on 14)

The adaptive complexity video decoder of claim 14, wherein the decoding complexity manager generates at least one control signal according to the bit stream information of the input bit stream, in order to control decoding complexity of the portion of components respectively controlled by the control signals within the adaptive complexity video decoder.

### Claim 18 (depends on 14)

The adaptive complexity video decoder of claim 14, wherein the plurality of components comprises:
a display time adjustment driver, wherein under control of the display time adjustment driver, image frames temporarily stored in a display buffer are retrieved as output frames for being displayed at time points of audio playback of associated audio information, respectively.

### Claim 19 (depends on 15)

The adaptive complexity video decoder of claim 15, wherein the temporal prediction unit operates in accordance with a resolution of the reconstructed frames, rather than a resolution of a plurality of original frames represented by the input bit stream, in order to reduce complexity of performing temporal prediction.

### Claim 20 (depends on 15)

The adaptive complexity video decoder of claim 15, wherein the spatial prediction unit operates in accordance with a resolution of the reconstructed frames, rather than a resolution of a plurality of original frames represented by the input bit stream, in order to reduce complexity of performing spatial prediction; and the spatial prediction unit further estimates at least a portion of partial information that has been omitted for use of performing spatial prediction.

### Claim 21 (depends on 15)

The adaptive complexity video decoder of claim 15, wherein the inverse transform unit operates in accordance with a resolution of the reconstructed frames, rather than a resolution of a plurality of original frames represented by the input bit stream, in order to reduce complexity of performing the inverse transform; and the inverse transform unit omits a portion of calculations of the inverse transform.

### Claim 22 (depends on 15)

The adaptive complexity video decoder of claim 15, wherein the reconstructed frame output unit operates in accordance with a resolution of the reconstructed frames, rather than a resolution of a plurality of original frames represented by the input bit stream, in order to reduce complexity of generating the reconstructed frames.

### Claim 23 (depends on 15)

The adaptive complexity video decoder of claim 15, wherein the fast VLD and inverse quantization module operates in accordance with a resolution of the reconstructed frames, rather than a resolution of a plurality of original frames represented by the input bit stream, in order to reduce complexity of performing fast VLD and inverse quantization; and the fast VLD and inverse quantization module utilizes a lookup table comprising a main table and at least one sub-table during decoding, and a probability of utilizing the main table is greater than that of the at least one sub-table.

### Claim 24 (depends on 14)

The adaptive complexity video decoder of claim 14, wherein the decoding complexity manager is arranged to perform decoding complexity management based upon at least one parameter representing processing capability of the adaptive complexity video decoder.

### Claim 25 (depends on 14)

The adaptive complexity video decoder of claim 14, wherein the bit stream information of the input bit stream comprises one or a combination of a bit rate, a resolution, and a frame type.

### Claim 26 (depends on 14)

The adaptive complexity video decoder of claim 14, wherein the bit stream information of the input bit stream comprises a derivation from one or a combination of motion vectors, residuals, and bit stream length.

### Claim 27 (depends on 14)

The adaptive complexity video decoder of claim 14, wherein the decoding complexity manager delays audio playback of audio information.

### Claim 28

An adaptive audio/video playback system, comprising:
an audio decoder arranged to perform audio decoding to generate audio information; and
a video decoder arranged to perform video decoding, wherein the video decoder comprises:
a decoding manager, wherein the decoding manager delays audio playback of the audio information according to bit stream information of an input bit stream.

### Claim 29 (depends on 28)

The adaptive audio/video playback system of claim 28, wherein the decoding manager of the video decoder determines delaying of audio playback further according to a parameter representing processing capability or display buffer status.
