# US 2013/0156101 A1 · Hardware-accelerated decoding of scalable video bitstreams

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0156101 A1
- Document ID: US20130156101A1
- Status: Published application
- Kind: A1
- Application number: 13/329,093
- Filed: 2011-12-16
- Published: 2013-06-20
- Earliest claimed: 2011-12-16
- Assignees: Microsoft Corporation
- Inventors: Mei-Hsuan Lu; Yongjun Wu; Ming-Chieh Lee; Gary J. Sullivan
- Web page: https://patentgrep.com/patent/US20130156101A1

## CPC

- H04N 19/42: characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence)
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/44: Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
- H04N 19/423: characterised by memory arrangements (H04N19/433 takes precedence)
- H04N 19/436: using parallelised computational arrangements
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In various respects, hardware-accelerated decoding is adapted for decoding of video that has been encoded using scalable video coding. For example, for a given picture to be decoded, a host decoder determines whether a corresponding base picture will be stored for use as a reference picture. If so, the host decoder directs decoding with an accelerator such that the some of the same decoding operations can be used for the given picture and the reference base picture. Or, as another example, the host decoder groups encoded data associated with a given layer representation in buffers. The host decoder provides the encoded data for the layer to the accelerator. The host decoder repeats the process layer-after-layer in the order that layers appear in the bitstream, according to a defined call pattern for an acceleration interface, which helps the accelerator determine the layers with which buffers are associated.

## Claims

### Claim 1

A tangible computer-readable medium storing computer-executable instructions for causing a computing system to perform a method comprising:
receiving at least part of a bitstream for video data having been encoded using scalable video coding, the bitstream including encoded data for a given picture to be decoded for output, the given picture having a reference base picture to be stored for use as a reference picture; and
with a host decoder, calling an acceleration interface to direct decoding of the given picture and decoding of the reference base picture by an accelerator, including interleaving at least some calls for the decoding of the reference base picture with at least some calls for the decoding of the given picture.

### Claim 2 (depends on 1)

The computer-readable medium of claim 1 wherein the interleaving facilitates recognition by the accelerator of opportunities to share operations between the decoding of the reference base picture and the decoding of the given picture.

### Claim 3 (depends on 1)

The computer-readable medium of claim 1 wherein the calling includes:
calling a first routine to signal initiation of the decoding of the reference base picture;
calling the first routine to signal initiation of the decoding of the given picture;
calling a second routine one or more times to signal that at least some of the encoded data for the given picture is accessible to the accelerator;
calling a third routine to signal that encoded data to be used in the decoding of the reference base picture has been provided; and
calling the third routine to signal that encoded data to be used in the decoding of the given picture has been provided.

### Claim 4 (depends on 3)

The computer-readable medium of claim 3 wherein the calls for the reference base picture include a first index to a first output surface, and wherein the calls for the given picture include a second index to a second output surface.

### Claim 5 (depends on 3)

The computer-readable medium of claim 3 wherein the bitstream further includes encoded data for a next picture to be decoded for output, the next picture lacking a base picture to be stored for use as a reference picture, the method further comprising:
calling the first routine to signal initiation of the decoding of the next picture;
calling the second routine one or more times to signal that the encoded data for the next picture is accessible to the accelerator; and
calling the third routine to signal that encoded data to be used in the decoding of the next picture has been provided.

### Claim 6 (depends on 1)

The computer-readable medium of claim 1 wherein the method further comprises:
deciding whether to initiate concurrent decoding of the given picture and the reference base picture;
if so, performing the calling the acceleration interface to direct the decoding of the given picture and the decoding of the reference base picture by the accelerator; and
otherwise, calling the acceleration interface to direct the decoding of the given picture by the accelerator.

### Claim 7 (depends on 1)

The computer-readable medium of claim 1 wherein the encoded data for the given picture has multiple layers in coded order in the bitstream, and wherein the calling the acceleration interface includes, in the coded order, for each of the multiple layers:
storing encoded data for that layer in multiple buffers accessible to the accelerator; and
calling a routine to signal that the encoded data for that layer is accessible.

### Claim 8 (depends on 7)

The computer-readable medium of claim 7 wherein the multiple layers include a target layer associated with the given picture and a reference layer associated with the reference base picture, and wherein the bitstream is a H.264/SVC bitstream.

### Claim 9 (depends on 7)

The computer-readable medium of claim 7 wherein the multiple buffers include a picture parameters buffer, a slice control buffer and one or more bitstream data buffers.

### Claim 10

A method of performing video decoding operations for a picture having encoded data in multiple layers in coded order in a bitstream, the method comprising:
with an accelerator, for each of the multiple layers, receiving encoded data for the layer from a host decoder, wherein the encoded data for the layer is grouped in multiple buffers accessible to the accelerator, and wherein the encoded data for the picture is received in the multiple buffers layer-after-layer in the coded order; and
with the accelerator, performing video decoding operations using the encoded data for the picture.

### Claim 11 (depends on 10)

The method of claim 10 further comprising:
before the receiving the encoded data, receiving a first signal to initiate decoding of the picture;
after the receiving the encoded data, receiving a second signal to indicate that the encoded data for the picture has been provided by the host decoder, the accelerator initiating the performance of the video decoding operations in reaction to the second signal.

### Claim 12 (depends on 10)

The method of claim 10 further comprising:
storing reconstructed values for the picture in an output surface.

### Claim 13 (depends on 10)

The method of claim 10 wherein the multiple layers include a target layer and a reference layer.

### Claim 14 (depends on 13)

The method of claim 13 wherein the multiple layers further include one or more other layers used for inter-layer prediction.

### Claim 15 (depends on 10)

The method of claim 10 wherein the multiple buffers include a picture parameters buffer, a slice control buffer and one or more bitstream data buffers.

### Claim 16 (depends on 15)

The method of claim 15 wherein, for each of the multiple layers, the picture parameters buffer stores a list of buffer surfaces and a list of multiple values, each of the multiple values indicating whether a corresponding reference picture in the list of buffer surfaces is a reference base picture or decoded picture.

### Claim 17 (depends on 15)

The method of claim 15 wherein, for each of the multiple layers, the slice control buffer stores a slice header including multiple slice control parameters for scalable video decoding.

### Claim 18

In a computing system that implements a host decoder, a method comprising:
receiving units of encoded data for video having been encoded using scalable video coding with multiple layers;
with the host decoder, detecting loss of units of encoded data used for reconstruction of a target layer of the multiple layers;
with the host decoder, changing the target layer for at least some of the received units of encoded data; and
providing the units of encoded data with the changed target layer to an accelerator.

### Claim 19 (depends on 18)

The method of claim 18 wherein the units of encoded data are units of slice data organized according to the H.264/SVC standard, wherein the lost units are for the target layer and/or for reference in inter-layer prediction.

### Claim 20 (depends on 18)

The method of claim 18 wherein the changing comprises altering layer type syntax elements in the units of encoded data.